2024-03-30T00:04:50.927 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-30T00:04:50.927 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} --name rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi --owner scheduled_rishabh@teuthology --verbose --archive /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224 -- /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224/orig.config.yaml 2024-03-30T00:04:50.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-30T00:04:51.094 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224 branch: wip-rishabh-testing-20240327.051042-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} email: null first_in_suite: false job_id: '7629224' kernel: flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-testing-20240327.051042-reef ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f kclient: client.0: mount_subvol_num: 1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240327.051042-reef sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8052 sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f sleep_before_teardown: 0 subset: 497/512 suite: fs suite_branch: wip-rishabh-testing-20240327.051042-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIoSjW3CRxe29HNsWJoUM5ocmpfMDfRmVCCK08neLVO2axhrGNYo0sBlmCg3K89B+9r0wgIXYaUBhxXcjs2K9ao= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDjqJ1yuF+P8cmOP1ktVk8gvV+4ub9nveBx1TeiCA1+pjAR2qF9IPIo7qExVfYfgjINT5fjScaW1x0hAX6I6KHM= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBObwWGzVz+IdEJn2UBpojh+5QUNYR2pvwgk7/PzEjR44QcnwvA7Vm2rn3m0tcg3TJF587MZ6TCyINXYgp6ySLfE= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-29_18:05:24 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-30T00:04:51.094 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa; will attempt to use it 2024-03-30T00:04:51.095 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks 2024-03-30T00:04:51.095 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-30T00:04:51.096 INFO:teuthology.task.internal:Checking packages... 2024-03-30T00:04:51.122 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '470100a4ccbcf7769052bf09a5413881ccf1b97f' 2024-03-30T00:04:51.122 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-30T00:04:51.122 INFO:teuthology.packaging:ref: None 2024-03-30T00:04:51.122 INFO:teuthology.packaging:tag: None 2024-03-30T00:04:51.122 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240327.051042-reef 2024-03-30T00:04:51.122 INFO:teuthology.packaging:sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:04:51.122 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-testing-20240327.051042-reef 2024-03-30T00:04:51.270 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1018.g470100a4 2024-03-30T00:04:51.272 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-30T00:04:51.283 INFO:teuthology.task.internal:no buildpackages task found 2024-03-30T00:04:51.283 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-30T00:04:51.289 INFO:teuthology.task.internal:Saving configuration 2024-03-30T00:04:51.310 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-30T00:04:51.323 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T00:04:51.345 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 23:59:04.700389', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIoSjW3CRxe29HNsWJoUM5ocmpfMDfRmVCCK08neLVO2axhrGNYo0sBlmCg3K89B+9r0wgIXYaUBhxXcjs2K9ao='} 2024-03-30T00:04:51.362 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 23:59:04.698954', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDjqJ1yuF+P8cmOP1ktVk8gvV+4ub9nveBx1TeiCA1+pjAR2qF9IPIo7qExVfYfgjINT5fjScaW1x0hAX6I6KHM='} 2024-03-30T00:04:51.382 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 23:59:04.701442', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBObwWGzVz+IdEJn2UBpojh+5QUNYR2pvwgk7/PzEjR44QcnwvA7Vm2rn3m0tcg3TJF587MZ6TCyINXYgp6ySLfE='} 2024-03-30T00:04:51.383 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-30T00:04:51.397 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-30T00:04:51.397 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-30T00:04:51.397 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-30T00:04:51.397 INFO:teuthology.run_tasks:Running task console_log... 2024-03-30T00:04:51.513 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f664547c160>, signals=[15]) 2024-03-30T00:04:51.514 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-30T00:04:51.523 INFO:teuthology.task.internal:Opening connections... 2024-03-30T00:04:51.523 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-03-30T00:04:51.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:04:51.609 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-03-30T00:04:51.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:04:51.689 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-03-30T00:04:51.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:04:51.765 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-30T00:04:51.789 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-03-30T00:04:51.812 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-03-30T00:04:51.812 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Red Hat Enterprise Linux" 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8.6 (Ootpa)" 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:ID="rhel" 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="fedora" 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8.6" 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.redhat.com/" 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-30T00:04:51.872 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-30T00:04:51.873 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:04:51.873 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-30T00:04:51.873 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-30T00:04:51.873 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-30T00:04:51.873 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-30T00:04:51.873 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-03-30T00:04:51.899 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-03-30T00:04:51.919 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-03-30T00:04:51.920 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-03-30T00:04:51.981 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="Red Hat Enterprise Linux" 2024-03-30T00:04:51.981 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8.6 (Ootpa)" 2024-03-30T00:04:51.982 INFO:teuthology.orchestra.run.smithi142.stdout:ID="rhel" 2024-03-30T00:04:51.982 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="fedora" 2024-03-30T00:04:51.982 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8.6" 2024-03-30T00:04:51.982 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8" 2024-03-30T00:04:51.982 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-30T00:04:51.982 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-03-30T00:04:51.982 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-30T00:04:51.982 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://www.redhat.com/" 2024-03-30T00:04:51.982 INFO:teuthology.orchestra.run.smithi142.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-30T00:04:51.983 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-30T00:04:51.983 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:04:51.983 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-30T00:04:51.983 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-30T00:04:51.983 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-30T00:04:51.983 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-30T00:04:51.983 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-03-30T00:04:52.008 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-03-30T00:04:52.029 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-03-30T00:04:52.029 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux" 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.6 (Ootpa)" 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel" 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora" 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.6" 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/" 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-30T00:04:52.089 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-30T00:04:52.090 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:04:52.090 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-30T00:04:52.090 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-30T00:04:52.090 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-30T00:04:52.090 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-30T00:04:52.090 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-03-30T00:04:52.108 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-30T00:04:52.119 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-30T00:04:52.126 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-30T00:04:52.126 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-03-30T00:04:52.129 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-03-30T00:04:52.131 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-03-30T00:04:52.150 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-30T00:04:52.159 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-30T00:04:52.159 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-03-30T00:04:52.188 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-03-30T00:04:52.190 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-03-30T00:04:52.236 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-30T00:04:52.323 INFO:teuthology.run_tasks:Running task kernel... 2024-03-30T00:04:52.356 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-30T00:04:52.356 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, timeout 300 2024-03-30T00:04:52.357 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T00:04:52.357 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T00:04:52.358 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T00:04:52.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:04:52.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:04:52.378 DEBUG:teuthology.task.kernel:sha1 for mon.a is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:04:52.378 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-30T00:04:52.378 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-03-30T00:04:52.378 DEBUG:teuthology.task.kernel:sha1 for mon.b is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:04:52.378 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-30T00:04:52.379 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-03-30T00:04:52.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:04:52.380 DEBUG:teuthology.task.kernel:sha1 for mon.c is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:04:52.380 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-30T00:04:52.380 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-03-30T00:04:52.437 INFO:teuthology.orchestra.run.smithi142.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-30T00:04:52.438 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:04:52.438 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-30T00:04:52.438 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:04:52.442 INFO:teuthology.orchestra.run.smithi161.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-30T00:04:52.442 INFO:teuthology.orchestra.run.smithi099.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-30T00:04:52.443 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:04:52.443 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-30T00:04:52.443 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:04:52.444 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:04:52.444 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-30T00:04:52.444 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:04:52.592 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum upgrade -y linux-firmware 2024-03-30T00:04:52.661 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum upgrade -y linux-firmware 2024-03-30T00:04:52.669 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum upgrade -y linux-firmware 2024-03-30T00:04:54.138 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2024-03-30T00:04:54.138 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2024-03-30T00:04:54.141 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-03-30T00:04:54.141 INFO:teuthology.orchestra.run.smithi142.stdout:Unable to read consumer identity 2024-03-30T00:04:54.170 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-03-30T00:04:54.170 INFO:teuthology.orchestra.run.smithi161.stdout:Unable to read consumer identity 2024-03-30T00:04:54.248 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:04:54.248 INFO:teuthology.orchestra.run.smithi142.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-30T00:04:54.249 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:04:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:04:54.262 INFO:teuthology.orchestra.run.smithi099.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-30T00:04:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:04:54.284 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:04:54.284 INFO:teuthology.orchestra.run.smithi161.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-30T00:04:54.284 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:04:55.978 INFO:teuthology.orchestra.run.smithi142.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2024-03-30T00:04:56.042 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2024-03-30T00:04:56.082 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-03-30T00:04:57.191 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2024-03-30T00:04:57.218 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2024-03-30T00:04:57.504 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-03-30T00:05:00.824 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 413 kB/s | 24 kB 00:00 2024-03-30T00:05:00.826 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 474 kB/s | 24 kB 00:00 2024-03-30T00:05:01.121 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 463 kB/s | 24 kB 00:00 2024-03-30T00:05:01.633 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-03-30T00:05:01.634 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2024-03-30T00:05:01.635 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-03-30T00:05:01.641 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-03-30T00:05:01.643 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-03-30T00:05:01.643 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-03-30T00:05:01.760 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-30T00:05:01.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:05:01.769 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.c... 2024-03-30T00:05:01.769 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:05:01.908 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ 2024-03-30T00:05:01.908 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ --input-file=- 2024-03-30T00:05:01.911 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi099.front.sepia.ceph.com to complete... 2024-03-30T00:05:01.915 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-30T00:05:01.917 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-03-30T00:05:01.917 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-30T00:05:01.986 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ 2024-03-30T00:05:01.987 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ --input-file=- 2024-03-30T00:05:01.989 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi161.front.sepia.ceph.com to complete... 2024-03-30T00:05:02.067 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-30T00:05:02.067 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T00:05:02.222 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ 2024-03-30T00:05:02.222 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ --input-file=- 2024-03-30T00:05:02.225 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi142.front.sepia.ceph.com to complete... 2024-03-30T00:05:31.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30 00:05:31 URL:https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [401601944/401601944] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-30T00:05:31.462 INFO:teuthology.orchestra.run.smithi161.stderr:FINISHED --2024-03-30 00:05:31-- 2024-03-30T00:05:31.462 INFO:teuthology.orchestra.run.smithi161.stderr:Total wall clock time: 29s 2024-03-30T00:05:31.462 INFO:teuthology.orchestra.run.smithi161.stderr:Downloaded: 1 files, 383M in 29s (13.1 MB/s) 2024-03-30T00:05:31.465 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.c... 2024-03-30T00:05:31.465 DEBUG:teuthology.orchestra.run.smithi161:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-30T00:05:33.315 INFO:teuthology.orchestra.run.smithi161.stdout:Verifying... ######################################## 2024-03-30T00:05:33.406 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing... ######################################## 2024-03-30T00:05:33.555 INFO:teuthology.orchestra.run.smithi161.stdout:Updating / installing... 2024-03-30T00:05:34.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30 00:05:34 URL:https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [401601944/401601944] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-30T00:05:34.296 INFO:teuthology.orchestra.run.smithi099.stderr:FINISHED --2024-03-30 00:05:34-- 2024-03-30T00:05:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:Total wall clock time: 32s 2024-03-30T00:05:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:Downloaded: 1 files, 383M in 32s (12.0 MB/s) 2024-03-30T00:05:34.308 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-30T00:05:34.308 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-30T00:05:37.119 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying... ######################################## 2024-03-30T00:05:37.213 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing... ######################################## 2024-03-30T00:05:37.377 INFO:teuthology.orchestra.run.smithi099.stdout:Updating / installing... 2024-03-30T00:05:55.557 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30 00:05:55 URL:https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [401601944/401601944] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-30T00:05:55.557 INFO:teuthology.orchestra.run.smithi142.stderr:FINISHED --2024-03-30 00:05:55-- 2024-03-30T00:05:55.557 INFO:teuthology.orchestra.run.smithi142.stderr:Total wall clock time: 53s 2024-03-30T00:05:55.558 INFO:teuthology.orchestra.run.smithi142.stderr:Downloaded: 1 files, 383M in 53s (7.23 MB/s) 2024-03-30T00:05:55.562 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-30T00:05:55.563 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-30T00:05:57.403 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying... ######################################## 2024-03-30T00:05:57.477 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing... ######################################## 2024-03-30T00:05:57.632 INFO:teuthology.orchestra.run.smithi142.stdout:Updating / installing... 2024-03-30T00:06:14.286 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-30T00:06:15.715 INFO:teuthology.orchestra.run.smithi161.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-30T00:06:17.817 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-30T00:06:19.222 INFO:teuthology.orchestra.run.smithi099.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-30T00:06:38.197 INFO:teuthology.orchestra.run.smithi161.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-30T00:06:38.485 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-30T00:06:41.048 INFO:teuthology.orchestra.run.smithi142.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-30T00:06:41.118 INFO:teuthology.orchestra.run.smithi099.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-30T00:07:01.302 INFO:teuthology.orchestra.run.smithi142.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-30T00:08:50.607 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-30T00:08:50.607 DEBUG:teuthology.orchestra.run.smithi161:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-30T00:08:50.670 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-30T00:08:50.670 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-30T00:08:50.671 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-30T00:08:50.671 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-30T00:08:50.671 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-30T00:08:50.671 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-30T00:08:50.671 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-30T00:08:50.671 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-30T00:08:50.671 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-30T00:08:50.671 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-30T00:08:50.671 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-30T00:08:50.672 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-30T00:08:50.672 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-30T00:08:50.672 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-30T00:08:50.672 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-30T00:08:50.672 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-30T00:08:50.672 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-30T00:08:50.672 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-30T00:08:50.673 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-30T00:08:50.674 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-30T00:08:50.675 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-30T00:08:50.676 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-30T00:08:50.677 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-30T00:08:50.678 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-30T00:08:50.678 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-30T00:08:50.678 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-30T00:08:50.678 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-30T00:08:50.678 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-30T00:08:50.678 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-30T00:08:50.678 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-30T00:08:50.678 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-30T00:08:50.678 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-30T00:08:50.678 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-30T00:08:50.678 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-30T00:08:50.679 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-30T00:08:50.680 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-30T00:08:50.681 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-30T00:08:50.681 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-30T00:08:50.681 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-30T00:08:50.681 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-30T00:08:50.681 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-30T00:08:50.681 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-30T00:08:50.681 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-30T00:08:50.681 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-30T00:08:50.681 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-30T00:08:50.681 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-30T00:08:50.681 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-30T00:08:50.682 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-30T00:08:50.682 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-30T00:08:50.682 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-30T00:08:50.682 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-30T00:08:50.682 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-30T00:08:50.682 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-30T00:08:50.682 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-30T00:08:50.682 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-30T00:08:50.683 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-30T00:08:50.683 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-30T00:08:50.683 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-30T00:08:50.683 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-30T00:08:50.683 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-30T00:08:50.683 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-30T00:08:50.683 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-30T00:08:50.683 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-30T00:08:50.684 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-30T00:08:50.685 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-30T00:08:50.686 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-30T00:08:50.687 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-30T00:08:50.688 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-30T00:08:50.689 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-30T00:08:50.690 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-30T00:08:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-30T00:08:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-30T00:08:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-30T00:08:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-30T00:08:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-30T00:08:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-30T00:08:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-30T00:08:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-30T00:08:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-30T00:08:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-30T00:08:50.692 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-30T00:08:50.692 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-30T00:08:50.692 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-30T00:08:50.692 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-30T00:08:50.692 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-30T00:08:50.692 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-30T00:08:50.692 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-30T00:08:50.692 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-30T00:08:50.693 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-30T00:08:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-30T00:08:50.695 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-30T00:08:50.696 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-30T00:08:50.697 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-30T00:08:50.698 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-30T00:08:50.699 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-30T00:08:50.700 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-30T00:08:50.700 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-30T00:08:50.700 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-30T00:08:50.700 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-30T00:08:50.700 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-30T00:08:50.700 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-30T00:08:50.700 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-30T00:08:50.700 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-30T00:08:50.700 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-30T00:08:50.700 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-30T00:08:50.700 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-30T00:08:50.701 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-30T00:08:50.702 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-30T00:08:50.702 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-30T00:08:50.702 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-30T00:08:50.702 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-30T00:08:50.702 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-30T00:08:50.702 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-30T00:08:50.702 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-30T00:08:50.702 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-30T00:08:50.702 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-30T00:08:50.703 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-30T00:08:50.703 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-30T00:08:50.703 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-30T00:08:50.703 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-30T00:08:50.703 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-30T00:08:50.703 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-30T00:08:50.703 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-30T00:08:50.703 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-30T00:08:50.703 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-30T00:08:50.704 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-30T00:08:50.705 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-30T00:08:50.706 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-30T00:08:50.707 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-30T00:08:50.708 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-30T00:08:50.709 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-30T00:08:50.710 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-30T00:08:50.711 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-30T00:08:50.712 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-30T00:08:50.713 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-30T00:08:50.713 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-30T00:08:50.713 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-30T00:08:50.713 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-30T00:08:50.713 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-30T00:08:50.713 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-30T00:08:50.713 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-30T00:08:50.713 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-30T00:08:50.713 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-30T00:08:50.714 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-30T00:08:50.715 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-30T00:08:50.716 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-30T00:08:50.717 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-30T00:08:50.718 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-30T00:08:50.719 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-30T00:08:50.720 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-30T00:08:50.721 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-30T00:08:50.722 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-30T00:08:50.723 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-30T00:08:50.724 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-30T00:08:50.725 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-30T00:08:50.726 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-30T00:08:50.727 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-30T00:08:50.728 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-30T00:08:50.729 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-30T00:08:50.730 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-30T00:08:50.731 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-30T00:08:50.732 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-30T00:08:50.733 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-30T00:08:50.734 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-30T00:08:50.735 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-30T00:08:50.736 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-30T00:08:50.737 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-30T00:08:50.738 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-30T00:08:50.739 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-30T00:08:50.740 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-30T00:08:50.741 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-30T00:08:50.742 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-30T00:08:50.743 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-30T00:08:50.743 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-30T00:08:50.743 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-30T00:08:50.743 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-30T00:08:50.743 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-30T00:08:50.743 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-30T00:08:50.743 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-30T00:08:50.743 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-30T00:08:50.743 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-30T00:08:50.743 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-30T00:08:50.744 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-30T00:08:50.745 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-30T00:08:50.746 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-30T00:08:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-30T00:08:50.748 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-30T00:08:50.749 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-30T00:08:50.750 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-30T00:08:50.751 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-30T00:08:50.752 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-30T00:08:50.753 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-30T00:08:50.753 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-30T00:08:50.753 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-30T00:08:50.753 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-30T00:08:50.753 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-30T00:08:50.753 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-30T00:08:50.753 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-30T00:08:50.753 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-30T00:08:50.753 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-30T00:08:50.753 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-30T00:08:50.754 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-30T00:08:50.755 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-30T00:08:50.756 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-30T00:08:50.757 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-30T00:08:50.758 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-30T00:08:50.759 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-30T00:08:50.760 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-30T00:08:50.761 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-30T00:08:50.762 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-30T00:08:50.762 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-30T00:08:50.762 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-30T00:08:50.762 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-30T00:08:50.762 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-30T00:08:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-30T00:08:50.764 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-30T00:08:50.765 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-30T00:08:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-30T00:08:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-30T00:08:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-30T00:08:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-30T00:08:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-30T00:08:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-30T00:08:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-30T00:08:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-30T00:08:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-30T00:08:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-30T00:08:50.766 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-30T00:08:50.767 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-30T00:08:50.768 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-30T00:08:50.769 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-30T00:08:50.770 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-30T00:08:50.771 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-30T00:08:50.772 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-30T00:08:50.773 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-30T00:08:50.774 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-30T00:08:50.775 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-30T00:08:50.776 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-30T00:08:50.777 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-30T00:08:50.778 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-30T00:08:50.779 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-30T00:08:50.780 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-30T00:08:50.781 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-30T00:08:50.782 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-30T00:08:50.783 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-30T00:08:50.784 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-30T00:08:50.785 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-30T00:08:50.786 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-30T00:08:50.787 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-30T00:08:50.788 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-30T00:08:50.789 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-30T00:08:50.790 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-30T00:08:50.791 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-30T00:08:50.792 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-30T00:08:50.792 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-30T00:08:50.792 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-30T00:08:50.792 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-30T00:08:50.792 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-30T00:08:50.792 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-30T00:08:50.792 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-30T00:08:50.792 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-30T00:08:50.792 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-30T00:08:50.793 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-30T00:08:50.793 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-30T00:08:50.793 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-30T00:08:50.793 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-30T00:08:50.793 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-30T00:08:50.794 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-30T00:08:50.795 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-30T00:08:50.796 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-30T00:08:50.797 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-30T00:08:50.798 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-30T00:08:50.799 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-30T00:08:50.800 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-30T00:08:50.801 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-30T00:08:50.802 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-30T00:08:50.803 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-30T00:08:50.804 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-30T00:08:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-30T00:08:50.806 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-30T00:08:50.807 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-30T00:08:50.808 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-30T00:08:50.809 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-30T00:08:50.810 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-30T00:08:50.811 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-30T00:08:50.815 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-30T00:08:50.816 DEBUG:teuthology.orchestra.run.smithi161:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-30T00:08:50.840 INFO:teuthology.orchestra.run.smithi161.stdout:postinstall scriptlet (using /bin/sh): 2024-03-30T00:08:50.840 INFO:teuthology.orchestra.run.smithi161.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-30T00:08:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-30T00:08:50.840 INFO:teuthology.orchestra.run.smithi161.stdout:fi 2024-03-30T00:08:50.840 INFO:teuthology.orchestra.run.smithi161.stdout:for file in vmlinuz System.map config; do 2024-03-30T00:08:50.840 INFO:teuthology.orchestra.run.smithi161.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout: fi 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout:done 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout:fi 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-30T00:08:50.841 INFO:teuthology.orchestra.run.smithi161.stdout:fi 2024-03-30T00:08:50.842 DEBUG:teuthology.orchestra.run.smithi161:> sudo rpm -qi grub2-tools 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:Name : grub2-tools 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:Epoch : 1 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:Version : 2.02 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:Release : 123.el8 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: x86_64 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:Group : System Environment/Base 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:Size : 9265450 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:License : GPLv3+ 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-03-30T00:08:50.914 INFO:teuthology.orchestra.run.smithi161.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:Relocations : (not relocatable) 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:Packager : Red Hat, Inc. 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:Vendor : Red Hat, Inc. 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:Summary : Support tools for GRUB. 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:Description : 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:hardware devices. 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:08:50.915 INFO:teuthology.orchestra.run.smithi161.stdout:This subpackage provides tools for support of all platforms. 2024-03-30T00:08:50.916 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-30T00:08:50.916 INFO:teuthology.task.kernel:Updating grub on smithi161 to boot 6.8.0-g1df0c7e8a235 2024-03-30T00:08:50.917 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-30T00:08:51.793 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2024-03-30T00:08:53.086 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-30T00:08:53.087 DEBUG:teuthology.orchestra.run.smithi099:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-30T00:08:53.157 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-30T00:08:53.157 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-30T00:08:53.157 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-30T00:08:53.157 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-30T00:08:53.157 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-30T00:08:53.157 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-30T00:08:53.158 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-30T00:08:53.158 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-30T00:08:53.158 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-30T00:08:53.158 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-30T00:08:53.158 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-30T00:08:53.158 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-30T00:08:53.159 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-30T00:08:53.159 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-30T00:08:53.159 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-30T00:08:53.159 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-30T00:08:53.159 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-30T00:08:53.159 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-30T00:08:53.160 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-30T00:08:53.160 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-30T00:08:53.160 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-30T00:08:53.160 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-30T00:08:53.160 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-30T00:08:53.160 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-30T00:08:53.161 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-30T00:08:53.161 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-30T00:08:53.161 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-30T00:08:53.161 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-30T00:08:53.161 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-30T00:08:53.161 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-30T00:08:53.161 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-30T00:08:53.162 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-30T00:08:53.162 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-30T00:08:53.162 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-30T00:08:53.162 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-30T00:08:53.162 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-30T00:08:53.162 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-30T00:08:53.163 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-30T00:08:53.163 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-30T00:08:53.163 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-30T00:08:53.163 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-30T00:08:53.163 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-30T00:08:53.163 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-30T00:08:53.164 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-30T00:08:53.164 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-30T00:08:53.164 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-30T00:08:53.164 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-30T00:08:53.164 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-30T00:08:53.164 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-30T00:08:53.165 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-30T00:08:53.165 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-30T00:08:53.165 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-30T00:08:53.165 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-30T00:08:53.165 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-30T00:08:53.165 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-30T00:08:53.166 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-30T00:08:53.166 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-30T00:08:53.166 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-30T00:08:53.166 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-30T00:08:53.166 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-30T00:08:53.166 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-30T00:08:53.167 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-30T00:08:53.167 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-30T00:08:53.167 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-30T00:08:53.167 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-30T00:08:53.167 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-30T00:08:53.167 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-30T00:08:53.167 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-30T00:08:53.168 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-30T00:08:53.168 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-30T00:08:53.168 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-30T00:08:53.168 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-30T00:08:53.168 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-30T00:08:53.168 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-30T00:08:53.168 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-30T00:08:53.169 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-30T00:08:53.169 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-30T00:08:53.169 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-30T00:08:53.169 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-30T00:08:53.169 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-30T00:08:53.169 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-30T00:08:53.170 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-30T00:08:53.170 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-30T00:08:53.170 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-30T00:08:53.170 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-30T00:08:53.170 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-30T00:08:53.170 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-30T00:08:53.171 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-30T00:08:53.171 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-30T00:08:53.171 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-30T00:08:53.171 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-30T00:08:53.171 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-30T00:08:53.171 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-30T00:08:53.171 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-30T00:08:53.172 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-30T00:08:53.172 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-30T00:08:53.172 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-30T00:08:53.172 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-30T00:08:53.172 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-30T00:08:53.172 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-30T00:08:53.173 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-30T00:08:53.173 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-30T00:08:53.173 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-30T00:08:53.173 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-30T00:08:53.173 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-30T00:08:53.173 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-30T00:08:53.173 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-30T00:08:53.174 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-30T00:08:53.174 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-30T00:08:53.174 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-30T00:08:53.174 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-30T00:08:53.174 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-30T00:08:53.174 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-30T00:08:53.175 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-30T00:08:53.175 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-30T00:08:53.175 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-30T00:08:53.175 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-30T00:08:53.175 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-30T00:08:53.175 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-30T00:08:53.175 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-30T00:08:53.176 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-30T00:08:53.176 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-30T00:08:53.176 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-30T00:08:53.176 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-30T00:08:53.176 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-30T00:08:53.176 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-30T00:08:53.176 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-30T00:08:53.176 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-30T00:08:53.177 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-30T00:08:53.177 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-30T00:08:53.177 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-30T00:08:53.177 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-30T00:08:53.177 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-30T00:08:53.177 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-30T00:08:53.177 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-30T00:08:53.177 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-30T00:08:53.177 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-30T00:08:53.178 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-30T00:08:53.178 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-30T00:08:53.178 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-30T00:08:53.178 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-30T00:08:53.178 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-30T00:08:53.178 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-30T00:08:53.178 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-30T00:08:53.178 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-30T00:08:53.179 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-30T00:08:53.179 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-30T00:08:53.179 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-30T00:08:53.179 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-30T00:08:53.179 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-30T00:08:53.179 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-30T00:08:53.179 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-30T00:08:53.180 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-30T00:08:53.180 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-30T00:08:53.180 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-30T00:08:53.180 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-30T00:08:53.180 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-30T00:08:53.180 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-30T00:08:53.181 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-30T00:08:53.181 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-30T00:08:53.181 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-30T00:08:53.181 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-30T00:08:53.181 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-30T00:08:53.181 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-30T00:08:53.181 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-30T00:08:53.182 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-30T00:08:53.182 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-30T00:08:53.182 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-30T00:08:53.182 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-30T00:08:53.182 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-30T00:08:53.182 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-30T00:08:53.183 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-30T00:08:53.183 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-30T00:08:53.183 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-30T00:08:53.183 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-30T00:08:53.183 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-30T00:08:53.183 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-30T00:08:53.183 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-30T00:08:53.184 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-30T00:08:53.184 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-30T00:08:53.184 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-30T00:08:53.184 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-30T00:08:53.184 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-30T00:08:53.184 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-30T00:08:53.184 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-30T00:08:53.185 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-30T00:08:53.185 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-30T00:08:53.185 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-30T00:08:53.185 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-30T00:08:53.185 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-30T00:08:53.185 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-30T00:08:53.185 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-30T00:08:53.185 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-30T00:08:53.185 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-30T00:08:53.186 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-30T00:08:53.186 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-30T00:08:53.186 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-30T00:08:53.186 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-30T00:08:53.186 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-30T00:08:53.187 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-30T00:08:53.187 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-30T00:08:53.187 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-30T00:08:53.187 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-30T00:08:53.187 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-30T00:08:53.187 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-30T00:08:53.187 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-30T00:08:53.187 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-30T00:08:53.187 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-30T00:08:53.188 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-30T00:08:53.188 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-30T00:08:53.188 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-30T00:08:53.188 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-30T00:08:53.188 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-30T00:08:53.188 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-30T00:08:53.188 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-30T00:08:53.188 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-30T00:08:53.188 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-30T00:08:53.189 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-30T00:08:53.189 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-30T00:08:53.189 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-30T00:08:53.189 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-30T00:08:53.189 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-30T00:08:53.189 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-30T00:08:53.189 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-30T00:08:53.189 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-30T00:08:53.189 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-30T00:08:53.190 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-30T00:08:53.190 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-30T00:08:53.190 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-30T00:08:53.190 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-30T00:08:53.190 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-30T00:08:53.190 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-30T00:08:53.190 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-30T00:08:53.190 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-30T00:08:53.190 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-30T00:08:53.191 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-30T00:08:53.191 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-30T00:08:53.191 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-30T00:08:53.191 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-30T00:08:53.191 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-30T00:08:53.191 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-30T00:08:53.191 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-30T00:08:53.191 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-30T00:08:53.191 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-30T00:08:53.192 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-30T00:08:53.192 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-30T00:08:53.192 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-30T00:08:53.192 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-30T00:08:53.192 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-30T00:08:53.192 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-30T00:08:53.192 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-30T00:08:53.192 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-30T00:08:53.193 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-30T00:08:53.193 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-30T00:08:53.193 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-30T00:08:53.193 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-30T00:08:53.193 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-30T00:08:53.193 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-30T00:08:53.193 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-30T00:08:53.193 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-30T00:08:53.193 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-30T00:08:53.194 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-30T00:08:53.194 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-30T00:08:53.194 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-30T00:08:53.194 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-30T00:08:53.194 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-30T00:08:53.194 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-30T00:08:53.194 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-30T00:08:53.194 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-30T00:08:53.195 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-30T00:08:53.195 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-30T00:08:53.195 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-30T00:08:53.195 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-30T00:08:53.195 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-30T00:08:53.195 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-30T00:08:53.195 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-30T00:08:53.195 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-30T00:08:53.195 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-30T00:08:53.196 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-30T00:08:53.196 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-30T00:08:53.196 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-30T00:08:53.196 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-30T00:08:53.196 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-30T00:08:53.196 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-30T00:08:53.196 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-30T00:08:53.196 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-30T00:08:53.196 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-30T00:08:53.197 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-30T00:08:53.197 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-30T00:08:53.197 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-30T00:08:53.197 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-30T00:08:53.197 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-30T00:08:53.197 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-30T00:08:53.197 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-30T00:08:53.197 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-30T00:08:53.197 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-30T00:08:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-30T00:08:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-30T00:08:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-30T00:08:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-30T00:08:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-30T00:08:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-30T00:08:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-30T00:08:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-30T00:08:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-30T00:08:53.198 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-30T00:08:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-30T00:08:53.200 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-30T00:08:53.201 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-30T00:08:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-30T00:08:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-30T00:08:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-30T00:08:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-30T00:08:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-30T00:08:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-30T00:08:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-30T00:08:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-30T00:08:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-30T00:08:53.203 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-30T00:08:53.203 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-30T00:08:53.203 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-30T00:08:53.203 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-30T00:08:53.203 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-30T00:08:53.203 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-30T00:08:53.203 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-30T00:08:53.204 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-30T00:08:53.204 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-30T00:08:53.204 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-30T00:08:53.204 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-30T00:08:53.204 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-30T00:08:53.204 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-30T00:08:53.204 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-30T00:08:53.204 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-30T00:08:53.205 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-30T00:08:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-30T00:08:53.207 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-30T00:08:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-30T00:08:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-30T00:08:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-30T00:08:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-30T00:08:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-30T00:08:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-30T00:08:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-30T00:08:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-30T00:08:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-30T00:08:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-30T00:08:53.210 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-30T00:08:53.211 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-30T00:08:53.211 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-30T00:08:53.211 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-30T00:08:53.211 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-30T00:08:53.211 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-30T00:08:53.211 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-30T00:08:53.211 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-30T00:08:53.212 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-30T00:08:53.212 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-30T00:08:53.212 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-30T00:08:53.212 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-30T00:08:53.212 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-30T00:08:53.212 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-30T00:08:53.212 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-30T00:08:53.213 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-30T00:08:53.213 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-30T00:08:53.213 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-30T00:08:53.213 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-30T00:08:53.213 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-30T00:08:53.213 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-30T00:08:53.213 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-30T00:08:53.213 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-30T00:08:53.214 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-30T00:08:53.214 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-30T00:08:53.214 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-30T00:08:53.214 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-30T00:08:53.214 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-30T00:08:53.214 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-30T00:08:53.214 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-30T00:08:53.215 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-30T00:08:53.216 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-30T00:08:53.217 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-30T00:08:53.218 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-30T00:08:53.219 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-30T00:08:53.220 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-30T00:08:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-30T00:08:53.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-30T00:08:53.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-30T00:08:53.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-30T00:08:53.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-30T00:08:53.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-30T00:08:53.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-30T00:08:53.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-30T00:08:53.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-30T00:08:53.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-30T00:08:53.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-30T00:08:53.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-30T00:08:53.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-30T00:08:53.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-30T00:08:53.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-30T00:08:53.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-30T00:08:53.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-30T00:08:53.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-30T00:08:53.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-30T00:08:53.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-30T00:08:53.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-30T00:08:53.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-30T00:08:53.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-30T00:08:53.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-30T00:08:53.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-30T00:08:53.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-30T00:08:53.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-30T00:08:53.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-30T00:08:53.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-30T00:08:53.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-30T00:08:53.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-30T00:08:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-30T00:08:53.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-30T00:08:53.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-30T00:08:53.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-30T00:08:53.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-30T00:08:53.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-30T00:08:53.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-30T00:08:53.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-30T00:08:53.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-30T00:08:53.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-30T00:08:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-30T00:08:53.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-30T00:08:53.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-30T00:08:53.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-30T00:08:53.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-30T00:08:53.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-30T00:08:53.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-30T00:08:53.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-30T00:08:53.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-30T00:08:53.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-30T00:08:53.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-30T00:08:53.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-30T00:08:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-30T00:08:53.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-30T00:08:53.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-30T00:08:53.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-30T00:08:53.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-30T00:08:53.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-30T00:08:53.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-30T00:08:53.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-30T00:08:53.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-30T00:08:53.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-30T00:08:53.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-30T00:08:53.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-30T00:08:53.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-30T00:08:53.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-30T00:08:53.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-30T00:08:53.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-30T00:08:53.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-30T00:08:53.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-30T00:08:53.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-30T00:08:53.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-30T00:08:53.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-30T00:08:53.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-30T00:08:53.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-30T00:08:53.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-30T00:08:53.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-30T00:08:53.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-30T00:08:53.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-30T00:08:53.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-30T00:08:53.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-30T00:08:53.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-30T00:08:53.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-30T00:08:53.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-30T00:08:53.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-30T00:08:53.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-30T00:08:53.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-30T00:08:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-30T00:08:53.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-30T00:08:53.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-30T00:08:53.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-30T00:08:53.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-30T00:08:53.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-30T00:08:53.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-30T00:08:53.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-30T00:08:53.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-30T00:08:53.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-30T00:08:53.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-30T00:08:53.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-30T00:08:53.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-30T00:08:53.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-30T00:08:53.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-30T00:08:53.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-30T00:08:53.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-30T00:08:53.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-30T00:08:53.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-30T00:08:53.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-30T00:08:53.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-30T00:08:53.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-30T00:08:53.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-30T00:08:53.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-30T00:08:53.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-30T00:08:53.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-30T00:08:53.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-30T00:08:53.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-30T00:08:53.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-30T00:08:53.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-30T00:08:53.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-30T00:08:53.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-30T00:08:53.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-30T00:08:53.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-30T00:08:53.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-30T00:08:53.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-30T00:08:53.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-30T00:08:53.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-30T00:08:53.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-30T00:08:53.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-30T00:08:53.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-30T00:08:53.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-30T00:08:53.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-30T00:08:53.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-30T00:08:53.318 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-30T00:08:53.318 DEBUG:teuthology.orchestra.run.smithi099:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-30T00:08:53.343 INFO:teuthology.orchestra.run.smithi099.stdout:postinstall scriptlet (using /bin/sh): 2024-03-30T00:08:53.343 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-30T00:08:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-30T00:08:53.343 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-03-30T00:08:53.343 INFO:teuthology.orchestra.run.smithi099.stdout:for file in vmlinuz System.map config; do 2024-03-30T00:08:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-30T00:08:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-30T00:08:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: fi 2024-03-30T00:08:53.343 INFO:teuthology.orchestra.run.smithi099.stdout:done 2024-03-30T00:08:53.343 INFO:teuthology.orchestra.run.smithi099.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-30T00:08:53.343 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-30T00:08:53.344 INFO:teuthology.orchestra.run.smithi099.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-30T00:08:53.344 INFO:teuthology.orchestra.run.smithi099.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-30T00:08:53.344 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-30T00:08:53.344 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-03-30T00:08:53.344 INFO:teuthology.orchestra.run.smithi099.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-30T00:08:53.344 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-30T00:08:53.344 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-30T00:08:53.344 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-03-30T00:08:53.344 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2024-03-30T00:08:53.417 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2024-03-30T00:08:53.417 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2024-03-30T00:08:53.417 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.02 2024-03-30T00:08:53.417 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 123.el8 2024-03-30T00:08:53.417 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2024-03-30T00:08:53.417 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-03-30T00:08:53.417 INFO:teuthology.orchestra.run.smithi099.stdout:Group : System Environment/Base 2024-03-30T00:08:53.417 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 9265450 2024-03-30T00:08:53.417 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:Relocations : (not relocatable) 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : Red Hat, Inc. 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : Red Hat, Inc. 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-30T00:08:53.418 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2024-03-30T00:08:53.419 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:08:53.419 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2024-03-30T00:08:53.419 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-30T00:08:53.420 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 6.8.0-g1df0c7e8a235 2024-03-30T00:08:53.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-30T00:08:53.635 INFO:teuthology.orchestra.run.smithi161.stderr:done 2024-03-30T00:08:53.636 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-03-30T00:08:53.654 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.dql6UQGBCt 2024-03-30T00:08:53.654 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dql6UQGBCt 2024-03-30T00:08:53.727 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.dql6UQGBCt 2024-03-30T00:08:53.882 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.dql6UQGBCt is 6KB 2024-03-30T00:08:53.930 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.dql6UQGBCt 2024-03-30T00:08:53.947 DEBUG:teuthology.orchestra.run.smithi161:> sudo /bin/ls /boot/loader/entries || true 2024-03-30T00:08:54.016 INFO:teuthology.orchestra.run.smithi161.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-03-30T00:08:54.016 INFO:teuthology.orchestra.run.smithi161.stdout:2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235.conf 2024-03-30T00:08:54.017 INFO:teuthology.orchestra.run.smithi161.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-03-30T00:08:54.017 INFO:teuthology.orchestra.run.smithi161.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-03-30T00:08:54.017 INFO:teuthology.orchestra.run.smithi161.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-03-30T00:08:54.018 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235 2024-03-30T00:08:54.091 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-03-30T00:08:54.175 DEBUG:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2024-03-30T00:08:56.057 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-03-30T00:08:56.059 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-03-30T00:08:56.077 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.uN4ca9sxOQ 2024-03-30T00:08:56.077 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uN4ca9sxOQ 2024-03-30T00:08:56.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.uN4ca9sxOQ 2024-03-30T00:08:56.307 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.uN4ca9sxOQ is 6KB 2024-03-30T00:08:56.355 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.uN4ca9sxOQ 2024-03-30T00:08:56.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2024-03-30T00:08:56.443 INFO:teuthology.orchestra.run.smithi099.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-03-30T00:08:56.443 INFO:teuthology.orchestra.run.smithi099.stdout:2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235.conf 2024-03-30T00:08:56.443 INFO:teuthology.orchestra.run.smithi099.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-03-30T00:08:56.443 INFO:teuthology.orchestra.run.smithi099.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-03-30T00:08:56.444 INFO:teuthology.orchestra.run.smithi099.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-03-30T00:08:56.445 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235 2024-03-30T00:08:56.612 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2024-03-30T00:09:13.437 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-30T00:09:13.437 DEBUG:teuthology.orchestra.run.smithi142:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-30T00:09:13.496 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-30T00:09:13.497 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-30T00:09:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-30T00:09:13.499 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-30T00:09:13.500 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-30T00:09:13.501 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-30T00:09:13.502 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-30T00:09:13.503 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-30T00:09:13.504 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-30T00:09:13.505 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-30T00:09:13.506 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-30T00:09:13.507 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-30T00:09:13.508 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-30T00:09:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-30T00:09:13.510 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-30T00:09:13.511 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-30T00:09:13.512 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-30T00:09:13.513 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-30T00:09:13.514 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-30T00:09:13.515 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-30T00:09:13.516 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-30T00:09:13.516 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-30T00:09:13.516 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-30T00:09:13.516 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-30T00:09:13.516 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-30T00:09:13.516 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-30T00:09:13.516 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-30T00:09:13.516 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-30T00:09:13.516 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-30T00:09:13.516 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-30T00:09:13.516 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-30T00:09:13.517 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-30T00:09:13.517 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-30T00:09:13.517 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-30T00:09:13.517 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-30T00:09:13.517 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-30T00:09:13.517 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-30T00:09:13.518 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-30T00:09:13.518 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-30T00:09:13.518 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-30T00:09:13.518 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-30T00:09:13.518 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-30T00:09:13.518 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-30T00:09:13.518 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-30T00:09:13.518 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-30T00:09:13.518 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-30T00:09:13.518 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-30T00:09:13.519 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-30T00:09:13.519 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-30T00:09:13.519 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-30T00:09:13.519 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-30T00:09:13.519 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-30T00:09:13.519 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-30T00:09:13.519 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-30T00:09:13.519 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-30T00:09:13.519 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-30T00:09:13.519 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-30T00:09:13.519 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-30T00:09:13.520 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-30T00:09:13.520 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-30T00:09:13.520 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-30T00:09:13.520 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-30T00:09:13.520 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-30T00:09:13.520 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-30T00:09:13.520 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-30T00:09:13.520 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-30T00:09:13.520 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-30T00:09:13.520 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-30T00:09:13.520 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-30T00:09:13.521 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-30T00:09:13.522 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-30T00:09:13.523 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-30T00:09:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-30T00:09:13.525 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-30T00:09:13.526 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-30T00:09:13.527 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-30T00:09:13.528 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-30T00:09:13.529 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-30T00:09:13.530 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-30T00:09:13.531 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-30T00:09:13.532 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-30T00:09:13.533 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-30T00:09:13.534 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-30T00:09:13.535 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-30T00:09:13.536 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-30T00:09:13.536 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-30T00:09:13.536 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-30T00:09:13.536 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-30T00:09:13.536 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-30T00:09:13.536 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-30T00:09:13.536 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-30T00:09:13.536 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-30T00:09:13.536 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-30T00:09:13.536 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-30T00:09:13.537 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-30T00:09:13.538 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-30T00:09:13.539 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-30T00:09:13.540 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-30T00:09:13.541 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-30T00:09:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-30T00:09:13.543 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-30T00:09:13.544 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-30T00:09:13.545 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-30T00:09:13.546 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-30T00:09:13.547 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-30T00:09:13.548 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-30T00:09:13.549 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-30T00:09:13.550 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-30T00:09:13.551 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-30T00:09:13.552 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-30T00:09:13.553 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-30T00:09:13.554 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-30T00:09:13.555 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-30T00:09:13.556 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-30T00:09:13.557 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-30T00:09:13.558 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-30T00:09:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-30T00:09:13.560 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-30T00:09:13.561 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-30T00:09:13.562 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-30T00:09:13.563 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-30T00:09:13.564 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-30T00:09:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-30T00:09:13.566 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-30T00:09:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-30T00:09:13.568 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-30T00:09:13.569 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-30T00:09:13.570 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-30T00:09:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-30T00:09:13.572 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-30T00:09:13.573 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-30T00:09:13.574 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-30T00:09:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-30T00:09:13.576 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-30T00:09:13.577 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-30T00:09:13.578 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-30T00:09:13.579 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-30T00:09:13.580 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-30T00:09:13.581 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-30T00:09:13.582 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-30T00:09:13.583 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-30T00:09:13.584 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-30T00:09:13.585 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-30T00:09:13.586 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-30T00:09:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-30T00:09:13.588 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-30T00:09:13.589 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-30T00:09:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-30T00:09:13.591 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-30T00:09:13.594 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-30T00:09:13.594 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-30T00:09:13.594 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-30T00:09:13.595 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-30T00:09:13.596 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-30T00:09:13.597 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-30T00:09:13.598 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-30T00:09:13.599 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-30T00:09:13.600 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-30T00:09:13.601 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-30T00:09:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-30T00:09:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-30T00:09:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-30T00:09:13.605 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-30T00:09:13.606 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-30T00:09:13.607 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-30T00:09:13.608 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-30T00:09:13.609 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-30T00:09:13.610 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-30T00:09:13.611 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-30T00:09:13.612 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-30T00:09:13.613 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-30T00:09:13.614 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-30T00:09:13.615 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-30T00:09:13.616 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-30T00:09:13.617 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-30T00:09:13.618 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-30T00:09:13.619 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-30T00:09:13.620 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-30T00:09:13.621 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-30T00:09:13.622 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-30T00:09:13.623 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-30T00:09:13.624 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-30T00:09:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-30T00:09:13.626 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-30T00:09:13.627 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-30T00:09:13.628 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-30T00:09:13.629 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-30T00:09:13.630 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-30T00:09:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-30T00:09:13.632 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-30T00:09:13.633 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-30T00:09:13.634 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-30T00:09:13.635 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-30T00:09:13.636 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-30T00:09:13.637 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-30T00:09:13.638 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-30T00:09:13.638 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-30T00:09:13.638 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-30T00:09:13.638 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-30T00:09:13.641 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-30T00:09:13.642 DEBUG:teuthology.orchestra.run.smithi142:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-30T00:09:13.665 INFO:teuthology.orchestra.run.smithi142.stdout:postinstall scriptlet (using /bin/sh): 2024-03-30T00:09:13.665 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-30T00:09:13.665 INFO:teuthology.orchestra.run.smithi142.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-30T00:09:13.665 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2024-03-30T00:09:13.665 INFO:teuthology.orchestra.run.smithi142.stdout:for file in vmlinuz System.map config; do 2024-03-30T00:09:13.665 INFO:teuthology.orchestra.run.smithi142.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-30T00:09:13.665 INFO:teuthology.orchestra.run.smithi142.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout: fi 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout:done 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-30T00:09:13.666 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2024-03-30T00:09:13.667 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -qi grub2-tools 2024-03-30T00:09:13.741 INFO:teuthology.orchestra.run.smithi142.stdout:Name : grub2-tools 2024-03-30T00:09:13.742 INFO:teuthology.orchestra.run.smithi142.stdout:Epoch : 1 2024-03-30T00:09:13.742 INFO:teuthology.orchestra.run.smithi142.stdout:Version : 2.02 2024-03-30T00:09:13.742 INFO:teuthology.orchestra.run.smithi142.stdout:Release : 123.el8 2024-03-30T00:09:13.742 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: x86_64 2024-03-30T00:09:13.742 INFO:teuthology.orchestra.run.smithi142.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-03-30T00:09:13.742 INFO:teuthology.orchestra.run.smithi142.stdout:Group : System Environment/Base 2024-03-30T00:09:13.742 INFO:teuthology.orchestra.run.smithi142.stdout:Size : 9265450 2024-03-30T00:09:13.742 INFO:teuthology.orchestra.run.smithi142.stdout:License : GPLv3+ 2024-03-30T00:09:13.742 INFO:teuthology.orchestra.run.smithi142.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-03-30T00:09:13.742 INFO:teuthology.orchestra.run.smithi142.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-03-30T00:09:13.742 INFO:teuthology.orchestra.run.smithi142.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:Relocations : (not relocatable) 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:Packager : Red Hat, Inc. 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:Vendor : Red Hat, Inc. 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:Summary : Support tools for GRUB. 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:Description : 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:hardware devices. 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:09:13.743 INFO:teuthology.orchestra.run.smithi142.stdout:This subpackage provides tools for support of all platforms. 2024-03-30T00:09:13.744 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-30T00:09:13.745 INFO:teuthology.task.kernel:Updating grub on smithi142 to boot 6.8.0-g1df0c7e8a235 2024-03-30T00:09:13.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-30T00:09:14.510 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2024-03-30T00:09:16.340 INFO:teuthology.orchestra.run.smithi142.stderr:done 2024-03-30T00:09:16.342 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-03-30T00:09:16.359 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.VC6nNo1Zjn 2024-03-30T00:09:16.359 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VC6nNo1Zjn 2024-03-30T00:09:16.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.VC6nNo1Zjn 2024-03-30T00:09:16.600 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.VC6nNo1Zjn is 6KB 2024-03-30T00:09:16.648 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.VC6nNo1Zjn 2024-03-30T00:09:16.665 DEBUG:teuthology.orchestra.run.smithi142:> sudo /bin/ls /boot/loader/entries || true 2024-03-30T00:09:16.737 INFO:teuthology.orchestra.run.smithi142.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-03-30T00:09:16.738 INFO:teuthology.orchestra.run.smithi142.stdout:2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235.conf 2024-03-30T00:09:16.738 INFO:teuthology.orchestra.run.smithi142.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-03-30T00:09:16.738 INFO:teuthology.orchestra.run.smithi142.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-03-30T00:09:16.738 INFO:teuthology.orchestra.run.smithi142.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-03-30T00:09:16.739 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235 2024-03-30T00:09:16.898 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2024-03-30T00:09:24.185 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-30T00:09:24.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-30T00:09:24.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:09:26.618 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-30T00:09:26.618 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-30T00:09:26.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:09:42.691 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-03-30T00:09:44.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-03-30T00:09:46.902 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-30T00:09:46.902 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-30T00:09:46.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:09:51.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-30T00:09:51.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:09:53.994 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-30T00:09:53.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:09:54.750 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-03-30T00:09:57.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-03-30T00:10:05.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-03-30T00:10:06.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-30T00:10:06.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:10:09.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-30T00:10:09.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:10:14.474 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-30T00:10:14.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:10:17.534 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-03-30T00:10:25.182 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-03-30T00:10:27.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-03-30T00:10:29.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-30T00:10:29.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:10:40.195 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-30T00:10:40.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:10:42.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-30T00:10:42.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:10:43.262 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-03-30T00:10:43.784 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-03-30T00:10:44.170 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-30T00:10:44.170 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g1df0c7e8a235"... 2024-03-30T00:10:44.170 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-03-30T00:10:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:6.8.0-g1df0c7e8a235 2024-03-30T00:10:44.210 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-30T00:10:44.210 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-30T00:10:44.210 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-03-30T00:10:45.211 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-30T00:10:45.211 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-30T00:10:45.373 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2024-03-30T00:10:45.392 DEBUG:teuthology.parallel:result is None 2024-03-30T00:11:01.263 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-30T00:11:01.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:11:01.485 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-03-30T00:11:01.661 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-03-30T00:11:01.928 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-30T00:11:01.928 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g1df0c7e8a235"... 2024-03-30T00:11:01.929 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-03-30T00:11:01.969 INFO:teuthology.orchestra.run.smithi142.stdout:6.8.0-g1df0c7e8a235 2024-03-30T00:11:01.970 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-30T00:11:01.970 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-30T00:11:01.970 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-03-30T00:11:02.106 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-30T00:11:02.107 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.8.0-g1df0c7e8a235"... 2024-03-30T00:11:02.107 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-03-30T00:11:02.147 INFO:teuthology.orchestra.run.smithi161.stdout:6.8.0-g1df0c7e8a235 2024-03-30T00:11:02.148 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-30T00:11:02.148 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-30T00:11:02.148 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-03-30T00:11:02.971 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-30T00:11:02.971 DEBUG:teuthology.orchestra.run.smithi142:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-30T00:11:03.110 INFO:teuthology.orchestra.run.smithi142.stdout:ttyS1 2024-03-30T00:11:03.144 DEBUG:teuthology.parallel:result is None 2024-03-30T00:11:03.149 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-30T00:11:03.149 DEBUG:teuthology.orchestra.run.smithi161:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-30T00:11:03.316 INFO:teuthology.orchestra.run.smithi161.stdout:ttyS1 2024-03-30T00:11:03.338 DEBUG:teuthology.parallel:result is None 2024-03-30T00:11:03.339 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-30T00:11:03.347 INFO:teuthology.task.internal:Creating test directory... 2024-03-30T00:11:03.347 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T00:11:03.352 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T00:11:03.357 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T00:11:03.398 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-30T00:11:03.405 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-30T00:11:03.447 INFO:teuthology.task.internal:Creating archive directory... 2024-03-30T00:11:03.447 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T00:11:03.452 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T00:11:03.456 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T00:11:03.516 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-30T00:11:03.522 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-30T00:11:03.523 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T00:11:03.527 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T00:11:03.532 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T00:11:03.601 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T00:11:03.608 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T00:11:03.609 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T00:11:03.626 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T00:11:03.635 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T00:11:03.636 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T00:11:03.640 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-30T00:11:03.647 INFO:teuthology.task.internal:Configuring sudo... 2024-03-30T00:11:03.647 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T00:11:03.651 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T00:11:03.654 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T00:11:03.719 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-30T00:11:03.741 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-30T00:11:03.741 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T00:11:03.746 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T00:11:03.750 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T00:11:03.791 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T00:11:03.894 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T00:11:03.985 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T00:11:04.057 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T00:11:04.134 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:11:04.134 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T00:11:04.205 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T00:11:04.275 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T00:11:04.364 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T00:11:04.437 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T00:11:04.515 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-30T00:11:04.515 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T00:11:04.583 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T00:11:04.658 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T00:11:04.727 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T00:11:04.797 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T00:11:04.877 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-30T00:11:04.878 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T00:11:04.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-03-30T00:11:04.947 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-03-30T00:11:04.951 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-03-30T00:11:05.060 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T00:11:05.076 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T00:11:05.082 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T00:11:05.570 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-30T00:11:05.579 INFO:teuthology.task.internal:Starting timer... 2024-03-30T00:11:05.579 INFO:teuthology.run_tasks:Running task pcp... 2024-03-30T00:11:05.589 INFO:teuthology.run_tasks:Running task selinux... 2024-03-30T00:11:05.599 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-30T00:11:05.599 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-03-30T00:11:05.727 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-03-30T00:11:05.731 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-03-30T00:11:05.843 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-03-30T00:11:05.846 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-03-30T00:11:05.980 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-03-30T00:11:05.984 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-30T00:11:05.984 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-03-30T00:11:06.039 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-03-30T00:11:06.039 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-03-30T00:11:06.087 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-03-30T00:11:06.087 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-03-30T00:11:06.141 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-03-30T00:11:06.141 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2024-03-30T00:11:06.141 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T00:11:06.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:11:06.211 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T00:11:06.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:11:06.276 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T00:11:06.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:11:06.349 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-30T00:11:06.349 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-03-30T00:11:06.421 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-03-30T00:11:06.492 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-03-30T00:11:06.559 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-30T00:11:06.571 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-30T00:11:06.577 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-30T00:11:06.736 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-30T00:11:06.753 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-30T00:11:06.755 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi099.front.sepia.ceph.com,smithi142.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-30T00:21:19.790 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2024-03-30T00:21:19.791 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-30T00:21:19.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:21:19.887 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-03-30T00:21:19.990 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-30T00:21:19.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-30T00:21:19.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:21:20.080 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-03-30T00:21:20.197 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-03-30T00:21:20.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-30T00:21:20.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T00:21:20.293 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-03-30T00:21:20.367 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-30T00:21:20.368 INFO:teuthology.run_tasks:Running task clock... 2024-03-30T00:21:20.380 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-30T00:21:20.380 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T00:21:20.381 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T00:21:20.385 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T00:21:20.385 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T00:21:20.390 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T00:21:20.390 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T00:21:20.452 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-30T00:21:20.460 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-30T00:21:20.482 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-30T00:21:20.484 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-30T00:21:20.493 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-30T00:21:20.515 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-30T00:21:20.579 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-03-30T00:21:20.586 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-03-30T00:21:20.605 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-03-30T00:21:20.606 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-03-30T00:21:20.614 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-03-30T00:21:20.633 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-03-30T00:21:20.638 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-30T00:21:20.645 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-30T00:21:20.665 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-30T00:21:20.670 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-30T00:21:20.677 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-30T00:21:20.698 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-30T00:21:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2024-03-30T00:21:20.766 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T00:21:20.766 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-03-30T00:21:20.766 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.766 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.766 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.766 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.773 INFO:teuthology.orchestra.run.smithi142.stderr:bash: ntpq: command not found 2024-03-30T00:21:20.781 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T00:21:20.782 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-03-30T00:21:20.782 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.782 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.782 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.782 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.785 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2024-03-30T00:21:20.793 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T00:21:20.793 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-03-30T00:21:20.793 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.793 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.793 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.793 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:21:20.795 INFO:teuthology.run_tasks:Running task pexec... 2024-03-30T00:21:20.817 INFO:teuthology.task.pexec:Executing custom commands... 2024-03-30T00:21:20.818 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi099.front.sepia.ceph.com 2024-03-30T00:21:20.818 DEBUG:teuthology.orchestra.run.smithi099:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-30T00:21:20.818 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi142.front.sepia.ceph.com 2024-03-30T00:21:20.818 DEBUG:teuthology.orchestra.run.smithi142:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-30T00:21:20.819 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi161.front.sepia.ceph.com 2024-03-30T00:21:20.819 DEBUG:teuthology.orchestra.run.smithi161:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-30T00:21:21.297 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-03-30T00:21:21.298 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2024-03-30T00:21:21.299 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-03-30T00:21:25.179 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-03-30T00:21:25.354 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-03-30T00:21:25.369 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-03-30T00:21:25.951 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-03-30T00:21:26.140 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 17 kB/s | 4.1 kB 00:00 2024-03-30T00:21:26.195 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 13 kB/s | 4.1 kB 00:00 2024-03-30T00:21:28.868 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-30T00:21:28.869 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-30T00:21:28.869 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-03-30T00:21:28.869 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-30T00:21:28.869 INFO:teuthology.orchestra.run.smithi142.stdout:Resetting modules: 2024-03-30T00:21:28.870 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools 2024-03-30T00:21:28.870 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:21:28.870 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-30T00:21:28.870 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-03-30T00:21:28.870 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:21:29.125 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-03-30T00:21:29.126 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-03-30T00:21:29.126 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2024-03-30T00:21:29.127 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-03-30T00:21:29.127 INFO:teuthology.orchestra.run.smithi099.stdout:Resetting modules: 2024-03-30T00:21:29.127 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools 2024-03-30T00:21:29.127 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:21:29.127 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-03-30T00:21:29.128 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-03-30T00:21:29.128 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:21:29.295 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-03-30T00:21:29.295 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-03-30T00:21:29.295 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2024-03-30T00:21:29.295 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-03-30T00:21:29.295 INFO:teuthology.orchestra.run.smithi161.stdout:Resetting modules: 2024-03-30T00:21:29.296 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools 2024-03-30T00:21:29.296 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:21:29.296 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-03-30T00:21:29.296 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-03-30T00:21:29.296 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:21:42.339 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-30T00:21:42.829 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-03-30T00:21:42.940 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-03-30T00:21:43.089 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-03-30T00:21:43.444 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-03-30T00:21:43.695 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2024-03-30T00:21:44.932 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-03-30T00:21:45.384 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 kB/s | 4.5 kB 00:00 2024-03-30T00:21:45.558 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-03-30T00:21:45.645 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 39 kB/s | 4.1 kB 00:00 2024-03-30T00:21:46.028 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-03-30T00:21:46.203 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-03-30T00:21:48.543 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-30T00:21:48.716 INFO:teuthology.orchestra.run.smithi142.stdout:========================================================================================================== 2024-03-30T00:21:48.716 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-03-30T00:21:48.716 INFO:teuthology.orchestra.run.smithi142.stdout:========================================================================================================== 2024-03-30T00:21:48.717 INFO:teuthology.orchestra.run.smithi142.stdout:Installing group/module packages: 2024-03-30T00:21:48.717 INFO:teuthology.orchestra.run.smithi142.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-30T00:21:48.717 INFO:teuthology.orchestra.run.smithi142.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-30T00:21:48.717 INFO:teuthology.orchestra.run.smithi142.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-30T00:21:48.717 INFO:teuthology.orchestra.run.smithi142.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-30T00:21:48.717 INFO:teuthology.orchestra.run.smithi142.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T00:21:48.717 INFO:teuthology.orchestra.run.smithi142.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-30T00:21:48.717 INFO:teuthology.orchestra.run.smithi142.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-30T00:21:48.717 INFO:teuthology.orchestra.run.smithi142.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-30T00:21:48.717 INFO:teuthology.orchestra.run.smithi142.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout:Installing module profiles: 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools/common 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling module streams: 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools rhel8 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:21:48.718 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-30T00:21:48.719 INFO:teuthology.orchestra.run.smithi142.stdout:========================================================================================================== 2024-03-30T00:21:48.719 INFO:teuthology.orchestra.run.smithi142.stdout:Install 13 Packages 2024-03-30T00:21:48.719 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:21:48.719 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 26 M 2024-03-30T00:21:48.720 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 87 M 2024-03-30T00:21:48.720 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-30T00:21:48.991 INFO:teuthology.orchestra.run.smithi142.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 91 kB/s | 25 kB 00:00 2024-03-30T00:21:49.006 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-03-30T00:21:49.008 INFO:teuthology.orchestra.run.smithi142.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 326 kB/s | 94 kB 00:00 2024-03-30T00:21:49.058 INFO:teuthology.orchestra.run.smithi142.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 116 kB/s | 39 kB 00:00 2024-03-30T00:21:49.140 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-03-30T00:21:49.175 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================================== 2024-03-30T00:21:49.175 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================================== 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout:Installing group/module packages: 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-30T00:21:49.176 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout:Installing module profiles: 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools/common 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools rhel8 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-03-30T00:21:49.177 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================================== 2024-03-30T00:21:49.178 INFO:teuthology.orchestra.run.smithi161.stdout:Install 13 Packages 2024-03-30T00:21:49.178 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:21:49.178 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 26 M 2024-03-30T00:21:49.178 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 87 M 2024-03-30T00:21:49.178 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-03-30T00:21:49.241 INFO:teuthology.orchestra.run.smithi142.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 164 kB/s | 41 kB 00:00 2024-03-30T00:21:49.307 INFO:teuthology.orchestra.run.smithi099.stdout:========================================================================================================== 2024-03-30T00:21:49.307 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-03-30T00:21:49.307 INFO:teuthology.orchestra.run.smithi099.stdout:========================================================================================================== 2024-03-30T00:21:49.307 INFO:teuthology.orchestra.run.smithi099.stdout:Installing group/module packages: 2024-03-30T00:21:49.307 INFO:teuthology.orchestra.run.smithi099.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-30T00:21:49.308 INFO:teuthology.orchestra.run.smithi099.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout:Installing module profiles: 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools/common 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling module streams: 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout: container-tools rhel8 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout:========================================================================================================== 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout:Install 13 Packages 2024-03-30T00:21:49.309 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:21:49.310 INFO:teuthology.orchestra.run.smithi142.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 3.4 MB/s | 1.0 MB 00:00 2024-03-30T00:21:49.310 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 26 M 2024-03-30T00:21:49.310 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 87 M 2024-03-30T00:21:49.310 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-03-30T00:21:49.350 INFO:teuthology.orchestra.run.smithi161.stdout:(1/13): criu-libs-3.18-4.module+el8.9.0+20326+3 231 kB/s | 39 kB 00:00 2024-03-30T00:21:49.367 INFO:teuthology.orchestra.run.smithi161.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 132 kB/s | 25 kB 00:00 2024-03-30T00:21:49.392 INFO:teuthology.orchestra.run.smithi161.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 441 kB/s | 94 kB 00:00 2024-03-30T00:21:49.451 INFO:teuthology.orchestra.run.smithi161.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 407 kB/s | 41 kB 00:00 2024-03-30T00:21:49.486 INFO:teuthology.orchestra.run.smithi099.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 141 kB/s | 25 kB 00:00 2024-03-30T00:21:49.512 INFO:teuthology.orchestra.run.smithi099.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 196 kB/s | 39 kB 00:00 2024-03-30T00:21:49.534 INFO:teuthology.orchestra.run.smithi161.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 6.0 MB/s | 1.0 MB 00:00 2024-03-30T00:21:49.562 INFO:teuthology.orchestra.run.smithi099.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 373 kB/s | 94 kB 00:00 2024-03-30T00:21:49.567 INFO:teuthology.orchestra.run.smithi142.stdout:(6/13): buildah-1.31.3-3.module+el8.9.0+21243+a 17 MB/s | 8.8 MB 00:00 2024-03-30T00:21:49.626 INFO:teuthology.orchestra.run.smithi142.stdout:(7/13): cockpit-podman-75-1.module+el8.9.0+2124 1.9 MB/s | 739 kB 00:00 2024-03-30T00:21:49.629 INFO:teuthology.orchestra.run.smithi099.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 288 kB/s | 41 kB 00:00 2024-03-30T00:21:49.671 INFO:teuthology.orchestra.run.smithi099.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 6.4 MB/s | 1.0 MB 00:00 2024-03-30T00:21:49.684 INFO:teuthology.orchestra.run.smithi161.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2124 3.1 MB/s | 739 kB 00:00 2024-03-30T00:21:49.726 INFO:teuthology.orchestra.run.smithi142.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 573 kB/s | 239 kB 00:00 2024-03-30T00:21:49.826 INFO:teuthology.orchestra.run.smithi161.stdout:(7/13): buildah-1.31.3-3.module+el8.9.0+21243+a 20 MB/s | 8.8 MB 00:00 2024-03-30T00:21:49.838 INFO:teuthology.orchestra.run.smithi099.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2124 3.5 MB/s | 739 kB 00:00 2024-03-30T00:21:49.851 INFO:teuthology.orchestra.run.smithi161.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 755 kB/s | 239 kB 00:00 2024-03-30T00:21:49.860 INFO:teuthology.orchestra.run.smithi142.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 657 kB/s | 154 kB 00:00 2024-03-30T00:21:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 1.2 MB/s | 239 kB 00:00 2024-03-30T00:21:49.960 INFO:teuthology.orchestra.run.smithi142.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21243+ 9.5 MB/s | 3.7 MB 00:00 2024-03-30T00:21:50.005 INFO:teuthology.orchestra.run.smithi099.stdout:(8/13): buildah-1.31.3-3.module+el8.9.0+21243+a 20 MB/s | 8.8 MB 00:00 2024-03-30T00:21:50.010 INFO:teuthology.orchestra.run.smithi161.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 838 kB/s | 154 kB 00:00 2024-03-30T00:21:50.064 INFO:teuthology.orchestra.run.smithi099.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 800 kB/s | 154 kB 00:00 2024-03-30T00:21:50.102 INFO:teuthology.orchestra.run.smithi161.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21243+ 8.9 MB/s | 3.7 MB 00:00 2024-03-30T00:21:50.127 INFO:teuthology.orchestra.run.smithi142.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 9.5 MB/s | 2.5 MB 00:00 2024-03-30T00:21:50.173 INFO:teuthology.orchestra.run.smithi099.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21243+ 11 MB/s | 3.7 MB 00:00 2024-03-30T00:21:50.211 INFO:teuthology.orchestra.run.smithi161.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 13 MB/s | 2.5 MB 00:00 2024-03-30T00:21:50.261 INFO:teuthology.orchestra.run.smithi142.stdout:(12/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 15 MB/s | 8.1 MB 00:00 2024-03-30T00:21:50.273 INFO:teuthology.orchestra.run.smithi099.stdout:(11/13): udica-0.2.6-20.module+el8.9.0+21243+a5 495 kB/s | 49 kB 00:00 2024-03-30T00:21:50.277 INFO:teuthology.orchestra.run.smithi142.stdout:(13/13): udica-0.2.6-20.module+el8.9.0+21243+a5 156 kB/s | 49 kB 00:00 2024-03-30T00:21:50.278 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-30T00:21:50.278 INFO:teuthology.orchestra.run.smithi142.stdout:Total 16 MB/s | 26 MB 00:01 2024-03-30T00:21:50.331 INFO:teuthology.orchestra.run.smithi099.stdout:(12/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 9.5 MB/s | 2.5 MB 00:00 2024-03-30T00:21:50.395 INFO:teuthology.orchestra.run.smithi161.stdout:(12/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 15 MB/s | 8.1 MB 00:00 2024-03-30T00:21:50.420 INFO:teuthology.orchestra.run.smithi161.stdout:(13/13): udica-0.2.6-20.module+el8.9.0+21243+a5 156 kB/s | 49 kB 00:00 2024-03-30T00:21:50.420 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-03-30T00:21:50.421 INFO:teuthology.orchestra.run.smithi161.stdout:Total 21 MB/s | 26 MB 00:01 2024-03-30T00:21:50.448 INFO:teuthology.orchestra.run.smithi099.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 18 MB/s | 8.1 MB 00:00 2024-03-30T00:21:50.449 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-03-30T00:21:50.449 INFO:teuthology.orchestra.run.smithi099.stdout:Total 22 MB/s | 26 MB 00:01 2024-03-30T00:21:50.740 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-30T00:21:50.765 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-30T00:21:50.765 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-30T00:21:50.880 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-03-30T00:21:50.905 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-03-30T00:21:50.905 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-03-30T00:21:50.921 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-03-30T00:21:50.946 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-03-30T00:21:50.946 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-03-30T00:21:51.220 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-30T00:21:51.230 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-30T00:21:51.406 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-03-30T00:21:51.413 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-03-30T00:21:51.416 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-03-30T00:21:51.426 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-03-30T00:21:51.935 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-30T00:21:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-03-30T00:21:52.204 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-03-30T00:21:52.457 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-30T00:21:52.634 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-30T00:21:52.664 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-30T00:21:52.728 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-30T00:21:52.846 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-30T00:21:52.869 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-30T00:21:52.943 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-30T00:21:53.003 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-30T00:21:53.044 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-30T00:21:53.148 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-30T00:21:53.219 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-30T00:21:53.248 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-30T00:21:53.320 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-30T00:21:53.432 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-30T00:21:53.457 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-30T00:21:53.538 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-30T00:21:53.668 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-30T00:21:53.689 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-30T00:21:53.802 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-30T00:21:53.886 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-30T00:21:54.058 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-30T00:21:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T00:21:54.813 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T00:21:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T00:21:55.007 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T00:21:55.074 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T00:21:55.191 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-30T00:21:55.260 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T00:21:55.497 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-30T00:21:55.663 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-30T00:21:56.120 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-30T00:21:56.361 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-30T00:21:56.433 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-30T00:21:56.557 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T00:21:56.609 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-30T00:21:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-30T00:21:56.864 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-30T00:21:56.896 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T00:21:57.068 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T00:21:57.214 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T00:21:57.214 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-30T00:21:57.214 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-30T00:21:57.214 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-30T00:21:57.215 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-30T00:21:57.215 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-30T00:21:57.215 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-30T00:21:57.215 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-30T00:21:57.215 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T00:21:57.215 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T00:21:57.215 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-30T00:21:57.215 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-30T00:21:57.215 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-30T00:21:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T00:21:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-30T00:21:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-30T00:21:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-30T00:21:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-30T00:21:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-30T00:21:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-30T00:21:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-30T00:21:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T00:21:57.572 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T00:21:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-30T00:21:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-30T00:21:57.573 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-30T00:21:57.780 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T00:21:57.780 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-30T00:21:57.780 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-30T00:21:57.780 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-30T00:21:57.781 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-30T00:21:57.781 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-30T00:21:57.781 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-30T00:21:57.781 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-30T00:21:57.781 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T00:21:57.781 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T00:21:57.781 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-30T00:21:57.781 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-30T00:21:57.781 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-30T00:22:08.337 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-30T00:22:08.338 INFO:teuthology.orchestra.run.smithi142.stdout:Installed products updated. 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:08.965 INFO:teuthology.orchestra.run.smithi142.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:08.966 INFO:teuthology.orchestra.run.smithi142.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-30T00:22:08.966 INFO:teuthology.orchestra.run.smithi142.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-30T00:22:08.966 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:22:08.966 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-30T00:22:09.207 DEBUG:teuthology.parallel:result is None 2024-03-30T00:22:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-30T00:22:10.786 INFO:teuthology.orchestra.run.smithi099.stdout:Installed products updated. 2024-03-30T00:22:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:22:11.392 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-03-30T00:22:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-30T00:22:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-30T00:22:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-30T00:22:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-30T00:22:11.393 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-30T00:22:11.393 INFO:teuthology.orchestra.run.smithi099.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:11.393 INFO:teuthology.orchestra.run.smithi099.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:11.393 INFO:teuthology.orchestra.run.smithi099.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-30T00:22:11.393 INFO:teuthology.orchestra.run.smithi099.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-30T00:22:11.393 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:22:11.393 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-03-30T00:22:11.614 DEBUG:teuthology.parallel:result is None 2024-03-30T00:22:12.528 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-30T00:22:12.528 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2024-03-30T00:22:13.145 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:22:13.145 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T00:22:13.146 INFO:teuthology.orchestra.run.smithi161.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-30T00:22:13.147 INFO:teuthology.orchestra.run.smithi161.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-30T00:22:13.147 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:22:13.147 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-03-30T00:22:13.373 DEBUG:teuthology.parallel:result is None 2024-03-30T00:22:13.374 INFO:teuthology.run_tasks:Running task install... 2024-03-30T00:22:13.382 DEBUG:teuthology.task.install:project ceph 2024-03-30T00:22:13.383 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}} 2024-03-30T00:22:13.383 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'} 2024-03-30T00:22:13.383 INFO:teuthology.task.install:Using flavor: default 2024-03-30T00:22:13.389 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-30T00:22:13.389 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-30T00:22:13.390 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'tag': None, 'wait_for_package': False} 2024-03-30T00:22:13.391 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:22:13.393 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'tag': None, 'wait_for_package': False} 2024-03-30T00:22:13.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:22:13.394 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'tag': None, 'wait_for_package': False} 2024-03-30T00:22:13.394 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:22:13.537 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/ 2024-03-30T00:22:13.537 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1018.g470100a4 2024-03-30T00:22:13.563 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/ 2024-03-30T00:22:13.563 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1018.g470100a4 2024-03-30T00:22:13.626 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/ 2024-03-30T00:22:13.627 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1018.g470100a4 2024-03-30T00:22:13.660 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-30T00:22:13.660 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:22:13.660 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-30T00:22:13.731 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-30T00:22:13.731 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/470100a4ccbcf7769052bf09a5413881ccf1b97f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-30T00:22:13.742 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-30T00:22:13.742 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-30T00:22:13.742 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-30T00:22:13.742 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-30T00:22:13.742 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-30T00:22:13.743 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-30T00:22:13.809 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-30T00:22:13.809 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/470100a4ccbcf7769052bf09a5413881ccf1b97f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-30T00:22:13.814 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-30T00:22:13.815 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/470100a4ccbcf7769052bf09a5413881ccf1b97f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-30T00:22:13.840 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-30T00:22:13.916 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-30T00:22:13.924 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-30T00:22:13.932 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-30T00:22:13.995 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-03-30T00:22:13.999 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-03-30T00:22:14.006 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-30T00:22:14.014 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-30T00:22:14.070 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-03-30T00:22:14.073 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-03-30T00:22:14.080 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-03-30T00:22:14.081 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-03-30T00:22:14.442 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2024-03-30T00:22:14.519 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-03-30T00:22:14.525 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-03-30T00:22:15.903 INFO:teuthology.orchestra.run.smithi161.stdout:61 files removed 2024-03-30T00:22:15.960 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-30T00:22:16.010 INFO:teuthology.orchestra.run.smithi099.stdout:61 files removed 2024-03-30T00:22:16.037 INFO:teuthology.orchestra.run.smithi142.stdout:61 files removed 2024-03-30T00:22:16.071 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-30T00:22:16.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-30T00:22:16.409 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2024-03-30T00:22:16.511 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2024-03-30T00:22:16.535 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-03-30T00:22:18.474 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 227 kB/s | 80 kB 00:00 2024-03-30T00:22:18.675 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 186 kB/s | 80 kB 00:00 2024-03-30T00:22:18.709 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 192 kB/s | 80 kB 00:00 2024-03-30T00:22:18.825 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 32 kB/s | 9.9 kB 00:00 2024-03-30T00:22:19.033 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 31 kB/s | 9.9 kB 00:00 2024-03-30T00:22:19.101 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 28 kB/s | 9.9 kB 00:00 2024-03-30T00:22:19.159 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 4.9 kB/s | 1.5 kB 00:00 2024-03-30T00:22:19.318 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-03-30T00:22:19.326 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 5.6 kB/s | 1.5 kB 00:00 2024-03-30T00:22:19.461 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2024-03-30T00:22:19.477 INFO:teuthology.orchestra.run.smithi142.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2024-03-30T00:22:19.628 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-03-30T00:22:20.854 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2024-03-30T00:22:20.894 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-03-30T00:22:23.772 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 3.9 MB/s | 16 MB 00:04 2024-03-30T00:22:25.069 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 427 kB/s | 24 kB 00:00 2024-03-30T00:22:25.103 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 395 kB/s | 24 kB 00:00 2024-03-30T00:22:25.770 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-03-30T00:22:26.605 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 60 MB 00:01 2024-03-30T00:22:28.228 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 74 kB/s | 24 kB 00:00 2024-03-30T00:22:28.955 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-03-30T00:22:28.981 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 MB/s | 60 MB 00:01 2024-03-30T00:22:32.031 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 MB/s | 60 MB 00:01 2024-03-30T00:22:37.872 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-03-30T00:22:41.006 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 66 MB 00:01 2024-03-30T00:22:41.132 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 MB/s | 66 MB 00:01 2024-03-30T00:22:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 44 MB/s | 66 MB 00:01 2024-03-30T00:22:56.331 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T00:22:56.332 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T00:22:56.352 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-30T00:22:56.353 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-30T00:22:56.355 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-30T00:22:56.357 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-30T00:22:56.360 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-30T00:22:56.364 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-30T00:22:56.365 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-30T00:22:56.448 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T00:22:56.449 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T00:22:56.469 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-30T00:22:56.470 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-30T00:22:56.472 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-30T00:22:56.473 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-30T00:22:56.476 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-30T00:22:56.480 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-30T00:22:56.482 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-30T00:22:56.598 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-03-30T00:22:56.612 INFO:teuthology.orchestra.run.smithi161.stdout:=================================================================================================================================== 2024-03-30T00:22:56.612 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-03-30T00:22:56.613 INFO:teuthology.orchestra.run.smithi161.stdout:=================================================================================================================================== 2024-03-30T00:22:56.613 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-03-30T00:22:56.613 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-30T00:22:56.613 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-30T00:22:56.613 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.4 k 2024-03-30T00:22:56.613 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:18.2.2-1018.g470100a4.el8 ceph 5.5 M 2024-03-30T00:22:56.613 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 854 k 2024-03-30T00:22:56.613 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1018.g470100a4.el8 ceph 166 k 2024-03-30T00:22:56.613 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:18.2.2-1018.g470100a4.el8 ceph 1.6 M 2024-03-30T00:22:56.613 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 138 k 2024-03-30T00:22:56.613 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 1.8 M 2024-03-30T00:22:56.614 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 7.4 M 2024-03-30T00:22:56.614 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 49 k 2024-03-30T00:22:56.614 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 13 M 2024-03-30T00:22:56.614 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:18.2.2-1018.g470100a4.el8 ceph 47 M 2024-03-30T00:22:56.614 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 263 k 2024-03-30T00:22:56.614 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 173 k 2024-03-30T00:22:56.614 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 261 k 2024-03-30T00:22:56.614 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 18 k 2024-03-30T00:22:56.614 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-30T00:22:56.614 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-30T00:22:56.615 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-30T00:22:56.615 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 31 k 2024-03-30T00:22:56.615 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 775 k 2024-03-30T00:22:56.615 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 139 k 2024-03-30T00:22:56.615 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-30T00:22:56.615 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-30T00:22:56.615 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-30T00:22:56.615 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-30T00:22:56.615 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-30T00:22:56.615 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-30T00:22:56.615 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:18.2.2-1018.g470100a4.el8 ceph 196 k 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:18.2.2-1018.g470100a4.el8 ceph 371 k 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 366 k 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 107 k 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 95 k 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.5 M 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 181 k 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-30T00:22:56.616 INFO:teuthology.orchestra.run.smithi161.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T00:22:56.617 INFO:teuthology.orchestra.run.smithi161.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-30T00:22:56.617 INFO:teuthology.orchestra.run.smithi161.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-30T00:22:56.617 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.7 M 2024-03-30T00:22:56.617 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.6 M 2024-03-30T00:22:56.617 INFO:teuthology.orchestra.run.smithi161.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-30T00:22:56.617 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-30T00:22:56.617 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-30T00:22:56.617 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-30T00:22:56.617 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-30T00:22:56.617 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-30T00:22:56.617 INFO:teuthology.orchestra.run.smithi161.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-30T00:22:56.618 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-30T00:22:56.618 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-03-30T00:22:56.618 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-30T00:22:56.618 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 22 M 2024-03-30T00:22:56.618 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 23 k 2024-03-30T00:22:56.618 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:18.2.2-1018.g470100a4.el8 ceph 2.3 M 2024-03-30T00:22:56.618 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 245 k 2024-03-30T00:22:56.618 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:18.2.2-1018.g470100a4.el8 ceph 4.8 M 2024-03-30T00:22:56.618 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 18 M 2024-03-30T00:22:56.618 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 15 k 2024-03-30T00:22:56.618 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:18.2.2-1018.g470100a4.el8 ceph 24 k 2024-03-30T00:22:56.619 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-30T00:22:56.619 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-30T00:22:56.619 INFO:teuthology.orchestra.run.smithi161.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-30T00:22:56.619 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-30T00:22:56.619 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-30T00:22:56.619 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:18.2.2-1018.g470100a4.el8 ceph 142 k 2024-03-30T00:22:56.619 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-30T00:22:56.619 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-30T00:22:56.619 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-30T00:22:56.619 INFO:teuthology.orchestra.run.smithi161.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-30T00:22:56.619 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-30T00:22:56.620 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-30T00:22:56.620 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 475 k 2024-03-30T00:22:56.620 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-30T00:22:56.620 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.5 M 2024-03-30T00:22:56.620 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-30T00:22:56.620 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-30T00:22:56.620 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-30T00:22:56.620 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-30T00:22:56.620 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-30T00:22:56.620 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-30T00:22:56.620 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-30T00:22:56.621 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-30T00:22:56.621 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-30T00:22:56.621 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-30T00:22:56.621 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-30T00:22:56.621 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T00:22:56.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-30T00:22:56.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-30T00:22:56.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-30T00:22:56.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-30T00:22:56.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-30T00:22:56.621 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 45 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 129 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-30T00:22:56.622 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-30T00:22:56.623 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-30T00:22:56.624 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-30T00:22:56.625 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-30T00:22:56.626 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-30T00:22:56.626 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-30T00:22:56.626 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-30T00:22:56.626 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2024-03-30T00:22:56.626 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql 10 2024-03-30T00:22:56.626 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:22:56.626 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-03-30T00:22:56.626 INFO:teuthology.orchestra.run.smithi161.stdout:=================================================================================================================================== 2024-03-30T00:22:56.626 INFO:teuthology.orchestra.run.smithi161.stdout:Install 130 Packages 2024-03-30T00:22:56.626 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 16 Packages 2024-03-30T00:22:56.626 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:22:56.627 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 213 M 2024-03-30T00:22:56.628 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-03-30T00:22:56.722 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-03-30T00:22:56.736 INFO:teuthology.orchestra.run.smithi142.stdout:=================================================================================================================================== 2024-03-30T00:22:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-03-30T00:22:56.737 INFO:teuthology.orchestra.run.smithi142.stdout:=================================================================================================================================== 2024-03-30T00:22:56.737 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-03-30T00:22:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-30T00:22:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-30T00:22:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.4 k 2024-03-30T00:22:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:18.2.2-1018.g470100a4.el8 ceph 5.5 M 2024-03-30T00:22:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 854 k 2024-03-30T00:22:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1018.g470100a4.el8 ceph 166 k 2024-03-30T00:22:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:18.2.2-1018.g470100a4.el8 ceph 1.6 M 2024-03-30T00:22:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 138 k 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 1.8 M 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 7.4 M 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 49 k 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 13 M 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:18.2.2-1018.g470100a4.el8 ceph 47 M 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 263 k 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 173 k 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 261 k 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 18 k 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-30T00:22:56.738 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 31 k 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 775 k 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 139 k 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:18.2.2-1018.g470100a4.el8 ceph 196 k 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:18.2.2-1018.g470100a4.el8 ceph 371 k 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 366 k 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 107 k 2024-03-30T00:22:56.739 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 95 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.5 M 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 181 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.7 M 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.6 M 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-30T00:22:56.740 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 22 M 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 23 k 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:18.2.2-1018.g470100a4.el8 ceph 2.3 M 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 245 k 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:18.2.2-1018.g470100a4.el8 ceph 4.8 M 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 18 M 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 15 k 2024-03-30T00:22:56.741 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:18.2.2-1018.g470100a4.el8 ceph 24 k 2024-03-30T00:22:56.742 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-30T00:22:56.742 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-30T00:22:56.742 INFO:teuthology.orchestra.run.smithi142.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-30T00:22:56.742 INFO:teuthology.orchestra.run.smithi142.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-30T00:22:56.742 INFO:teuthology.orchestra.run.smithi142.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-30T00:22:56.742 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:18.2.2-1018.g470100a4.el8 ceph 142 k 2024-03-30T00:22:56.742 INFO:teuthology.orchestra.run.smithi142.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-30T00:22:56.742 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-30T00:22:56.742 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-30T00:22:56.742 INFO:teuthology.orchestra.run.smithi142.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 475 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.5 M 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-30T00:22:56.743 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 45 k 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 129 k 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-30T00:22:56.744 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-30T00:22:56.745 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-30T00:22:56.746 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-30T00:22:56.747 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling module streams: 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql 10 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout:=================================================================================================================================== 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout:Install 130 Packages 2024-03-30T00:22:56.748 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 16 Packages 2024-03-30T00:22:56.749 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:22:56.749 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 213 M 2024-03-30T00:22:56.749 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-03-30T00:22:56.972 INFO:teuthology.orchestra.run.smithi161.stdout:(1/146): ceph-18.2.2-1018.g470100a4.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2024-03-30T00:22:57.123 INFO:teuthology.orchestra.run.smithi142.stdout:(1/146): ceph-18.2.2-1018.g470100a4.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2024-03-30T00:22:57.981 INFO:teuthology.orchestra.run.smithi142.stdout:(2/146): ceph-fuse-18.2.2-1018.g470100a4.el8.x8 995 kB/s | 854 kB 00:00 2024-03-30T00:22:58.090 INFO:teuthology.orchestra.run.smithi142.stdout:(3/146): ceph-immutable-object-cache-18.2.2-101 1.5 MB/s | 166 kB 00:00 2024-03-30T00:22:58.214 INFO:teuthology.orchestra.run.smithi161.stdout:(2/146): ceph-common-18.2.2-1018.g470100a4.el8. 14 MB/s | 22 MB 00:01 2024-03-30T00:22:58.256 INFO:teuthology.orchestra.run.smithi161.stdout:(3/146): ceph-fuse-18.2.2-1018.g470100a4.el8.x8 665 kB/s | 854 kB 00:01 2024-03-30T00:22:58.281 INFO:teuthology.orchestra.run.smithi161.stdout:(4/146): ceph-immutable-object-cache-18.2.2-101 2.4 MB/s | 166 kB 00:00 2024-03-30T00:22:58.448 INFO:teuthology.orchestra.run.smithi161.stdout:(5/146): ceph-mgr-18.2.2-1018.g470100a4.el8.x86 9.3 MB/s | 1.6 MB 00:00 2024-03-30T00:22:58.824 INFO:teuthology.orchestra.run.smithi161.stdout:(6/146): ceph-mon-18.2.2-1018.g470100a4.el8.x86 13 MB/s | 4.8 MB 00:00 2024-03-30T00:22:58.898 INFO:teuthology.orchestra.run.smithi142.stdout:(4/146): ceph-mds-18.2.2-1018.g470100a4.el8.x86 2.8 MB/s | 2.3 MB 00:00 2024-03-30T00:22:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:(7/146): ceph-base-18.2.2-1018.g470100a4.el8.x8 2.5 MB/s | 5.5 MB 00:02 2024-03-30T00:22:59.382 INFO:teuthology.orchestra.run.smithi142.stdout:(5/146): ceph-base-18.2.2-1018.g470100a4.el8.x8 2.2 MB/s | 5.5 MB 00:02 2024-03-30T00:22:59.482 INFO:teuthology.orchestra.run.smithi142.stdout:(6/146): ceph-mgr-18.2.2-1018.g470100a4.el8.x86 2.7 MB/s | 1.6 MB 00:00 2024-03-30T00:22:59.774 INFO:teuthology.orchestra.run.smithi161.stdout:(8/146): ceph-mds-18.2.2-1018.g470100a4.el8.x86 1.5 MB/s | 2.3 MB 00:01 2024-03-30T00:22:59.913 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T00:22:59.915 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T00:22:59.934 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-30T00:22:59.935 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-30T00:22:59.937 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-30T00:22:59.938 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-30T00:22:59.941 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-30T00:22:59.946 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-30T00:22:59.947 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-30T00:23:00.066 INFO:teuthology.orchestra.run.smithi161.stdout:(9/146): ceph-selinux-18.2.2-1018.g470100a4.el8 83 kB/s | 24 kB 00:00 2024-03-30T00:23:00.174 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-03-30T00:23:00.189 INFO:teuthology.orchestra.run.smithi099.stdout:=================================================================================================================================== 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout:=================================================================================================================================== 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.4 k 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:18.2.2-1018.g470100a4.el8 ceph 5.5 M 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 854 k 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1018.g470100a4.el8 ceph 166 k 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:18.2.2-1018.g470100a4.el8 ceph 1.6 M 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 138 k 2024-03-30T00:23:00.190 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 1.8 M 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 7.4 M 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 49 k 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 13 M 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:18.2.2-1018.g470100a4.el8 ceph 47 M 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 263 k 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 173 k 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 261 k 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 18 k 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 31 k 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 775 k 2024-03-30T00:23:00.191 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 139 k 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:18.2.2-1018.g470100a4.el8 ceph 196 k 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:18.2.2-1018.g470100a4.el8 ceph 371 k 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 366 k 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 107 k 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 95 k 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.5 M 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 181 k 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-30T00:23:00.192 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.7 M 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.6 M 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-30T00:23:00.193 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 22 M 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 23 k 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:18.2.2-1018.g470100a4.el8 ceph 2.3 M 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 245 k 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:18.2.2-1018.g470100a4.el8 ceph 4.8 M 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 18 M 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 15 k 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:18.2.2-1018.g470100a4.el8 ceph 24 k 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-30T00:23:00.194 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:18.2.2-1018.g470100a4.el8 ceph 142 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 475 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.5 M 2024-03-30T00:23:00.195 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-30T00:23:00.196 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 45 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 129 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-30T00:23:00.197 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-30T00:23:00.198 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-30T00:23:00.199 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-30T00:23:00.200 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling module streams: 2024-03-30T00:23:00.201 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql 10 2024-03-30T00:23:00.201 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:23:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-03-30T00:23:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:=================================================================================================================================== 2024-03-30T00:23:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:Install 130 Packages 2024-03-30T00:23:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 16 Packages 2024-03-30T00:23:00.201 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:23:00.201 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 213 M 2024-03-30T00:23:00.202 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-03-30T00:23:00.758 INFO:teuthology.orchestra.run.smithi161.stdout:(10/146): ceph-osd-18.2.2-1018.g470100a4.el8.x8 9.5 MB/s | 18 MB 00:01 2024-03-30T00:23:00.841 INFO:teuthology.orchestra.run.smithi161.stdout:(11/146): cephfs-mirror-18.2.2-1018.g470100a4.e 3.1 MB/s | 261 kB 00:00 2024-03-30T00:23:00.853 INFO:teuthology.orchestra.run.smithi099.stdout:(1/146): ceph-18.2.2-1018.g470100a4.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2024-03-30T00:23:00.874 INFO:teuthology.orchestra.run.smithi142.stdout:(7/146): ceph-mon-18.2.2-1018.g470100a4.el8.x86 3.2 MB/s | 4.8 MB 00:01 2024-03-30T00:23:00.938 INFO:teuthology.orchestra.run.smithi161.stdout:(12/146): libcephfs-devel-18.2.2-1018.g470100a4 322 kB/s | 31 kB 00:00 2024-03-30T00:23:01.058 INFO:teuthology.orchestra.run.smithi161.stdout:(13/146): libcephfs2-18.2.2-1018.g470100a4.el8. 6.3 MB/s | 775 kB 00:00 2024-03-30T00:23:01.109 INFO:teuthology.orchestra.run.smithi161.stdout:(14/146): libcephsqlite-18.2.2-1018.g470100a4.e 2.8 MB/s | 142 kB 00:00 2024-03-30T00:23:01.184 INFO:teuthology.orchestra.run.smithi161.stdout:(15/146): librados-devel-18.2.2-1018.g470100a4. 1.8 MB/s | 139 kB 00:00 2024-03-30T00:23:01.219 INFO:teuthology.orchestra.run.smithi142.stdout:(8/146): ceph-common-18.2.2-1018.g470100a4.el8. 5.0 MB/s | 22 MB 00:04 2024-03-30T00:23:01.268 INFO:teuthology.orchestra.run.smithi161.stdout:(16/146): libradosstriper1-18.2.2-1018.g470100a 5.6 MB/s | 475 kB 00:00 2024-03-30T00:23:01.308 INFO:teuthology.orchestra.run.smithi142.stdout:(9/146): ceph-selinux-18.2.2-1018.g470100a4.el8 274 kB/s | 24 kB 00:00 2024-03-30T00:23:01.571 INFO:teuthology.orchestra.run.smithi099.stdout:(2/146): ceph-fuse-18.2.2-1018.g470100a4.el8.x8 1.2 MB/s | 854 kB 00:00 2024-03-30T00:23:01.796 INFO:teuthology.orchestra.run.smithi099.stdout:(3/146): ceph-immutable-object-cache-18.2.2-101 737 kB/s | 166 kB 00:00 2024-03-30T00:23:02.284 INFO:teuthology.orchestra.run.smithi161.stdout:(17/146): librgw2-18.2.2-1018.g470100a4.el8.x86 6.4 MB/s | 6.5 MB 00:01 2024-03-30T00:23:02.376 INFO:teuthology.orchestra.run.smithi161.stdout:(18/146): python3-ceph-argparse-18.2.2-1018.g47 488 kB/s | 45 kB 00:00 2024-03-30T00:23:02.719 INFO:teuthology.orchestra.run.smithi161.stdout:(19/146): python3-ceph-common-18.2.2-1018.g4701 377 kB/s | 129 kB 00:00 2024-03-30T00:23:02.933 INFO:teuthology.orchestra.run.smithi142.stdout:(10/146): ceph-osd-18.2.2-1018.g470100a4.el8.x8 5.3 MB/s | 18 MB 00:03 2024-03-30T00:23:02.947 INFO:teuthology.orchestra.run.smithi099.stdout:(4/146): ceph-mds-18.2.2-1018.g470100a4.el8.x86 2.0 MB/s | 2.3 MB 00:01 2024-03-30T00:23:03.002 INFO:teuthology.orchestra.run.smithi161.stdout:(20/146): python3-cephfs-18.2.2-1018.g470100a4. 694 kB/s | 196 kB 00:00 2024-03-30T00:23:03.017 INFO:teuthology.orchestra.run.smithi142.stdout:(11/146): cephfs-mirror-18.2.2-1018.g470100a4.e 3.1 MB/s | 261 kB 00:00 2024-03-30T00:23:03.029 INFO:teuthology.orchestra.run.smithi099.stdout:(5/146): ceph-base-18.2.2-1018.g470100a4.el8.x8 2.2 MB/s | 5.5 MB 00:02 2024-03-30T00:23:03.122 INFO:teuthology.orchestra.run.smithi142.stdout:(12/146): libcephfs-devel-18.2.2-1018.g470100a4 293 kB/s | 31 kB 00:00 2024-03-30T00:23:03.127 INFO:teuthology.orchestra.run.smithi161.stdout:(21/146): python3-rados-18.2.2-1018.g470100a4.e 2.9 MB/s | 371 kB 00:00 2024-03-30T00:23:03.269 INFO:teuthology.orchestra.run.smithi161.stdout:(22/146): python3-rbd-18.2.2-1018.g470100a4.el8 2.5 MB/s | 366 kB 00:00 2024-03-30T00:23:03.378 INFO:teuthology.orchestra.run.smithi161.stdout:(23/146): python3-rgw-18.2.2-1018.g470100a4.el8 989 kB/s | 107 kB 00:00 2024-03-30T00:23:03.409 INFO:teuthology.orchestra.run.smithi142.stdout:(13/146): libcephfs2-18.2.2-1018.g470100a4.el8. 2.6 MB/s | 775 kB 00:00 2024-03-30T00:23:03.453 INFO:teuthology.orchestra.run.smithi161.stdout:(24/146): rbd-fuse-18.2.2-1018.g470100a4.el8.x8 1.2 MB/s | 95 kB 00:00 2024-03-30T00:23:03.480 INFO:teuthology.orchestra.run.smithi099.stdout:(6/146): ceph-mgr-18.2.2-1018.g470100a4.el8.x86 2.9 MB/s | 1.6 MB 00:00 2024-03-30T00:23:03.501 INFO:teuthology.orchestra.run.smithi142.stdout:(14/146): libcephsqlite-18.2.2-1018.g470100a4.e 1.5 MB/s | 142 kB 00:00 2024-03-30T00:23:03.618 INFO:teuthology.orchestra.run.smithi142.stdout:(15/146): librados-devel-18.2.2-1018.g470100a4. 1.2 MB/s | 139 kB 00:00 2024-03-30T00:23:03.901 INFO:teuthology.orchestra.run.smithi142.stdout:(16/146): libradosstriper1-18.2.2-1018.g470100a 1.6 MB/s | 475 kB 00:00 2024-03-30T00:23:04.128 INFO:teuthology.orchestra.run.smithi161.stdout:(25/146): rbd-mirror-18.2.2-1018.g470100a4.el8. 5.1 MB/s | 3.5 MB 00:00 2024-03-30T00:23:04.229 INFO:teuthology.orchestra.run.smithi161.stdout:(26/146): rbd-nbd-18.2.2-1018.g470100a4.el8.x86 1.8 MB/s | 181 kB 00:00 2024-03-30T00:23:04.312 INFO:teuthology.orchestra.run.smithi161.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1018.g 280 kB/s | 23 kB 00:00 2024-03-30T00:23:04.462 INFO:teuthology.orchestra.run.smithi161.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1018.g470100a 920 kB/s | 138 kB 00:00 2024-03-30T00:23:04.896 INFO:teuthology.orchestra.run.smithi161.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1018.g47010 4.1 MB/s | 1.8 MB 00:00 2024-03-30T00:23:05.104 INFO:teuthology.orchestra.run.smithi161.stdout:(30/146): ceph-radosgw-18.2.2-1018.g470100a4.el 2.1 MB/s | 13 MB 00:06 2024-03-30T00:23:05.321 INFO:teuthology.orchestra.run.smithi161.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1018.g47 1.1 MB/s | 245 kB 00:00 2024-03-30T00:23:05.447 INFO:teuthology.orchestra.run.smithi161.stdout:(32/146): ceph-mgr-rook-18.2.2-1018.g470100a4.e 395 kB/s | 49 kB 00:00 2024-03-30T00:23:05.521 INFO:teuthology.orchestra.run.smithi099.stdout:(7/146): ceph-mon-18.2.2-1018.g470100a4.el8.x86 1.9 MB/s | 4.8 MB 00:02 2024-03-30T00:23:05.555 INFO:teuthology.orchestra.run.smithi161.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1018.g4 136 kB/s | 15 kB 00:00 2024-03-30T00:23:05.772 INFO:teuthology.orchestra.run.smithi161.stdout:(34/146): ceph-volume-18.2.2-1018.g470100a4.el8 1.2 MB/s | 263 kB 00:00 2024-03-30T00:23:05.864 INFO:teuthology.orchestra.run.smithi161.stdout:(35/146): cephadm-18.2.2-1018.g470100a4.el8.noa 1.8 MB/s | 173 kB 00:00 2024-03-30T00:23:05.931 INFO:teuthology.orchestra.run.smithi161.stdout:(36/146): cephfs-top-18.2.2-1018.g470100a4.el8. 273 kB/s | 18 kB 00:00 2024-03-30T00:23:06.018 INFO:teuthology.orchestra.run.smithi142.stdout:(17/146): ceph-radosgw-18.2.2-1018.g470100a4.el 2.5 MB/s | 13 MB 00:05 2024-03-30T00:23:06.046 INFO:teuthology.orchestra.run.smithi099.stdout:(8/146): ceph-osd-18.2.2-1018.g470100a4.el8.x86 7.2 MB/s | 18 MB 00:02 2024-03-30T00:23:06.110 INFO:teuthology.orchestra.run.smithi142.stdout:(18/146): librgw2-18.2.2-1018.g470100a4.el8.x86 3.0 MB/s | 6.5 MB 00:02 2024-03-30T00:23:06.113 INFO:teuthology.orchestra.run.smithi099.stdout:(9/146): ceph-selinux-18.2.2-1018.g470100a4.el8 367 kB/s | 24 kB 00:00 2024-03-30T00:23:06.135 INFO:teuthology.orchestra.run.smithi142.stdout:(19/146): python3-ceph-argparse-18.2.2-1018.g47 382 kB/s | 45 kB 00:00 2024-03-30T00:23:06.169 INFO:teuthology.orchestra.run.smithi142.stdout:(20/146): python3-ceph-common-18.2.2-1018.g4701 2.1 MB/s | 129 kB 00:00 2024-03-30T00:23:06.265 INFO:teuthology.orchestra.run.smithi161.stdout:(37/146): ceph-mgr-diskprediction-local-18.2.2- 5.4 MB/s | 7.4 MB 00:01 2024-03-30T00:23:06.269 INFO:teuthology.orchestra.run.smithi142.stdout:(21/146): python3-cephfs-18.2.2-1018.g470100a4. 1.4 MB/s | 196 kB 00:00 2024-03-30T00:23:06.319 INFO:teuthology.orchestra.run.smithi142.stdout:(22/146): python3-rados-18.2.2-1018.g470100a4.e 2.4 MB/s | 371 kB 00:00 2024-03-30T00:23:06.403 INFO:teuthology.orchestra.run.smithi142.stdout:(23/146): python3-rgw-18.2.2-1018.g470100a4.el8 1.3 MB/s | 107 kB 00:00 2024-03-30T00:23:06.407 INFO:teuthology.orchestra.run.smithi161.stdout:(38/146): gperftools-libs-2.7-9.el8.x86_64.rpm 644 kB/s | 306 kB 00:00 2024-03-30T00:23:06.490 INFO:teuthology.orchestra.run.smithi161.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 905 kB/s | 75 kB 00:00 2024-03-30T00:23:06.495 INFO:teuthology.orchestra.run.smithi142.stdout:(24/146): python3-rbd-18.2.2-1018.g470100a4.el8 1.6 MB/s | 366 kB 00:00 2024-03-30T00:23:06.520 INFO:teuthology.orchestra.run.smithi142.stdout:(25/146): rbd-fuse-18.2.2-1018.g470100a4.el8.x8 813 kB/s | 95 kB 00:00 2024-03-30T00:23:06.566 INFO:teuthology.orchestra.run.smithi161.stdout:(40/146): liboath-2.6.2-3.el8.x86_64.rpm 196 kB/s | 59 kB 00:00 2024-03-30T00:23:06.621 INFO:teuthology.orchestra.run.smithi142.stdout:(26/146): rbd-nbd-18.2.2-1018.g470100a4.el8.x86 1.8 MB/s | 181 kB 00:00 2024-03-30T00:23:06.633 INFO:teuthology.orchestra.run.smithi161.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 3.3 MB/s | 476 kB 00:00 2024-03-30T00:23:06.649 INFO:teuthology.orchestra.run.smithi161.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 522 kB/s | 44 kB 00:00 2024-03-30T00:23:06.696 INFO:teuthology.orchestra.run.smithi142.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1018.g 311 kB/s | 23 kB 00:00 2024-03-30T00:23:06.716 INFO:teuthology.orchestra.run.smithi161.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2024-03-30T00:23:06.742 INFO:teuthology.orchestra.run.smithi161.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 358 kB/s | 33 kB 00:00 2024-03-30T00:23:06.784 INFO:teuthology.orchestra.run.smithi161.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 243 kB/s | 16 kB 00:00 2024-03-30T00:23:06.821 INFO:teuthology.orchestra.run.smithi142.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1018.g470100a 1.1 MB/s | 138 kB 00:00 2024-03-30T00:23:06.876 INFO:teuthology.orchestra.run.smithi161.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2024-03-30T00:23:06.901 INFO:teuthology.orchestra.run.smithi161.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2024-03-30T00:23:06.972 INFO:teuthology.orchestra.run.smithi161.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 848 kB/s | 81 kB 00:00 2024-03-30T00:23:07.002 INFO:teuthology.orchestra.run.smithi161.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-03-30T00:23:07.044 INFO:teuthology.orchestra.run.smithi161.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 248 kB/s | 18 kB 00:00 2024-03-30T00:23:07.119 INFO:teuthology.orchestra.run.smithi161.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 651 kB/s | 49 kB 00:00 2024-03-30T00:23:07.194 INFO:teuthology.orchestra.run.smithi161.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 786 kB/s | 59 kB 00:00 2024-03-30T00:23:07.236 INFO:teuthology.orchestra.run.smithi161.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2024-03-30T00:23:07.270 INFO:teuthology.orchestra.run.smithi161.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 802 kB/s | 60 kB 00:00 2024-03-30T00:23:07.320 INFO:teuthology.orchestra.run.smithi161.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2024-03-30T00:23:07.345 INFO:teuthology.orchestra.run.smithi161.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2024-03-30T00:23:07.431 INFO:teuthology.orchestra.run.smithi161.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 305 kB/s | 34 kB 00:00 2024-03-30T00:23:07.463 INFO:teuthology.orchestra.run.smithi161.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2024-03-30T00:23:07.463 INFO:teuthology.orchestra.run.smithi142.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1018.g47010 2.8 MB/s | 1.8 MB 00:00 2024-03-30T00:23:07.505 INFO:teuthology.orchestra.run.smithi161.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 874 kB/s | 64 kB 00:00 2024-03-30T00:23:07.538 INFO:teuthology.orchestra.run.smithi161.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2024-03-30T00:23:07.580 INFO:teuthology.orchestra.run.smithi161.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 319 kB/s | 24 kB 00:00 2024-03-30T00:23:07.622 INFO:teuthology.orchestra.run.smithi161.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 351 kB/s | 29 kB 00:00 2024-03-30T00:23:07.656 INFO:teuthology.orchestra.run.smithi161.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 366 kB/s | 27 kB 00:00 2024-03-30T00:23:07.707 INFO:teuthology.orchestra.run.smithi161.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2024-03-30T00:23:07.740 INFO:teuthology.orchestra.run.smithi161.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2024-03-30T00:23:07.782 INFO:teuthology.orchestra.run.smithi161.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 816 kB/s | 61 kB 00:00 2024-03-30T00:23:07.815 INFO:teuthology.orchestra.run.smithi161.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2024-03-30T00:23:07.821 INFO:teuthology.orchestra.run.smithi142.stdout:(30/146): rbd-mirror-18.2.2-1018.g470100a4.el8. 2.6 MB/s | 3.5 MB 00:01 2024-03-30T00:23:07.882 INFO:teuthology.orchestra.run.smithi161.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2024-03-30T00:23:07.958 INFO:teuthology.orchestra.run.smithi161.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 932 kB/s | 70 kB 00:00 2024-03-30T00:23:07.989 INFO:teuthology.orchestra.run.smithi142.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1018.g47 1.4 MB/s | 245 kB 00:00 2024-03-30T00:23:07.999 INFO:teuthology.orchestra.run.smithi161.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 9.4 MB/s | 1.7 MB 00:00 2024-03-30T00:23:08.064 INFO:teuthology.orchestra.run.smithi142.stdout:(32/146): ceph-mgr-rook-18.2.2-1018.g470100a4.e 656 kB/s | 49 kB 00:00 2024-03-30T00:23:08.139 INFO:teuthology.orchestra.run.smithi142.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1018.g4 196 kB/s | 15 kB 00:00 2024-03-30T00:23:08.337 INFO:teuthology.orchestra.run.smithi161.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 60 kB/s | 20 kB 00:00 2024-03-30T00:23:08.363 INFO:teuthology.orchestra.run.smithi142.stdout:(34/146): ceph-volume-18.2.2-1018.g470100a4.el8 1.1 MB/s | 263 kB 00:00 2024-03-30T00:23:08.367 INFO:teuthology.orchestra.run.smithi161.stdout:(72/146): python3-packaging-16.8-9.el8.noarch.r 118 kB/s | 48 kB 00:00 2024-03-30T00:23:08.456 INFO:teuthology.orchestra.run.smithi142.stdout:(35/146): cephadm-18.2.2-1018.g470100a4.el8.noa 1.8 MB/s | 173 kB 00:00 2024-03-30T00:23:08.534 INFO:teuthology.orchestra.run.smithi161.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 223 kB/s | 44 kB 00:00 2024-03-30T00:23:08.668 INFO:teuthology.orchestra.run.smithi161.stdout:(74/146): python3-pyasn1-modules-0.3.7-6.el8.no 367 kB/s | 110 kB 00:00 2024-03-30T00:23:08.793 INFO:teuthology.orchestra.run.smithi161.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 103 kB/s | 27 kB 00:00 2024-03-30T00:23:08.943 INFO:teuthology.orchestra.run.smithi161.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 460 kB/s | 126 kB 00:00 2024-03-30T00:23:09.010 INFO:teuthology.orchestra.run.smithi161.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.1 MB/s | 457 kB 00:00 2024-03-30T00:23:09.205 INFO:teuthology.orchestra.run.smithi099.stdout:(10/146): ceph-radosgw-18.2.2-1018.g470100a4.el 3.6 MB/s | 13 MB 00:03 2024-03-30T00:23:09.235 INFO:teuthology.orchestra.run.smithi161.stdout:(78/146): bison-3.0.4-10.el8.x86_64.rpm 2.3 MB/s | 688 kB 00:00 2024-03-30T00:23:09.275 INFO:teuthology.orchestra.run.smithi099.stdout:(11/146): cephfs-mirror-18.2.2-1018.g470100a4.e 3.8 MB/s | 261 kB 00:00 2024-03-30T00:23:09.314 INFO:teuthology.orchestra.run.smithi099.stdout:(12/146): libcephfs-devel-18.2.2-1018.g470100a4 741 kB/s | 31 kB 00:00 2024-03-30T00:23:09.344 INFO:teuthology.orchestra.run.smithi161.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 959 kB/s | 320 kB 00:00 2024-03-30T00:23:09.578 INFO:teuthology.orchestra.run.smithi161.stdout:(80/146): libtool-2.4.6-25.el8.x86_64.rpm 2.0 MB/s | 709 kB 00:00 2024-03-30T00:23:09.622 INFO:teuthology.orchestra.run.smithi099.stdout:(13/146): ceph-common-18.2.2-1018.g470100a4.el8 2.4 MB/s | 22 MB 00:09 2024-03-30T00:23:09.628 INFO:teuthology.orchestra.run.smithi161.stdout:(81/146): python3-markupsafe-0.23-19.el8.x86_64 137 kB/s | 39 kB 00:00 2024-03-30T00:23:09.656 INFO:teuthology.orchestra.run.smithi099.stdout:(14/146): libcephfs2-18.2.2-1018.g470100a4.el8. 2.2 MB/s | 775 kB 00:00 2024-03-30T00:23:09.689 INFO:teuthology.orchestra.run.smithi099.stdout:(15/146): libcephsqlite-18.2.2-1018.g470100a4.e 2.1 MB/s | 142 kB 00:00 2024-03-30T00:23:09.714 INFO:teuthology.orchestra.run.smithi099.stdout:(16/146): librados-devel-18.2.2-1018.g470100a4. 2.3 MB/s | 139 kB 00:00 2024-03-30T00:23:09.795 INFO:teuthology.orchestra.run.smithi161.stdout:(82/146): uuid-1.6.2-43.el8.x86_64.rpm 384 kB/s | 64 kB 00:00 2024-03-30T00:23:09.806 INFO:teuthology.orchestra.run.smithi099.stdout:(17/146): libradosstriper1-18.2.2-1018.g470100a 4.0 MB/s | 475 kB 00:00 2024-03-30T00:23:09.849 INFO:teuthology.orchestra.run.smithi099.stdout:(18/146): python3-ceph-argparse-18.2.2-1018.g47 1.0 MB/s | 45 kB 00:00 2024-03-30T00:23:09.870 INFO:teuthology.orchestra.run.smithi161.stdout:(83/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 888 kB/s | 259 kB 00:00 2024-03-30T00:23:09.899 INFO:teuthology.orchestra.run.smithi099.stdout:(19/146): python3-ceph-common-18.2.2-1018.g4701 2.5 MB/s | 129 kB 00:00 2024-03-30T00:23:09.965 INFO:teuthology.orchestra.run.smithi099.stdout:(20/146): python3-cephfs-18.2.2-1018.g470100a4. 2.9 MB/s | 196 kB 00:00 2024-03-30T00:23:10.058 INFO:teuthology.orchestra.run.smithi099.stdout:(21/146): python3-rados-18.2.2-1018.g470100a4.e 3.9 MB/s | 371 kB 00:00 2024-03-30T00:23:10.120 INFO:teuthology.orchestra.run.smithi161.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 318 kB/s | 103 kB 00:00 2024-03-30T00:23:10.158 INFO:teuthology.orchestra.run.smithi099.stdout:(22/146): python3-rbd-18.2.2-1018.g470100a4.el8 3.6 MB/s | 366 kB 00:00 2024-03-30T00:23:10.208 INFO:teuthology.orchestra.run.smithi099.stdout:(23/146): python3-rgw-18.2.2-1018.g470100a4.el8 2.1 MB/s | 107 kB 00:00 2024-03-30T00:23:10.246 INFO:teuthology.orchestra.run.smithi161.stdout:(85/146): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2024-03-30T00:23:10.258 INFO:teuthology.orchestra.run.smithi099.stdout:(24/146): rbd-fuse-18.2.2-1018.g470100a4.el8.x8 1.9 MB/s | 95 kB 00:00 2024-03-30T00:23:10.362 INFO:teuthology.orchestra.run.smithi161.stdout:(86/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.2 MB/s | 538 kB 00:00 2024-03-30T00:23:10.446 INFO:teuthology.orchestra.run.smithi161.stdout:(87/146): socat-1.7.4.1-1.el8.x86_64.rpm 3.8 MB/s | 323 kB 00:00 2024-03-30T00:23:10.575 INFO:teuthology.orchestra.run.smithi099.stdout:(25/146): librgw2-18.2.2-1018.g470100a4.el8.x86 7.6 MB/s | 6.5 MB 00:00 2024-03-30T00:23:10.580 INFO:teuthology.orchestra.run.smithi161.stdout:(88/146): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2024-03-30T00:23:10.613 INFO:teuthology.orchestra.run.smithi161.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2024-03-30T00:23:10.659 INFO:teuthology.orchestra.run.smithi099.stdout:(26/146): rbd-nbd-18.2.2-1018.g470100a4.el8.x86 2.1 MB/s | 181 kB 00:00 2024-03-30T00:23:10.701 INFO:teuthology.orchestra.run.smithi099.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1018.g 558 kB/s | 23 kB 00:00 2024-03-30T00:23:10.751 INFO:teuthology.orchestra.run.smithi099.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1018.g470100a 2.7 MB/s | 138 kB 00:00 2024-03-30T00:23:10.851 INFO:teuthology.orchestra.run.smithi099.stdout:(29/146): rbd-mirror-18.2.2-1018.g470100a4.el8. 5.8 MB/s | 3.5 MB 00:00 2024-03-30T00:23:10.880 INFO:teuthology.orchestra.run.smithi161.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.2 MB/s | 354 kB 00:00 2024-03-30T00:23:10.989 INFO:teuthology.orchestra.run.smithi161.stdout:(91/146): openblas-threads-0.3.15-6.el8.x86_64. 14 MB/s | 5.2 MB 00:00 2024-03-30T00:23:11.018 INFO:teuthology.orchestra.run.smithi099.stdout:(30/146): ceph-mgr-dashboard-18.2.2-1018.g47010 6.7 MB/s | 1.8 MB 00:00 2024-03-30T00:23:11.085 INFO:teuthology.orchestra.run.smithi099.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1018.g47 3.6 MB/s | 245 kB 00:00 2024-03-30T00:23:11.097 INFO:teuthology.orchestra.run.smithi161.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 729 kB/s | 157 kB 00:00 2024-03-30T00:23:11.161 INFO:teuthology.orchestra.run.smithi099.stdout:(32/146): ceph-mgr-rook-18.2.2-1018.g470100a4.e 656 kB/s | 49 kB 00:00 2024-03-30T00:23:11.202 INFO:teuthology.orchestra.run.smithi099.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1018.g4 355 kB/s | 15 kB 00:00 2024-03-30T00:23:11.214 INFO:teuthology.orchestra.run.smithi161.stdout:(93/146): openblas-0.3.15-6.el8.x86_64.rpm 22 MB/s | 5.0 MB 00:00 2024-03-30T00:23:11.273 INFO:teuthology.orchestra.run.smithi161.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 4.0 MB/s | 714 kB 00:00 2024-03-30T00:23:11.277 INFO:teuthology.orchestra.run.smithi099.stdout:(34/146): ceph-volume-18.2.2-1018.g470100a4.el8 3.4 MB/s | 263 kB 00:00 2024-03-30T00:23:11.328 INFO:teuthology.orchestra.run.smithi099.stdout:(35/146): cephadm-18.2.2-1018.g470100a4.el8.noa 3.4 MB/s | 173 kB 00:00 2024-03-30T00:23:11.370 INFO:teuthology.orchestra.run.smithi099.stdout:(36/146): cephfs-top-18.2.2-1018.g470100a4.el8. 437 kB/s | 18 kB 00:00 2024-03-30T00:23:11.440 INFO:teuthology.orchestra.run.smithi161.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 880 kB/s | 198 kB 00:00 2024-03-30T00:23:11.565 INFO:teuthology.orchestra.run.smithi161.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 693 kB/s | 202 kB 00:00 2024-03-30T00:23:11.658 INFO:teuthology.orchestra.run.smithi099.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2024-03-30T00:23:11.737 INFO:teuthology.orchestra.run.smithi099.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 754 kB/s | 59 kB 00:00 2024-03-30T00:23:11.765 INFO:teuthology.orchestra.run.smithi161.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 167 kB/s | 54 kB 00:00 2024-03-30T00:23:11.795 INFO:teuthology.orchestra.run.smithi099.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2024-03-30T00:23:11.924 INFO:teuthology.orchestra.run.smithi161.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-03-30T00:23:11.929 INFO:teuthology.orchestra.run.smithi099.stdout:(40/146): ceph-mgr-diskprediction-local-18.2.2- 6.9 MB/s | 7.4 MB 00:01 2024-03-30T00:23:11.971 INFO:teuthology.orchestra.run.smithi099.stdout:(41/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.7 MB/s | 476 kB 00:00 2024-03-30T00:23:12.046 INFO:teuthology.orchestra.run.smithi099.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2024-03-30T00:23:12.105 INFO:teuthology.orchestra.run.smithi099.stdout:(43/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 248 kB/s | 44 kB 00:00 2024-03-30T00:23:12.130 INFO:teuthology.orchestra.run.smithi099.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 393 kB/s | 33 kB 00:00 2024-03-30T00:23:12.197 INFO:teuthology.orchestra.run.smithi099.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 178 kB/s | 16 kB 00:00 2024-03-30T00:23:12.223 INFO:teuthology.orchestra.run.smithi099.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2024-03-30T00:23:12.233 INFO:teuthology.orchestra.run.smithi161.stdout:(99/146): python3-scipy-1.0.0-21.module+el8.9.0 21 MB/s | 14 MB 00:00 2024-03-30T00:23:12.298 INFO:teuthology.orchestra.run.smithi099.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2024-03-30T00:23:12.308 INFO:teuthology.orchestra.run.smithi161.stdout:(100/146): postgresql-10.23-4.module+el8.9.0+21 3.9 MB/s | 1.5 MB 00:00 2024-03-30T00:23:12.348 INFO:teuthology.orchestra.run.smithi099.stdout:(48/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2024-03-30T00:23:12.365 INFO:teuthology.orchestra.run.smithi099.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2024-03-30T00:23:12.399 INFO:teuthology.orchestra.run.smithi099.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2024-03-30T00:23:12.449 INFO:teuthology.orchestra.run.smithi099.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 979 kB/s | 49 kB 00:00 2024-03-30T00:23:12.499 INFO:teuthology.orchestra.run.smithi099.stdout:(52/146): python3-kubernetes-11.0.0-6.el8.noarc 8.7 MB/s | 1.2 MB 00:00 2024-03-30T00:23:12.516 INFO:teuthology.orchestra.run.smithi099.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 877 kB/s | 59 kB 00:00 2024-03-30T00:23:12.549 INFO:teuthology.orchestra.run.smithi099.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2024-03-30T00:23:12.600 INFO:teuthology.orchestra.run.smithi099.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2024-03-30T00:23:12.617 INFO:teuthology.orchestra.run.smithi099.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2024-03-30T00:23:12.633 INFO:teuthology.orchestra.run.smithi161.stdout:(101/146): postgresql-contrib-10.23-4.module+el 2.0 MB/s | 812 kB 00:00 2024-03-30T00:23:12.691 INFO:teuthology.orchestra.run.smithi099.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 371 kB/s | 34 kB 00:00 2024-03-30T00:23:12.726 INFO:teuthology.orchestra.run.smithi099.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2024-03-30T00:23:12.751 INFO:teuthology.orchestra.run.smithi099.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2024-03-30T00:23:12.767 INFO:teuthology.orchestra.run.smithi161.stdout:(102/146): postgresql-server-10.23-4.module+el8 11 MB/s | 5.1 MB 00:00 2024-03-30T00:23:12.793 INFO:teuthology.orchestra.run.smithi099.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 280 kB/s | 19 kB 00:00 2024-03-30T00:23:12.818 INFO:teuthology.orchestra.run.smithi099.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 357 kB/s | 24 kB 00:00 2024-03-30T00:23:12.852 INFO:teuthology.orchestra.run.smithi099.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 499 kB/s | 29 kB 00:00 2024-03-30T00:23:12.877 INFO:teuthology.orchestra.run.smithi099.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 473 kB/s | 27 kB 00:00 2024-03-30T00:23:12.892 INFO:teuthology.orchestra.run.smithi161.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 69 kB/s | 18 kB 00:00 2024-03-30T00:23:12.917 INFO:teuthology.orchestra.run.smithi161.stdout:(104/146): python3-jwt-1.6.1-2.el8.noarch.rpm 284 kB/s | 43 kB 00:00 2024-03-30T00:23:12.919 INFO:teuthology.orchestra.run.smithi099.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2024-03-30T00:23:12.954 INFO:teuthology.orchestra.run.smithi099.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2024-03-30T00:23:12.978 INFO:teuthology.orchestra.run.smithi099.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2024-03-30T00:23:13.011 INFO:teuthology.orchestra.run.smithi099.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2024-03-30T00:23:13.037 INFO:teuthology.orchestra.run.smithi099.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 393 kB/s | 23 kB 00:00 2024-03-30T00:23:13.062 INFO:teuthology.orchestra.run.smithi099.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2024-03-30T00:23:13.171 INFO:teuthology.orchestra.run.smithi099.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2024-03-30T00:23:13.229 INFO:teuthology.orchestra.run.smithi099.stdout:(71/146): python3-packaging-16.8-9.el8.noarch.r 289 kB/s | 48 kB 00:00 2024-03-30T00:23:13.321 INFO:teuthology.orchestra.run.smithi099.stdout:(72/146): python3-jsonpointer-1.10-11.el8.noarc 133 kB/s | 20 kB 00:00 2024-03-30T00:23:13.343 INFO:teuthology.orchestra.run.smithi161.stdout:(105/146): python3-oauthlib-2.1.0-1.el8.noarch. 344 kB/s | 155 kB 00:00 2024-03-30T00:23:13.372 INFO:teuthology.orchestra.run.smithi099.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 309 kB/s | 44 kB 00:00 2024-03-30T00:23:13.377 INFO:teuthology.orchestra.run.smithi161.stdout:(106/146): python3-pycparser-2.14-14.el8.noarch 238 kB/s | 109 kB 00:00 2024-03-30T00:23:13.472 INFO:teuthology.orchestra.run.smithi099.stdout:(74/146): python3-pyasn1-modules-0.3.7-6.el8.no 733 kB/s | 110 kB 00:00 2024-03-30T00:23:13.505 INFO:teuthology.orchestra.run.smithi099.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 199 kB/s | 27 kB 00:00 2024-03-30T00:23:13.594 INFO:teuthology.orchestra.run.smithi161.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 171 kB/s | 43 kB 00:00 2024-03-30T00:23:13.619 INFO:teuthology.orchestra.run.smithi161.stdout:(108/146): libaio-devel-0.3.112-1.el8.x86_64.rp 77 kB/s | 19 kB 00:00 2024-03-30T00:23:13.656 INFO:teuthology.orchestra.run.smithi099.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 689 kB/s | 126 kB 00:00 2024-03-30T00:23:13.689 INFO:teuthology.orchestra.run.smithi099.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.4 MB/s | 457 kB 00:00 2024-03-30T00:23:13.952 INFO:teuthology.orchestra.run.smithi161.stdout:(109/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 122 kB/s | 44 kB 00:00 2024-03-30T00:23:14.036 INFO:teuthology.orchestra.run.smithi161.stdout:(110/146): pcre2-devel-10.32-2.el8.x86_64.rpm 1.4 MB/s | 605 kB 00:00 2024-03-30T00:23:14.123 INFO:teuthology.orchestra.run.smithi099.stdout:(78/146): bison-3.0.4-10.el8.x86_64.rpm 1.4 MB/s | 688 kB 00:00 2024-03-30T00:23:14.156 INFO:teuthology.orchestra.run.smithi099.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 684 kB/s | 320 kB 00:00 2024-03-30T00:23:14.161 INFO:teuthology.orchestra.run.smithi161.stdout:(111/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.1 MB/s | 229 kB 00:00 2024-03-30T00:23:14.298 INFO:teuthology.orchestra.run.smithi099.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 274 kB/s | 39 kB 00:00 2024-03-30T00:23:14.348 INFO:teuthology.orchestra.run.smithi099.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 3.1 MB/s | 709 kB 00:00 2024-03-30T00:23:14.370 INFO:teuthology.orchestra.run.smithi161.stdout:(112/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 661 kB/s | 220 kB 00:00 2024-03-30T00:23:14.407 INFO:teuthology.orchestra.run.smithi099.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.3 MB/s | 259 kB 00:00 2024-03-30T00:23:14.444 INFO:teuthology.orchestra.run.smithi161.stdout:(113/146): libselinux-devel-2.9-5.el8.x86_64.rp 706 kB/s | 200 kB 00:00 2024-03-30T00:23:14.499 INFO:teuthology.orchestra.run.smithi099.stdout:(83/146): uuid-1.6.2-43.el8.x86_64.rpm 426 kB/s | 64 kB 00:00 2024-03-30T00:23:14.524 INFO:teuthology.orchestra.run.smithi099.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 887 kB/s | 103 kB 00:00 2024-03-30T00:23:14.654 INFO:teuthology.orchestra.run.smithi161.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 170 kB/s | 48 kB 00:00 2024-03-30T00:23:14.679 INFO:teuthology.orchestra.run.smithi161.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 372 kB/s | 87 kB 00:00 2024-03-30T00:23:14.691 INFO:teuthology.orchestra.run.smithi099.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 3.2 MB/s | 538 kB 00:00 2024-03-30T00:23:14.800 INFO:teuthology.orchestra.run.smithi099.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2024-03-30T00:23:14.896 INFO:teuthology.orchestra.run.smithi161.stdout:(116/146): libcom_err-devel-1.45.6-4.el8.x86_64 178 kB/s | 39 kB 00:00 2024-03-30T00:23:14.921 INFO:teuthology.orchestra.run.smithi161.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 216 kB/s | 58 kB 00:00 2024-03-30T00:23:14.925 INFO:teuthology.orchestra.run.smithi099.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 16 MB/s | 3.7 MB 00:00 2024-03-30T00:23:14.958 INFO:teuthology.orchestra.run.smithi099.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.0 MB/s | 323 kB 00:00 2024-03-30T00:23:15.084 INFO:teuthology.orchestra.run.smithi099.stdout:(89/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2024-03-30T00:23:15.126 INFO:teuthology.orchestra.run.smithi099.stdout:(90/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.1 MB/s | 225 kB 00:00 2024-03-30T00:23:15.138 INFO:teuthology.orchestra.run.smithi161.stdout:(118/146): fuse-2.9.7-17.el8.x86_64.rpm 343 kB/s | 83 kB 00:00 2024-03-30T00:23:15.172 INFO:teuthology.orchestra.run.smithi161.stdout:(119/146): libgfortran-8.5.0-20.el8.x86_64.rpm 2.5 MB/s | 645 kB 00:00 2024-03-30T00:23:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 896 kB/s | 157 kB 00:00 2024-03-30T00:23:15.359 INFO:teuthology.orchestra.run.smithi161.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 782 kB/s | 172 kB 00:00 2024-03-30T00:23:15.389 INFO:teuthology.orchestra.run.smithi161.stdout:(121/146): libuuid-devel-2.32.1-43.el8.x86_64.r 460 kB/s | 100 kB 00:00 2024-03-30T00:23:15.418 INFO:teuthology.orchestra.run.smithi099.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 16 MB/s | 5.2 MB 00:00 2024-03-30T00:23:15.433 INFO:teuthology.orchestra.run.smithi142.stdout:(36/146): cephfs-top-18.2.2-1018.g470100a4.el8. 2.6 kB/s | 18 kB 00:06 2024-03-30T00:23:15.543 INFO:teuthology.orchestra.run.smithi099.stdout:(93/146): automake-1.16.1-8.el8.noarch.rpm 5.6 MB/s | 714 kB 00:00 2024-03-30T00:23:15.606 INFO:teuthology.orchestra.run.smithi161.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 966 kB/s | 238 kB 00:00 2024-03-30T00:23:15.631 INFO:teuthology.orchestra.run.smithi161.stdout:(123/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 300 kB/s | 73 kB 00:00 2024-03-30T00:23:15.635 INFO:teuthology.orchestra.run.smithi099.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-03-30T00:23:15.694 INFO:teuthology.orchestra.run.smithi099.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 1.3 MB/s | 198 kB 00:00 2024-03-30T00:23:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2024-03-30T00:23:15.803 INFO:teuthology.orchestra.run.smithi099.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 500 kB/s | 54 kB 00:00 2024-03-30T00:23:15.953 INFO:teuthology.orchestra.run.smithi099.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-03-30T00:23:16.073 INFO:teuthology.orchestra.run.smithi142.stdout:(37/146): ceph-test-18.2.2-1018.g470100a4.el8.x 3.2 MB/s | 47 MB 00:14 2024-03-30T00:23:16.107 INFO:teuthology.orchestra.run.smithi142.stdout:(38/146): gperftools-libs-2.7-9.el8.x86_64.rpm 455 kB/s | 306 kB 00:00 2024-03-30T00:23:16.145 INFO:teuthology.orchestra.run.smithi099.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 7.9 MB/s | 1.5 MB 00:00 2024-03-30T00:23:16.208 INFO:teuthology.orchestra.run.smithi142.stdout:(39/146): ceph-mgr-diskprediction-local-18.2.2- 866 kB/s | 7.4 MB 00:08 2024-03-30T00:23:16.208 INFO:teuthology.orchestra.run.smithi161.stdout:(124/146): ceph-test-18.2.2-1018.g470100a4.el8. 2.9 MB/s | 47 MB 00:16 2024-03-30T00:23:16.249 INFO:teuthology.orchestra.run.smithi142.stdout:(40/146): liboath-2.6.2-3.el8.x86_64.rpm 336 kB/s | 59 kB 00:00 2024-03-30T00:23:16.256 INFO:teuthology.orchestra.run.smithi161.stdout:(125/146): elfutils-libelf-devel-0.189-3.el8.x8 99 kB/s | 62 kB 00:00 2024-03-30T00:23:16.290 INFO:teuthology.orchestra.run.smithi161.stdout:(126/146): NetworkManager-initscripts-updown-1. 211 kB/s | 144 kB 00:00 2024-03-30T00:23:16.324 INFO:teuthology.orchestra.run.smithi142.stdout:(41/146): libunwind-1.3.1-3.el8.x86_64.rpm 346 kB/s | 75 kB 00:00 2024-03-30T00:23:16.337 INFO:teuthology.orchestra.run.smithi099.stdout:(100/146): python3-scipy-1.0.0-21.module+el8.9. 23 MB/s | 14 MB 00:00 2024-03-30T00:23:16.387 INFO:teuthology.orchestra.run.smithi099.stdout:(101/146): postgresql-contrib-10.23-4.module+el 3.3 MB/s | 812 kB 00:00 2024-03-30T00:23:16.442 INFO:teuthology.orchestra.run.smithi142.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 228 kB/s | 44 kB 00:00 2024-03-30T00:23:16.490 INFO:teuthology.orchestra.run.smithi161.stdout:(127/146): openssl-devel-1.1.1k-12.el8_9.x86_64 10 MB/s | 2.3 MB 00:00 2024-03-30T00:23:16.516 INFO:teuthology.orchestra.run.smithi161.stdout:(128/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 209 kB/s | 47 kB 00:00 2024-03-30T00:23:16.554 INFO:teuthology.orchestra.run.smithi099.stdout:(102/146): libverto-devel-0.3.0-5.el8.x86_64.rp 107 kB/s | 18 kB 00:00 2024-03-30T00:23:16.558 INFO:teuthology.orchestra.run.smithi161.stdout:(129/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 1.6 MB/s | 562 kB 00:00 2024-03-30T00:23:16.575 INFO:teuthology.orchestra.run.smithi142.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 248 kB/s | 33 kB 00:00 2024-03-30T00:23:16.609 INFO:teuthology.orchestra.run.smithi142.stdout:(44/146): python3-beautifulsoup4-4.6.3-2.el8.1. 653 kB/s | 185 kB 00:00 2024-03-30T00:23:16.621 INFO:teuthology.orchestra.run.smithi099.stdout:(103/146): postgresql-server-10.23-4.module+el8 18 MB/s | 5.1 MB 00:00 2024-03-30T00:23:16.676 INFO:teuthology.orchestra.run.smithi142.stdout:(45/146): python3-asyncssh-2.7.0-2.el8.noarch.r 1.0 MB/s | 476 kB 00:00 2024-03-30T00:23:16.709 INFO:teuthology.orchestra.run.smithi142.stdout:(46/146): python3-certifi-2018.10.15-7.el8.noar 122 kB/s | 16 kB 00:00 2024-03-30T00:23:16.741 INFO:teuthology.orchestra.run.smithi161.stdout:(130/146): python3-cryptography-3.2.1-7.el8_9.x 2.2 MB/s | 559 kB 00:00 2024-03-30T00:23:16.810 INFO:teuthology.orchestra.run.smithi142.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 861 kB/s | 173 kB 00:00 2024-03-30T00:23:16.926 INFO:teuthology.orchestra.run.smithi142.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2024-03-30T00:23:16.960 INFO:teuthology.orchestra.run.smithi142.stdout:(49/146): python3-google-auth-1.1.1-10.el8.noar 324 kB/s | 81 kB 00:00 2024-03-30T00:23:17.035 INFO:teuthology.orchestra.run.smithi142.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2024-03-30T00:23:17.067 INFO:teuthology.orchestra.run.smithi161.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 715 kB/s | 232 kB 00:00 2024-03-30T00:23:17.071 INFO:teuthology.orchestra.run.smithi099.stdout:(104/146): ceph-test-18.2.2-1018.g470100a4.el8. 4.3 MB/s | 47 MB 00:10 2024-03-30T00:23:17.097 INFO:teuthology.orchestra.run.smithi099.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 78 kB/s | 43 kB 00:00 2024-03-30T00:23:17.130 INFO:teuthology.orchestra.run.smithi099.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 304 kB/s | 155 kB 00:00 2024-03-30T00:23:17.135 INFO:teuthology.orchestra.run.smithi142.stdout:(51/146): python3-cherrypy-18.4.0-1.el8.noarch. 837 kB/s | 384 kB 00:00 2024-03-30T00:23:17.169 INFO:teuthology.orchestra.run.smithi142.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 365 kB/s | 49 kB 00:00 2024-03-30T00:23:17.222 INFO:teuthology.orchestra.run.smithi099.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 728 kB/s | 109 kB 00:00 2024-03-30T00:23:17.247 INFO:teuthology.orchestra.run.smithi099.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 295 kB/s | 43 kB 00:00 2024-03-30T00:23:17.267 INFO:teuthology.orchestra.run.smithi161.stdout:(132/146): librados2-18.2.2-1018.g470100a4.el8. 4.9 MB/s | 3.7 MB 00:00 2024-03-30T00:23:17.278 INFO:teuthology.orchestra.run.smithi142.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 414 kB/s | 59 kB 00:00 2024-03-30T00:23:17.281 INFO:teuthology.orchestra.run.smithi099.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 124 kB/s | 19 kB 00:00 2024-03-30T00:23:17.317 INFO:teuthology.orchestra.run.smithi161.stdout:(133/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.2 MB/s | 303 kB 00:00 2024-03-30T00:23:17.340 INFO:teuthology.orchestra.run.smithi142.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 354 kB/s | 60 kB 00:00 2024-03-30T00:23:17.365 INFO:teuthology.orchestra.run.smithi099.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 308 kB/s | 44 kB 00:00 2024-03-30T00:23:17.378 INFO:teuthology.orchestra.run.smithi142.stdout:(55/146): python3-kubernetes-11.0.0-6.el8.noarc 2.8 MB/s | 1.2 MB 00:00 2024-03-30T00:23:17.406 INFO:teuthology.orchestra.run.smithi099.stdout:(111/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.8 MB/s | 229 kB 00:00 2024-03-30T00:23:17.440 INFO:teuthology.orchestra.run.smithi099.stdout:(112/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.1 MB/s | 605 kB 00:00 2024-03-30T00:23:17.462 INFO:teuthology.orchestra.run.smithi142.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 128 kB/s | 16 kB 00:00 2024-03-30T00:23:17.512 INFO:teuthology.orchestra.run.smithi142.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 252 kB/s | 34 kB 00:00 2024-03-30T00:23:17.552 INFO:teuthology.orchestra.run.smithi161.stdout:(134/146): libsmartcols-2.32.1-43.el8.x86_64.rp 767 kB/s | 179 kB 00:00 2024-03-30T00:23:17.590 INFO:teuthology.orchestra.run.smithi099.stdout:(113/146): keyutils-libs-devel-1.5.10-9.el8.x86 320 kB/s | 48 kB 00:00 2024-03-30T00:23:17.596 INFO:teuthology.orchestra.run.smithi142.stdout:(58/146): python3-pecan-1.3.2-9.el8.noarch.rpm 890 kB/s | 283 kB 00:00 2024-03-30T00:23:17.609 INFO:teuthology.orchestra.run.smithi161.stdout:(135/146): librbd1-18.2.2-1018.g470100a4.el8.x8 3.4 MB/s | 3.6 MB 00:01 2024-03-30T00:23:17.632 INFO:teuthology.orchestra.run.smithi099.stdout:(114/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 825 kB/s | 220 kB 00:00 2024-03-30T00:23:17.658 INFO:teuthology.orchestra.run.smithi099.stdout:(115/146): libselinux-devel-2.9-5.el8.x86_64.rp 798 kB/s | 200 kB 00:00 2024-03-30T00:23:17.660 INFO:teuthology.orchestra.run.smithi161.stdout:(136/146): libmount-2.32.1-43.el8.x86_64.rpm 603 kB/s | 237 kB 00:00 2024-03-30T00:23:17.680 INFO:teuthology.orchestra.run.smithi142.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 905 kB/s | 196 kB 00:00 2024-03-30T00:23:17.683 INFO:teuthology.orchestra.run.smithi099.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 947 kB/s | 87 kB 00:00 2024-03-30T00:23:17.705 INFO:teuthology.orchestra.run.smithi142.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 334 kB/s | 64 kB 00:00 2024-03-30T00:23:17.738 INFO:teuthology.orchestra.run.smithi142.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 132 kB/s | 19 kB 00:00 2024-03-30T00:23:17.750 INFO:teuthology.orchestra.run.smithi099.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 493 kB/s | 58 kB 00:00 2024-03-30T00:23:17.817 INFO:teuthology.orchestra.run.smithi099.stdout:(118/146): fuse-2.9.7-17.el8.x86_64.rpm 622 kB/s | 83 kB 00:00 2024-03-30T00:23:17.867 INFO:teuthology.orchestra.run.smithi099.stdout:(119/146): libcom_err-devel-1.45.6-4.el8.x86_64 185 kB/s | 39 kB 00:00 2024-03-30T00:23:17.897 INFO:teuthology.orchestra.run.smithi142.stdout:(62/146): python3-trustme-0.6.0-4.el8.noarch.rp 174 kB/s | 27 kB 00:00 2024-03-30T00:23:17.922 INFO:teuthology.orchestra.run.smithi142.stdout:(63/146): python3-singledispatch-3.4.0.3-18.el8 99 kB/s | 24 kB 00:00 2024-03-30T00:23:17.925 INFO:teuthology.orchestra.run.smithi099.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.5 MB/s | 172 kB 00:00 2024-03-30T00:23:17.939 INFO:teuthology.orchestra.run.smithi142.stdout:(64/146): python3-tempora-1.14.1-5.el8.noarch.r 125 kB/s | 29 kB 00:00 2024-03-30T00:23:17.943 INFO:teuthology.orchestra.run.smithi161.stdout:(137/146): libuuid-2.32.1-43.el8.x86_64.rpm 253 kB/s | 99 kB 00:00 2024-03-30T00:23:18.009 INFO:teuthology.orchestra.run.smithi099.stdout:(121/146): libgfortran-8.5.0-20.el8.x86_64.rpm 2.4 MB/s | 645 kB 00:00 2024-03-30T00:23:18.027 INFO:teuthology.orchestra.run.smithi161.stdout:(138/146): util-linux-2.32.1-43.el8.x86_64.rpm 5.9 MB/s | 2.5 MB 00:00 2024-03-30T00:23:18.034 INFO:teuthology.orchestra.run.smithi099.stdout:(122/146): libuuid-devel-2.32.1-43.el8.x86_64.r 596 kB/s | 100 kB 00:00 2024-03-30T00:23:18.060 INFO:teuthology.orchestra.run.smithi161.stdout:(139/146): util-linux-user-2.32.1-43.el8.x86_64 256 kB/s | 103 kB 00:00 2024-03-30T00:23:18.102 INFO:teuthology.orchestra.run.smithi099.stdout:(123/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2024-03-30T00:23:18.127 INFO:teuthology.orchestra.run.smithi099.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 621 kB/s | 73 kB 00:00 2024-03-30T00:23:18.156 INFO:teuthology.orchestra.run.smithi142.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 986 kB/s | 255 kB 00:00 2024-03-30T00:23:18.173 INFO:teuthology.orchestra.run.smithi142.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 262 kB/s | 61 kB 00:00 2024-03-30T00:23:18.194 INFO:teuthology.orchestra.run.smithi161.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 3.3 MB/s | 854 kB 00:00 2024-03-30T00:23:18.202 INFO:teuthology.orchestra.run.smithi099.stdout:(125/146): NetworkManager-initscripts-updown-1. 863 kB/s | 144 kB 00:00 2024-03-30T00:23:18.229 INFO:teuthology.orchestra.run.smithi099.stdout:(126/146): elfutils-libelf-devel-0.189-3.el8.x8 492 kB/s | 62 kB 00:00 2024-03-30T00:23:18.277 INFO:teuthology.orchestra.run.smithi099.stdout:(127/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.7 MB/s | 562 kB 00:00 2024-03-30T00:23:18.279 INFO:teuthology.orchestra.run.smithi142.stdout:(67/146): python3-webob-1.8.5-1.el8.1.noarch.rp 705 kB/s | 251 kB 00:00 2024-03-30T00:23:18.316 INFO:teuthology.orchestra.run.smithi142.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 161 kB/s | 23 kB 00:00 2024-03-30T00:23:18.341 INFO:teuthology.orchestra.run.smithi142.stdout:(69/146): python3-webtest-2.0.33-1.el8.noarch.r 464 kB/s | 85 kB 00:00 2024-03-30T00:23:18.344 INFO:teuthology.orchestra.run.smithi161.stdout:(141/146): libfdisk-2.32.1-43.el8.x86_64.rpm 895 kB/s | 254 kB 00:00 2024-03-30T00:23:18.366 INFO:teuthology.orchestra.run.smithi142.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-03-30T00:23:18.394 INFO:teuthology.orchestra.run.smithi099.stdout:(128/146): openssl-devel-1.1.1k-12.el8_9.x86_64 12 MB/s | 2.3 MB 00:00 2024-03-30T00:23:18.416 INFO:teuthology.orchestra.run.smithi161.stdout:(142/146): libblkid-2.32.1-43.el8.x86_64.rpm 567 kB/s | 220 kB 00:00 2024-03-30T00:23:18.419 INFO:teuthology.orchestra.run.smithi099.stdout:(129/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 245 kB/s | 47 kB 00:00 2024-03-30T00:23:18.453 INFO:teuthology.orchestra.run.smithi099.stdout:(130/146): python3-cryptography-3.2.1-7.el8_9.x 3.1 MB/s | 559 kB 00:00 2024-03-30T00:23:18.470 INFO:teuthology.orchestra.run.smithi161.stdout:(143/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 2.5 MB/s | 711 kB 00:00 2024-03-30T00:23:18.508 INFO:teuthology.orchestra.run.smithi142.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 120 kB/s | 20 kB 00:00 2024-03-30T00:23:18.620 INFO:teuthology.orchestra.run.smithi099.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.4 MB/s | 232 kB 00:00 2024-03-30T00:23:18.637 INFO:teuthology.orchestra.run.smithi161.stdout:(144/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 5.0 MB/s | 1.5 MB 00:00 2024-03-30T00:23:18.658 INFO:teuthology.orchestra.run.smithi142.stdout:(72/146): python3-prettytable-0.7.2-14.el8.noar 151 kB/s | 44 kB 00:00 2024-03-30T00:23:18.670 INFO:teuthology.orchestra.run.smithi161.stdout:(145/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.3 MB/s | 333 kB 00:00 2024-03-30T00:23:18.754 INFO:teuthology.orchestra.run.smithi099.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.2 MB/s | 303 kB 00:00 2024-03-30T00:23:18.775 INFO:teuthology.orchestra.run.smithi142.stdout:(73/146): thrift-0.13.0-2.el8.x86_64.rpm 3.5 MB/s | 1.7 MB 00:00 2024-03-30T00:23:18.801 INFO:teuthology.orchestra.run.smithi142.stdout:(74/146): python3-pyasn1-modules-0.3.7-6.el8.no 377 kB/s | 110 kB 00:00 2024-03-30T00:23:18.834 INFO:teuthology.orchestra.run.smithi142.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 151 kB/s | 27 kB 00:00 2024-03-30T00:23:18.912 INFO:teuthology.orchestra.run.smithi161.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 2.4 MB/s | 1.1 MB 00:00 2024-03-30T00:23:18.915 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-03-30T00:23:18.916 INFO:teuthology.orchestra.run.smithi161.stdout:Total 9.6 MB/s | 213 MB 00:22 2024-03-30T00:23:18.937 INFO:teuthology.orchestra.run.smithi099.stdout:(133/146): librados2-18.2.2-1018.g470100a4.el8. 6.8 MB/s | 3.7 MB 00:00 2024-03-30T00:23:18.971 INFO:teuthology.orchestra.run.smithi099.stdout:(134/146): libmount-2.32.1-43.el8.x86_64.rpm 1.1 MB/s | 237 kB 00:00 2024-03-30T00:23:18.976 INFO:teuthology.orchestra.run.smithi142.stdout:(76/146): bison-3.0.4-10.el8.x86_64.rpm 4.8 MB/s | 688 kB 00:00 2024-03-30T00:23:19.018 INFO:teuthology.orchestra.run.smithi142.stdout:(77/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.1 MB/s | 457 kB 00:00 2024-03-30T00:23:19.043 INFO:teuthology.orchestra.run.smithi142.stdout:(78/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 473 kB/s | 126 kB 00:00 2024-03-30T00:23:19.077 INFO:teuthology.orchestra.run.smithi142.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 3.1 MB/s | 320 kB 00:00 2024-03-30T00:23:19.102 INFO:teuthology.orchestra.run.smithi099.stdout:(135/146): librbd1-18.2.2-1018.g470100a4.el8.x8 5.2 MB/s | 3.6 MB 00:00 2024-03-30T00:23:19.130 INFO:teuthology.orchestra.run.smithi099.stdout:(136/146): libuuid-2.32.1-43.el8.x86_64.rpm 625 kB/s | 99 kB 00:00 2024-03-30T00:23:19.144 INFO:teuthology.orchestra.run.smithi142.stdout:(80/146): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2024-03-30T00:23:19.155 INFO:teuthology.orchestra.run.smithi099.stdout:(137/146): libsmartcols-2.32.1-43.el8.x86_64.rp 826 kB/s | 179 kB 00:00 2024-03-30T00:23:19.169 INFO:teuthology.orchestra.run.smithi142.stdout:(81/146): python3-markupsafe-0.23-19.el8.x86_64 309 kB/s | 39 kB 00:00 2024-03-30T00:23:19.194 INFO:teuthology.orchestra.run.smithi142.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2024-03-30T00:23:19.256 INFO:teuthology.orchestra.run.smithi099.stdout:(138/146): util-linux-user-2.32.1-43.el8.x86_64 820 kB/s | 103 kB 00:00 2024-03-30T00:23:19.262 INFO:teuthology.orchestra.run.smithi142.stdout:(83/146): uuid-1.6.2-43.el8.x86_64.rpm 546 kB/s | 64 kB 00:00 2024-03-30T00:23:19.339 INFO:teuthology.orchestra.run.smithi099.stdout:(139/146): util-linux-2.32.1-43.el8.x86_64.rpm 10 MB/s | 2.5 MB 00:00 2024-03-30T00:23:19.345 INFO:teuthology.orchestra.run.smithi142.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 590 kB/s | 103 kB 00:00 2024-03-30T00:23:19.381 INFO:teuthology.orchestra.run.smithi099.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 3.7 MB/s | 854 kB 00:00 2024-03-30T00:23:19.406 INFO:teuthology.orchestra.run.smithi099.stdout:(141/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-03-30T00:23:19.412 INFO:teuthology.orchestra.run.smithi142.stdout:(85/146): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2024-03-30T00:23:19.462 INFO:teuthology.orchestra.run.smithi142.stdout:(86/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.6 MB/s | 538 kB 00:00 2024-03-30T00:23:19.482 INFO:teuthology.orchestra.run.smithi099.stdout:(142/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 6.9 MB/s | 711 kB 00:00 2024-03-30T00:23:19.507 INFO:teuthology.orchestra.run.smithi099.stdout:(143/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.5 MB/s | 254 kB 00:00 2024-03-30T00:23:19.579 INFO:teuthology.orchestra.run.smithi142.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 16 MB/s | 3.7 MB 00:00 2024-03-30T00:23:19.599 INFO:teuthology.orchestra.run.smithi099.stdout:(144/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.6 MB/s | 1.5 MB 00:00 2024-03-30T00:23:19.613 INFO:teuthology.orchestra.run.smithi142.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2024-03-30T00:23:19.682 INFO:teuthology.orchestra.run.smithi099.stdout:(145/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.1 MB/s | 1.1 MB 00:00 2024-03-30T00:23:19.707 INFO:teuthology.orchestra.run.smithi099.stdout:(146/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.4 MB/s | 333 kB 00:00 2024-03-30T00:23:19.710 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-03-30T00:23:19.711 INFO:teuthology.orchestra.run.smithi099.stdout:Total 11 MB/s | 213 MB 00:19 2024-03-30T00:23:19.810 INFO:teuthology.orchestra.run.smithi142.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 649 kB/s | 225 kB 00:00 2024-03-30T00:23:19.980 INFO:teuthology.orchestra.run.smithi142.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 883 kB/s | 354 kB 00:00 2024-03-30T00:23:20.006 INFO:teuthology.orchestra.run.smithi142.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 803 kB/s | 157 kB 00:00 2024-03-30T00:23:20.131 INFO:teuthology.orchestra.run.smithi142.stdout:(92/146): automake-1.16.1-8.el8.noarch.rpm 5.6 MB/s | 714 kB 00:00 2024-03-30T00:23:20.223 INFO:teuthology.orchestra.run.smithi142.stdout:(93/146): openblas-threads-0.3.15-6.el8.x86_64. 8.5 MB/s | 5.2 MB 00:00 2024-03-30T00:23:20.273 INFO:teuthology.orchestra.run.smithi142.stdout:(94/146): libpq-13.11-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2024-03-30T00:23:20.407 INFO:teuthology.orchestra.run.smithi142.stdout:(95/146): openblas-0.3.15-6.el8.x86_64.rpm 12 MB/s | 5.0 MB 00:00 2024-03-30T00:23:20.440 INFO:teuthology.orchestra.run.smithi142.stdout:(96/146): python3-pytz-2017.2-11.el8.noarch.rpm 325 kB/s | 54 kB 00:00 2024-03-30T00:23:20.465 INFO:teuthology.orchestra.run.smithi142.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 836 kB/s | 202 kB 00:00 2024-03-30T00:23:20.566 INFO:teuthology.orchestra.run.smithi142.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-03-30T00:23:20.624 INFO:teuthology.orchestra.run.smithi142.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 9.5 MB/s | 1.5 MB 00:00 2024-03-30T00:23:20.733 INFO:teuthology.orchestra.run.smithi142.stdout:(100/146): postgresql-contrib-10.23-4.module+el 4.8 MB/s | 812 kB 00:00 2024-03-30T00:23:20.892 INFO:teuthology.orchestra.run.smithi142.stdout:(101/146): postgresql-server-10.23-4.module+el8 19 MB/s | 5.1 MB 00:00 2024-03-30T00:23:20.908 INFO:teuthology.orchestra.run.smithi142.stdout:(102/146): python3-packaging-16.8-9.el8.noarch. 276 kB/s | 48 kB 00:00 2024-03-30T00:23:21.038 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-03-30T00:23:21.051 INFO:teuthology.orchestra.run.smithi142.stdout:(103/146): python3-scipy-1.0.0-21.module+el8.9. 21 MB/s | 14 MB 00:00 2024-03-30T00:23:21.117 INFO:teuthology.orchestra.run.smithi142.stdout:(104/146): libverto-devel-0.3.0-5.el8.x86_64.rp 79 kB/s | 18 kB 00:00 2024-03-30T00:23:21.151 INFO:teuthology.orchestra.run.smithi142.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 176 kB/s | 43 kB 00:00 2024-03-30T00:23:21.293 INFO:teuthology.orchestra.run.smithi142.stdout:(106/146): python3-requests-oauthlib-1.0.0-1.el 301 kB/s | 43 kB 00:00 2024-03-30T00:23:21.318 INFO:teuthology.orchestra.run.smithi142.stdout:(107/146): python3-oauthlib-2.1.0-1.el8.noarch. 580 kB/s | 155 kB 00:00 2024-03-30T00:23:21.353 INFO:teuthology.orchestra.run.smithi142.stdout:(108/146): python3-pycparser-2.14-14.el8.noarch 465 kB/s | 109 kB 00:00 2024-03-30T00:23:21.419 INFO:teuthology.orchestra.run.smithi142.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 149 kB/s | 19 kB 00:00 2024-03-30T00:23:21.481 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-03-30T00:23:21.481 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-03-30T00:23:21.486 INFO:teuthology.orchestra.run.smithi142.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 262 kB/s | 44 kB 00:00 2024-03-30T00:23:21.528 INFO:teuthology.orchestra.run.smithi142.stdout:(111/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2024-03-30T00:23:21.561 INFO:teuthology.orchestra.run.smithi142.stdout:(112/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2024-03-30T00:23:21.637 INFO:teuthology.orchestra.run.smithi142.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-03-30T00:23:21.662 INFO:teuthology.orchestra.run.smithi142.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.5 MB/s | 200 kB 00:00 2024-03-30T00:23:21.769 INFO:teuthology.orchestra.run.smithi142.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 232 kB/s | 48 kB 00:00 2024-03-30T00:23:21.821 INFO:teuthology.orchestra.run.smithi142.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 364 kB/s | 58 kB 00:00 2024-03-30T00:23:21.823 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-03-30T00:23:21.846 INFO:teuthology.orchestra.run.smithi142.stdout:(117/146): libsepol-devel-2.9-3.el8.x86_64.rpm 416 kB/s | 87 kB 00:00 2024-03-30T00:23:21.871 INFO:teuthology.orchestra.run.smithi142.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 378 kB/s | 39 kB 00:00 2024-03-30T00:23:21.938 INFO:teuthology.orchestra.run.smithi142.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 709 kB/s | 83 kB 00:00 2024-03-30T00:23:21.972 INFO:teuthology.orchestra.run.smithi142.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.7 MB/s | 172 kB 00:00 2024-03-30T00:23:22.039 INFO:teuthology.orchestra.run.smithi142.stdout:(121/146): libuuid-devel-2.32.1-43.el8.x86_64.r 997 kB/s | 100 kB 00:00 2024-03-30T00:23:22.073 INFO:teuthology.orchestra.run.smithi142.stdout:(122/146): libgfortran-8.5.0-20.el8.x86_64.rpm 2.8 MB/s | 645 kB 00:00 2024-03-30T00:23:22.140 INFO:teuthology.orchestra.run.smithi142.stdout:(123/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.4 MB/s | 238 kB 00:00 2024-03-30T00:23:22.165 INFO:teuthology.orchestra.run.smithi142.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 578 kB/s | 73 kB 00:00 2024-03-30T00:23:22.198 INFO:teuthology.orchestra.run.smithi142.stdout:(125/146): NetworkManager-initscripts-updown-1. 1.1 MB/s | 144 kB 00:00 2024-03-30T00:23:22.265 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-03-30T00:23:22.265 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-03-30T00:23:22.265 INFO:teuthology.orchestra.run.smithi142.stdout:(126/146): elfutils-libelf-devel-0.189-3.el8.x8 493 kB/s | 62 kB 00:00 2024-03-30T00:23:22.341 INFO:teuthology.orchestra.run.smithi142.stdout:(127/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.1 MB/s | 562 kB 00:00 2024-03-30T00:23:22.383 INFO:teuthology.orchestra.run.smithi142.stdout:(128/146): openssl-devel-1.1.1k-12.el8_9.x86_64 13 MB/s | 2.3 MB 00:00 2024-03-30T00:23:22.441 INFO:teuthology.orchestra.run.smithi142.stdout:(129/146): python3-cryptography-3.2.1-7.el8_9.x 5.4 MB/s | 559 kB 00:00 2024-03-30T00:23:22.475 INFO:teuthology.orchestra.run.smithi142.stdout:(130/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 225 kB/s | 47 kB 00:00 2024-03-30T00:23:22.584 INFO:teuthology.orchestra.run.smithi142.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 2.1 MB/s | 232 kB 00:00 2024-03-30T00:23:22.759 INFO:teuthology.orchestra.run.smithi142.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.7 MB/s | 303 kB 00:00 2024-03-30T00:23:22.901 INFO:teuthology.orchestra.run.smithi142.stdout:(133/146): libmount-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 237 kB 00:00 2024-03-30T00:23:23.050 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-03-30T00:23:23.059 INFO:teuthology.orchestra.run.smithi142.stdout:(134/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.1 MB/s | 179 kB 00:00 2024-03-30T00:23:23.063 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-03-30T00:23:23.226 INFO:teuthology.orchestra.run.smithi142.stdout:(135/146): librados2-18.2.2-1018.g470100a4.el8. 4.3 MB/s | 3.7 MB 00:00 2024-03-30T00:23:23.251 INFO:teuthology.orchestra.run.smithi142.stdout:(136/146): libuuid-2.32.1-43.el8.x86_64.rpm 516 kB/s | 99 kB 00:00 2024-03-30T00:23:23.319 INFO:teuthology.orchestra.run.smithi142.stdout:(137/146): librbd1-18.2.2-1018.g470100a4.el8.x8 4.1 MB/s | 3.6 MB 00:00 2024-03-30T00:23:23.402 INFO:teuthology.orchestra.run.smithi142.stdout:(138/146): util-linux-user-2.32.1-43.el8.x86_64 682 kB/s | 103 kB 00:00 2024-03-30T00:23:23.469 INFO:teuthology.orchestra.run.smithi142.stdout:(139/146): util-linux-2.32.1-43.el8.x86_64.rpm 10 MB/s | 2.5 MB 00:00 2024-03-30T00:23:23.519 INFO:teuthology.orchestra.run.smithi142.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.2 MB/s | 854 kB 00:00 2024-03-30T00:23:23.595 INFO:teuthology.orchestra.run.smithi142.stdout:(141/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2024-03-30T00:23:23.628 INFO:teuthology.orchestra.run.smithi142.stdout:(142/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 254 kB 00:00 2024-03-30T00:23:23.704 INFO:teuthology.orchestra.run.smithi142.stdout:(143/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.8 MB/s | 711 kB 00:00 2024-03-30T00:23:23.762 INFO:teuthology.orchestra.run.smithi142.stdout:(144/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.4 MB/s | 333 kB 00:00 2024-03-30T00:23:23.804 INFO:teuthology.orchestra.run.smithi142.stdout:(145/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 7.0 MB/s | 1.5 MB 00:00 2024-03-30T00:23:23.854 INFO:teuthology.orchestra.run.smithi142.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.1 MB/s | 1.1 MB 00:00 2024-03-30T00:23:23.857 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-03-30T00:23:23.857 INFO:teuthology.orchestra.run.smithi142.stdout:Total 7.9 MB/s | 213 MB 00:27 2024-03-30T00:23:23.859 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-03-30T00:23:23.872 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-03-30T00:23:25.967 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-03-30T00:23:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-03-30T00:23:26.413 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-03-30T00:23:28.011 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-03-30T00:23:28.023 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-03-30T00:23:29.043 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-03-30T00:23:29.228 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-30T00:23:29.497 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T00:23:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T00:23:29.844 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-03-30T00:23:29.990 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-30T00:23:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-30T00:23:30.154 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T00:23:30.278 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T00:23:30.287 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T00:23:30.453 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T00:23:30.546 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T00:23:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T00:23:30.757 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-30T00:23:30.788 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-30T00:23:30.942 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T00:23:30.978 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T00:23:31.060 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T00:23:31.064 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T00:23:31.250 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T00:23:31.321 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-30T00:23:31.342 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T00:23:31.520 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T00:23:31.593 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-30T00:23:31.802 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T00:23:31.890 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T00:23:31.986 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T00:23:32.143 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-30T00:23:32.222 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T00:23:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T00:23:32.660 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T00:23:32.756 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T00:23:32.878 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T00:23:32.986 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T00:23:33.031 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T00:23:33.211 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T00:23:33.351 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T00:23:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T00:23:33.649 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T00:23:33.665 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-03-30T00:23:33.696 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T00:23:33.802 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T00:23:33.858 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-30T00:23:34.003 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T00:23:34.024 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T00:23:34.112 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T00:23:34.114 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T00:23:34.170 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T00:23:34.352 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-30T00:23:34.381 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T00:23:34.602 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-30T00:23:34.627 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T00:23:34.628 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-30T00:23:34.766 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T00:23:34.833 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-30T00:23:34.843 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T00:23:34.966 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T00:23:35.057 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T00:23:35.075 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T00:23:35.147 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T00:23:35.253 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-30T00:23:35.356 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-30T00:23:35.358 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-30T00:23:35.478 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-30T00:23:35.565 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T00:23:35.636 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T00:23:35.653 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T00:23:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-30T00:23:35.767 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T00:23:35.873 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-30T00:23:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-30T00:23:35.969 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T00:23:36.066 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T00:23:36.090 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T00:23:36.235 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T00:23:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-30T00:23:36.402 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T00:23:36.547 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T00:23:36.552 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T00:23:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-30T00:23:36.764 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T00:23:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T00:23:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T00:23:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T00:23:37.199 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T00:23:37.239 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T00:23:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T00:23:37.373 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T00:23:37.427 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T00:23:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T00:23:37.531 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T00:23:37.652 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T00:23:37.707 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T00:23:37.817 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T00:23:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T00:23:38.093 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T00:23:38.182 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T00:23:38.277 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T00:23:38.473 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T00:23:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T00:23:38.587 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T00:23:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T00:23:38.795 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-30T00:23:39.086 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T00:23:39.123 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-30T00:23:39.326 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-30T00:23:39.345 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-30T00:23:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T00:23:39.536 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-30T00:23:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T00:23:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-30T00:23:39.765 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-30T00:23:39.995 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-30T00:23:40.065 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-30T00:23:40.123 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T00:23:40.236 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T00:23:40.389 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T00:23:40.396 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-30T00:23:40.409 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T00:23:40.494 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T00:23:40.663 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-30T00:23:40.664 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T00:23:40.697 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-30T00:23:40.844 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T00:23:40.892 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 31/162 2024-03-30T00:23:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-30T00:23:40.998 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T00:23:41.023 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 32/162 2024-03-30T00:23:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-30T00:23:41.321 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 33/162 2024-03-30T00:23:41.465 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-30T00:23:41.665 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T00:23:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-30T00:23:41.814 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T00:23:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-30T00:23:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 31/162 2024-03-30T00:23:42.376 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-30T00:23:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 32/162 2024-03-30T00:23:42.480 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T00:23:42.673 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T00:23:42.721 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 33/162 2024-03-30T00:23:42.745 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-30T00:23:43.077 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-30T00:23:43.293 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T00:23:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T00:23:43.763 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-30T00:23:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-30T00:23:43.923 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-30T00:23:44.104 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-30T00:23:44.241 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-30T00:23:44.401 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-30T00:23:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-30T00:23:44.706 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-30T00:23:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T00:23:44.967 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-30T00:23:45.138 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 31/162 2024-03-30T00:23:45.260 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 32/162 2024-03-30T00:23:45.325 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T00:23:45.550 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 33/162 2024-03-30T00:23:45.667 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-30T00:23:45.912 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-30T00:23:46.173 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-30T00:23:46.374 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 41/162 2024-03-30T00:23:46.592 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 42/162 2024-03-30T00:23:46.628 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-30T00:23:46.809 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T00:23:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T00:23:46.950 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T00:23:47.016 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-30T00:23:47.167 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T00:23:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-30T00:23:47.265 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T00:23:47.290 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-30T00:23:47.462 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-30T00:23:47.522 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T00:23:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-30T00:23:47.639 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-30T00:23:47.810 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-30T00:23:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T00:23:48.037 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 41/162 2024-03-30T00:23:48.275 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 42/162 2024-03-30T00:23:48.426 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T00:23:48.534 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T00:23:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T00:23:48.768 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T00:23:48.802 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T00:23:48.896 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T00:23:48.973 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-03-30T00:23:48.991 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T00:23:49.225 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-30T00:23:49.226 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-03-30T00:23:49.432 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-03-30T00:23:49.443 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-30T00:23:49.575 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T00:23:49.647 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-03-30T00:23:49.679 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T00:23:49.838 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-03-30T00:23:49.904 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-30T00:23:50.107 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-30T00:23:50.164 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-03-30T00:23:50.282 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T00:23:50.402 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-30T00:23:50.409 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-03-30T00:23:50.648 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-03-30T00:23:50.680 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T00:23:50.713 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T00:23:50.819 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-03-30T00:23:50.862 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 41/162 2024-03-30T00:23:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-03-30T00:23:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 42/162 2024-03-30T00:23:51.060 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-03-30T00:23:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-03-30T00:23:51.286 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T00:23:51.378 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-03-30T00:23:51.388 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T00:23:51.527 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-03-30T00:23:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-03-30T00:23:51.618 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T00:23:51.727 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T00:23:51.826 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-03-30T00:23:51.865 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-03-30T00:23:51.908 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-30T00:23:52.143 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-30T00:23:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-03-30T00:23:52.172 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-03-30T00:23:52.306 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T00:23:52.426 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-03-30T00:23:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-03-30T00:23:52.665 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-03-30T00:23:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-03-30T00:23:52.924 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T00:23:52.935 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-03-30T00:23:52.956 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-03-30T00:23:53.209 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-03-30T00:23:53.214 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-03-30T00:23:53.230 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T00:23:53.262 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T00:23:53.426 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-03-30T00:23:53.461 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-03-30T00:23:53.639 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-03-30T00:23:53.669 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-03-30T00:23:53.675 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-03-30T00:23:53.837 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-03-30T00:23:53.890 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-03-30T00:23:54.035 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-03-30T00:23:54.042 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-03-30T00:23:54.163 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-03-30T00:23:54.227 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-03-30T00:23:54.325 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-30T00:23:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-03-30T00:23:54.421 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-30T00:23:54.540 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-03-30T00:23:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-03-30T00:23:54.717 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-03-30T00:23:54.797 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-03-30T00:23:54.951 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-03-30T00:23:54.987 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-03-30T00:23:55.060 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-03-30T00:23:55.182 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-03-30T00:23:55.213 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-03-30T00:23:55.214 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-03-30T00:23:55.273 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-03-30T00:23:55.466 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-30T00:23:55.466 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-03-30T00:23:55.546 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-03-30T00:23:55.708 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-03-30T00:23:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-03-30T00:23:55.900 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-03-30T00:23:55.921 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-30T00:23:56.030 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-03-30T00:23:56.169 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-03-30T00:23:56.202 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-03-30T00:23:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-03-30T00:23:56.351 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-30T00:23:56.553 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-03-30T00:23:56.558 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-03-30T00:23:56.590 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-30T00:23:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-30T00:23:56.788 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-03-30T00:23:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-30T00:23:56.903 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-30T00:23:57.003 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-03-30T00:23:57.133 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-30T00:23:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-03-30T00:23:57.235 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-03-30T00:23:57.305 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-30T00:23:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-03-30T00:23:57.513 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-30T00:23:57.522 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-03-30T00:23:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-03-30T00:23:57.724 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-03-30T00:23:57.742 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-03-30T00:23:57.757 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-30T00:23:57.948 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-03-30T00:23:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-30T00:23:58.009 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-30T00:23:58.164 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-03-30T00:23:58.215 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-30T00:23:58.227 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-03-30T00:23:58.437 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-30T00:23:58.461 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-03-30T00:23:58.499 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-30T00:23:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-30T00:23:58.634 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-16.8-9.el8.noarch 70/162 2024-03-30T00:23:58.722 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-03-30T00:23:58.812 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-03-30T00:23:58.907 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-30T00:23:58.919 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-03-30T00:23:58.939 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-30T00:23:59.113 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-30T00:23:59.194 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-30T00:23:59.220 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-03-30T00:23:59.339 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-30T00:23:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-03-30T00:23:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-30T00:23:59.637 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-30T00:23:59.670 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-03-30T00:23:59.761 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-03-30T00:23:59.802 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-30T00:23:59.866 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-30T00:23:59.954 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-03-30T00:24:00.092 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-30T00:24:00.162 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-03-30T00:24:00.300 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-30T00:24:00.331 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-30T00:24:00.356 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-03-30T00:24:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-30T00:24:00.514 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-30T00:24:00.763 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-30T00:24:00.767 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-30T00:24:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-30T00:24:01.058 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-30T00:24:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-30T00:24:01.295 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-30T00:24:01.456 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-30T00:24:01.467 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-30T00:24:01.635 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-30T00:24:01.659 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-30T00:24:01.675 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-30T00:24:01.885 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-30T00:24:01.924 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-30T00:24:01.951 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-30T00:24:02.104 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-30T00:24:02.166 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-30T00:24:02.202 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-30T00:24:02.277 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-30T00:24:02.358 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-30T00:24:02.432 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-30T00:24:02.559 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-30T00:24:02.618 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-30T00:24:02.654 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-30T00:24:02.843 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-30T00:24:02.855 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-30T00:24:02.936 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-30T00:24:03.076 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-30T00:24:03.079 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-30T00:24:03.187 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-30T00:24:03.244 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-30T00:24:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-30T00:24:03.517 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-30T00:24:03.753 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-30T00:24:03.913 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-30T00:24:03.980 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-30T00:24:04.106 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T00:24:04.199 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T00:24:04.235 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-30T00:24:04.749 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 104/162 2024-03-30T00:24:04.825 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-30T00:24:05.154 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-30T00:24:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-30T00:24:05.570 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-30T00:24:06.015 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-30T00:24:06.183 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-30T00:24:06.233 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-30T00:24:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-30T00:24:06.583 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-30T00:24:06.780 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-30T00:24:06.857 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-30T00:24:07.058 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-30T00:24:07.648 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-30T00:24:07.851 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-30T00:24:07.858 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T00:24:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T00:24:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T00:24:08.097 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T00:24:08.184 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 104/162 2024-03-30T00:24:08.311 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 104/162 2024-03-30T00:24:09.677 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T00:24:10.489 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T00:24:11.186 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T00:24:11.457 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T00:24:11.613 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T00:24:11.755 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T00:24:13.160 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T00:24:13.377 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T00:24:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T00:24:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T00:24:14.099 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T00:24:14.333 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T00:24:14.481 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T00:24:14.617 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T00:24:14.802 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T00:24:15.144 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T00:24:15.270 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T00:24:15.465 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T00:24:32.586 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T00:24:32.586 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-03-30T00:24:32.586 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-03-30T00:24:32.586 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2024-03-30T00:24:32.586 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /run 2024-03-30T00:24:32.586 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-03-30T00:24:32.586 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-03-30T00:24:32.586 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-03-30T00:24:32.587 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-03-30T00:24:32.587 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2024-03-30T00:24:32.587 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:24:33.002 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T00:24:34.754 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T00:24:34.962 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T00:24:35.227 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T00:24:35.385 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T00:24:35.545 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T00:24:35.545 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-03-30T00:24:35.545 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-03-30T00:24:35.545 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /dev 2024-03-30T00:24:35.545 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /run 2024-03-30T00:24:35.545 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-03-30T00:24:35.545 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-03-30T00:24:35.545 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-03-30T00:24:35.546 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-03-30T00:24:35.546 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /dev 2024-03-30T00:24:35.546 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:24:35.923 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T00:24:35.931 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T00:24:36.093 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T00:24:36.136 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T00:24:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T00:24:36.328 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-03-30T00:24:36.328 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-03-30T00:24:36.328 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2024-03-30T00:24:36.328 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /run 2024-03-30T00:24:36.328 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-03-30T00:24:36.328 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-03-30T00:24:36.328 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-03-30T00:24:36.328 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-03-30T00:24:36.328 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2024-03-30T00:24:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:24:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T00:24:37.692 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T00:24:37.888 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T00:24:38.152 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T00:24:38.289 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T00:24:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T00:24:38.781 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T00:24:38.827 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T00:24:38.970 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T00:24:39.014 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T00:24:39.045 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T00:24:39.057 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T00:24:39.148 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T00:24:39.258 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 113/162 2024-03-30T00:24:39.268 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T00:24:39.476 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 114/162 2024-03-30T00:24:39.715 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T00:24:39.806 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T00:24:39.989 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T00:24:39.993 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T00:24:40.036 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T00:24:40.252 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T00:24:41.331 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T00:24:41.516 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T00:24:41.648 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T00:24:41.873 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T00:24:42.024 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T00:24:42.042 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 118/162 2024-03-30T00:24:42.179 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 113/162 2024-03-30T00:24:42.211 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T00:24:42.280 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T00:24:42.353 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 114/162 2024-03-30T00:24:42.475 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T00:24:42.505 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T00:24:42.571 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T00:24:42.825 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T00:24:42.841 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T00:24:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T00:24:43.145 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T00:24:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 113/162 2024-03-30T00:24:43.433 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 114/162 2024-03-30T00:24:43.720 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T00:24:43.835 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 121/162 2024-03-30T00:24:43.986 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T00:24:44.035 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T00:24:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T00:24:44.222 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T00:24:44.229 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T00:24:44.317 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T00:24:44.387 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T00:24:44.390 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T00:24:44.518 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T00:24:44.558 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T00:24:44.912 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 118/162 2024-03-30T00:24:44.959 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T00:24:45.112 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T00:24:45.181 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T00:24:45.190 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T00:24:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T00:24:45.329 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T00:24:45.357 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T00:24:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T00:24:45.670 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T00:24:46.114 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 118/162 2024-03-30T00:24:46.312 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T00:24:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T00:24:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T00:24:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T00:24:46.722 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 121/162 2024-03-30T00:24:46.945 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T00:24:47.006 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T00:24:47.170 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T00:24:47.241 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T00:24:47.445 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T00:24:47.845 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T00:24:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 121/162 2024-03-30T00:24:48.022 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T00:24:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T00:24:48.241 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T00:24:48.483 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T00:24:48.555 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T00:24:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T00:24:49.160 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T00:24:49.371 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T00:24:50.012 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T00:24:50.557 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 126/162 2024-03-30T00:24:50.915 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-30T00:24:51.132 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T00:24:52.150 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T00:24:52.429 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-30T00:24:52.720 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-30T00:24:52.837 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T00:24:52.956 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T00:24:53.171 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-30T00:24:53.369 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 126/162 2024-03-30T00:24:53.404 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-30T00:24:53.633 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 134/162 2024-03-30T00:24:53.718 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-30T00:24:53.902 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T00:24:54.178 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T00:24:54.734 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 126/162 2024-03-30T00:24:54.934 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T00:24:55.096 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-30T00:24:55.233 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-30T00:24:55.331 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T00:24:55.522 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-30T00:24:55.739 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T00:24:55.983 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-30T00:24:56.165 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-30T00:24:56.344 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T00:24:56.386 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 134/162 2024-03-30T00:24:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-30T00:24:57.003 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-30T00:24:57.038 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T00:24:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T00:24:57.374 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T00:24:57.454 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T00:24:57.553 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-30T00:24:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-30T00:24:57.850 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-30T00:24:58.056 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 134/162 2024-03-30T00:24:58.161 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-30T00:24:58.411 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-30T00:24:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 139/162 2024-03-30T00:24:58.825 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 140/162 2024-03-30T00:24:59.161 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 141/162 2024-03-30T00:24:59.441 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-30T00:24:59.664 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T00:24:59.691 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T00:24:59.777 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T00:24:59.892 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-30T00:25:00.085 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T00:25:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T00:25:00.155 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T00:25:00.253 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T00:25:00.489 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T00:25:00.520 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T00:25:00.546 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-30T00:25:00.695 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-30T00:25:00.904 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-30T00:25:00.942 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-30T00:25:01.069 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-30T00:25:01.069 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T00:25:01.215 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-30T00:25:01.393 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 139/162 2024-03-30T00:25:01.410 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T00:25:01.411 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T00:25:01.456 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T00:25:01.636 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 140/162 2024-03-30T00:25:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T00:25:01.638 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T00:25:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T00:25:01.881 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T00:25:01.937 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 141/162 2024-03-30T00:25:01.950 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T00:25:02.182 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-30T00:25:02.236 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-30T00:25:02.310 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-30T00:25:02.348 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-30T00:25:02.409 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T00:25:02.436 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T00:25:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-30T00:25:02.566 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T00:25:02.642 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-30T00:25:02.731 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-30T00:25:02.825 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T00:25:02.825 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T00:25:02.845 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T00:25:02.966 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T00:25:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-30T00:25:03.109 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T00:25:03.109 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T00:25:03.151 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T00:25:03.180 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T00:25:03.289 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 139/162 2024-03-30T00:25:03.354 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-30T00:25:03.367 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T00:25:03.367 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T00:25:03.549 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-30T00:25:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 140/162 2024-03-30T00:25:03.650 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T00:25:03.650 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T00:25:03.713 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-30T00:25:03.713 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T00:25:03.915 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 141/162 2024-03-30T00:25:03.942 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T00:25:04.030 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T00:25:04.030 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T00:25:04.108 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-30T00:25:04.109 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T00:25:04.231 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-30T00:25:04.265 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T00:25:04.266 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T00:25:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T00:25:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T00:25:04.544 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T00:25:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-30T00:25:04.713 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T00:25:04.713 INFO:teuthology.orchestra.run.smithi161.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-30T00:25:04.713 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:25:04.761 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-30T00:25:04.984 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-30T00:25:05.032 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T00:25:05.152 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T00:25:05.221 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-30T00:25:05.222 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T00:25:05.387 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T00:25:05.418 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T00:25:05.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 1/162 2024-03-30T00:25:05.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2/162 2024-03-30T00:25:05.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 3/162 2024-03-30T00:25:05.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 4/162 2024-03-30T00:25:05.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1018.g47010 5/162 2024-03-30T00:25:05.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 6/162 2024-03-30T00:25:05.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 7/162 2024-03-30T00:25:05.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 8/162 2024-03-30T00:25:05.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T00:25:05.435 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 13/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 14/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 15/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 16/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 17/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 18/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 20/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 21/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 22/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 23/162 2024-03-30T00:25:05.436 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 24/162 2024-03-30T00:25:05.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 25/162 2024-03-30T00:25:05.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 26/162 2024-03-30T00:25:05.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 27/162 2024-03-30T00:25:05.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 28/162 2024-03-30T00:25:05.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 29/162 2024-03-30T00:25:05.437 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 30/162 2024-03-30T00:25:05.452 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 31/162 2024-03-30T00:25:05.452 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 32/162 2024-03-30T00:25:05.452 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 33/162 2024-03-30T00:25:05.452 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 34/162 2024-03-30T00:25:05.452 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 35/162 2024-03-30T00:25:05.452 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 36/162 2024-03-30T00:25:05.452 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 37/162 2024-03-30T00:25:05.452 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 38/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-30T00:25:05.453 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-30T00:25:05.454 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-30T00:25:05.454 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-30T00:25:05.454 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-30T00:25:05.454 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-30T00:25:05.454 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-30T00:25:05.454 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-30T00:25:05.454 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-30T00:25:05.454 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-30T00:25:05.454 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-30T00:25:05.454 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-30T00:25:05.455 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-30T00:25:05.455 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-30T00:25:05.455 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-30T00:25:05.455 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-30T00:25:05.455 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-30T00:25:05.455 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-30T00:25:05.455 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-30T00:25:05.455 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-30T00:25:05.455 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-30T00:25:05.455 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-03-30T00:25:05.455 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-03-30T00:25:05.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-03-30T00:25:05.457 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-03-30T00:25:05.457 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-03-30T00:25:05.457 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-03-30T00:25:05.457 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-03-30T00:25:05.457 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-03-30T00:25:05.457 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-03-30T00:25:05.457 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-03-30T00:25:05.457 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-03-30T00:25:05.457 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-03-30T00:25:05.458 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-03-30T00:25:05.458 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-03-30T00:25:05.458 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-03-30T00:25:05.458 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-03-30T00:25:05.458 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-03-30T00:25:05.458 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-03-30T00:25:05.458 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-03-30T00:25:05.458 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-03-30T00:25:05.459 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-03-30T00:25:05.459 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-03-30T00:25:05.459 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-03-30T00:25:05.459 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-03-30T00:25:05.459 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-03-30T00:25:05.460 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-03-30T00:25:05.461 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/162 2024-03-30T00:25:05.461 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/162 2024-03-30T00:25:05.461 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 124/162 2024-03-30T00:25:05.461 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 125/162 2024-03-30T00:25:05.461 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 126/162 2024-03-30T00:25:05.461 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 127/162 2024-03-30T00:25:05.461 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/162 2024-03-30T00:25:05.461 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 129/162 2024-03-30T00:25:05.461 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 130/162 2024-03-30T00:25:05.461 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 133/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-30T00:25:05.462 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-30T00:25:05.463 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-30T00:25:05.463 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-30T00:25:05.463 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-30T00:25:05.463 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-30T00:25:05.463 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-30T00:25:05.463 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-30T00:25:05.463 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-30T00:25:05.463 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-30T00:25:05.463 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-30T00:25:05.463 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-30T00:25:05.463 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-30T00:25:05.464 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-30T00:25:05.464 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-30T00:25:05.464 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-30T00:25:05.464 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-30T00:25:05.464 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-30T00:25:05.464 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-30T00:25:05.464 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-30T00:25:05.471 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T00:25:05.471 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T00:25:05.630 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-30T00:25:05.754 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T00:25:05.754 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T00:25:05.833 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-30T00:25:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T00:25:06.004 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T00:25:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-30T00:25:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T00:25:06.287 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T00:25:06.287 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T00:25:06.342 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T00:25:06.342 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T00:25:06.521 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T00:25:06.619 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T00:25:06.619 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T00:25:06.722 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-30T00:25:06.722 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T00:25:06.903 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T00:25:07.181 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-30T00:25:07.284 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T00:25:07.284 INFO:teuthology.orchestra.run.smithi142.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-30T00:25:07.285 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:25:07.386 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-30T00:25:07.689 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-30T00:25:07.689 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T00:25:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T00:25:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T00:25:07.977 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 1/162 2024-03-30T00:25:07.977 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2/162 2024-03-30T00:25:07.977 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 3/162 2024-03-30T00:25:07.978 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 4/162 2024-03-30T00:25:07.978 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1018.g47010 5/162 2024-03-30T00:25:07.978 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 6/162 2024-03-30T00:25:07.978 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 7/162 2024-03-30T00:25:07.978 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 8/162 2024-03-30T00:25:07.978 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T00:25:07.978 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T00:25:07.978 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T00:25:07.979 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T00:25:07.979 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 13/162 2024-03-30T00:25:07.979 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 14/162 2024-03-30T00:25:07.979 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 15/162 2024-03-30T00:25:07.979 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 16/162 2024-03-30T00:25:07.979 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 17/162 2024-03-30T00:25:07.979 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 18/162 2024-03-30T00:25:07.979 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T00:25:07.980 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 20/162 2024-03-30T00:25:07.980 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 21/162 2024-03-30T00:25:07.980 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 22/162 2024-03-30T00:25:07.980 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 23/162 2024-03-30T00:25:07.980 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 24/162 2024-03-30T00:25:07.980 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 25/162 2024-03-30T00:25:07.980 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 26/162 2024-03-30T00:25:07.980 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 27/162 2024-03-30T00:25:07.981 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 28/162 2024-03-30T00:25:07.981 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 29/162 2024-03-30T00:25:07.997 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 30/162 2024-03-30T00:25:07.997 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 31/162 2024-03-30T00:25:07.997 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 32/162 2024-03-30T00:25:07.997 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 33/162 2024-03-30T00:25:07.997 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 34/162 2024-03-30T00:25:07.997 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 35/162 2024-03-30T00:25:07.997 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 36/162 2024-03-30T00:25:07.997 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 37/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 38/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-30T00:25:07.998 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-30T00:25:07.999 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-03-30T00:25:08.000 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-03-30T00:25:08.001 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 103/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-03-30T00:25:08.002 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-03-30T00:25:08.003 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 124/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 125/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 126/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 127/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 129/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 130/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 133/162 2024-03-30T00:25:08.004 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-30T00:25:08.005 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-30T00:25:08.006 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-30T00:25:08.198 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T00:25:08.198 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T00:25:08.456 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T00:25:08.457 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T00:25:08.781 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T00:25:08.781 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T00:25:09.031 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T00:25:09.240 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-30T00:25:09.240 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T00:25:09.872 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T00:25:09.872 INFO:teuthology.orchestra.run.smithi099.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-30T00:25:09.872 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:25:10.515 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 1/162 2024-03-30T00:25:10.515 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 3/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 4/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1018.g47010 5/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 6/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 7/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 8/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 13/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 14/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 15/162 2024-03-30T00:25:10.516 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 16/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 17/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 18/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 20/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 21/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 22/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 23/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 24/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 25/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 26/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 27/162 2024-03-30T00:25:10.517 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 28/162 2024-03-30T00:25:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 29/162 2024-03-30T00:25:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 30/162 2024-03-30T00:25:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 31/162 2024-03-30T00:25:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 32/162 2024-03-30T00:25:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 33/162 2024-03-30T00:25:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 34/162 2024-03-30T00:25:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 35/162 2024-03-30T00:25:10.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 36/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 37/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 38/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-30T00:25:10.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-30T00:25:10.533 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-03-30T00:25:10.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-03-30T00:25:10.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-03-30T00:25:10.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-03-30T00:25:10.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-03-30T00:25:10.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-03-30T00:25:10.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-03-30T00:25:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-03-30T00:25:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-03-30T00:25:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-03-30T00:25:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-03-30T00:25:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-03-30T00:25:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-03-30T00:25:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-03-30T00:25:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-03-30T00:25:10.538 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-03-30T00:25:10.539 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 124/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 125/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 126/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 127/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 129/162 2024-03-30T00:25:10.540 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 130/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 133/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-30T00:25:10.541 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-30T00:25:10.542 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-30T00:25:10.543 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-30T00:25:10.545 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-30T00:25:10.545 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-30T00:25:10.545 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-30T00:25:22.876 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-30T00:25:22.876 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2024-03-30T00:25:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:25:23.510 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-03-30T00:25:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-30T00:25:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-30T00:25:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-30T00:25:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-30T00:25:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-30T00:25:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-30T00:25:23.510 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.1-8.el8.noarch 2024-03-30T00:25:23.511 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.0.4-10.el8.x86_64 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:23.512 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.1-9.el8.x86_64 2024-03-30T00:25:23.513 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: jq-1.6-7.el8.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-30T00:25:23.514 INFO:teuthology.orchestra.run.smithi161.stdout: libpq-13.11-1.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-30T00:25:23.515 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-30T00:25:23.516 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-30T00:25:23.517 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-30T00:25:23.518 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-2.6-1.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-30T00:25:23.519 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-30T00:25:23.520 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-30T00:25:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-30T00:25:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-30T00:25:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-30T00:25:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-30T00:25:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-30T00:25:23.521 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:25:23.522 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-03-30T00:25:23.876 DEBUG:teuthology.parallel:result is None 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-30T00:25:24.281 INFO:teuthology.orchestra.run.smithi142.stdout:Installed products updated. 2024-03-30T00:25:24.962 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:25:24.962 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-03-30T00:25:24.962 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-30T00:25:24.963 INFO:teuthology.orchestra.run.smithi142.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.1-8.el8.noarch 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.0.4-10.el8.x86_64 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.964 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.965 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.1-9.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: jq-1.6-7.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-30T00:25:24.966 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: libpq-13.11-1.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-30T00:25:24.967 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-30T00:25:24.968 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-30T00:25:24.969 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-2.6-1.el8.noarch 2024-03-30T00:25:24.970 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-30T00:25:24.971 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-30T00:25:24.972 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-30T00:25:24.973 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-30T00:25:24.973 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-30T00:25:24.973 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-30T00:25:24.973 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-03-30T00:25:24.973 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-03-30T00:25:25.344 DEBUG:teuthology.parallel:result is None 2024-03-30T00:25:26.341 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-30T00:25:26.341 INFO:teuthology.orchestra.run.smithi099.stdout:Installed products updated. 2024-03-30T00:25:26.976 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:25:26.976 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-30T00:25:26.977 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.1-8.el8.noarch 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.0.4-10.el8.x86_64 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.978 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.979 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.1-9.el8.x86_64 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: jq-1.6-7.el8.x86_64 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-30T00:25:26.980 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: libpq-13.11-1.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-30T00:25:26.981 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-30T00:25:26.982 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-30T00:25:26.983 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-30T00:25:26.984 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-2.6-1.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.985 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-30T00:25:26.986 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-30T00:25:26.987 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-30T00:25:26.987 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-30T00:25:26.987 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-30T00:25:26.987 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.987 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.988 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T00:25:26.988 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-30T00:25:26.988 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-30T00:25:26.988 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-30T00:25:26.988 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-30T00:25:26.988 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-30T00:25:26.988 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-30T00:25:26.989 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:25:26.989 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-03-30T00:25:27.323 DEBUG:teuthology.parallel:result is None 2024-03-30T00:25:27.324 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:25:27.469 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-30T00:25:27.522 INFO:teuthology.orchestra.run.smithi099.stdout:18.2.2-1018.g470100a4.el8 2024-03-30T00:25:27.522 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1018.g470100a4.el8 2024-03-30T00:25:27.523 INFO:teuthology.task.install:The correct ceph version 18.2.2-1018.g470100a4 is installed. 2024-03-30T00:25:27.525 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:25:27.746 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-30T00:25:27.806 INFO:teuthology.orchestra.run.smithi142.stdout:18.2.2-1018.g470100a4.el8 2024-03-30T00:25:27.806 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1018.g470100a4.el8 2024-03-30T00:25:27.806 INFO:teuthology.task.install:The correct ceph version 18.2.2-1018.g470100a4 is installed. 2024-03-30T00:25:27.809 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:25:27.952 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-30T00:25:28.010 INFO:teuthology.orchestra.run.smithi161.stdout:18.2.2-1018.g470100a4.el8 2024-03-30T00:25:28.011 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1018.g470100a4.el8 2024-03-30T00:25:28.011 INFO:teuthology.task.install:The correct ceph version 18.2.2-1018.g470100a4 is installed. 2024-03-30T00:25:28.012 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-30T00:25:28.013 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:25:28.013 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T00:25:28.101 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-30T00:25:28.101 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T00:25:28.193 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-30T00:25:28.193 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T00:25:28.285 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-30T00:25:28.285 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:25:28.286 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T00:25:28.354 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T00:25:28.458 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-30T00:25:28.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T00:25:28.526 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T00:25:28.632 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-30T00:25:28.632 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T00:25:28.700 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T00:25:28.806 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-30T00:25:28.807 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:25:28.807 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T00:25:28.871 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T00:25:28.975 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-30T00:25:28.975 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T00:25:29.041 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T00:25:29.146 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-30T00:25:29.146 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T00:25:29.210 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T00:25:29.314 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-30T00:25:29.315 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:25:29.315 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T00:25:29.379 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T00:25:29.482 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-30T00:25:29.482 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T00:25:29.548 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T00:25:29.653 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-30T00:25:29.653 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T00:25:29.719 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T00:25:29.824 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-30T00:25:29.975 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-30T00:25:29.975 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:25:29.976 INFO:tasks.cephadm:Cluster fsid is 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:25:29.976 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-30T00:25:29.976 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.99', 'mon.b': '172.21.15.142', 'mon.c': '172.21.15.161'} 2024-03-30T00:25:29.976 INFO:tasks.cephadm:First mon is mon.a on smithi099 2024-03-30T00:25:29.976 INFO:tasks.cephadm:First mgr is x 2024-03-30T00:25:29.977 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-30T00:25:29.977 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2024-03-30T00:25:30.045 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-03-30T00:25:30.112 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-03-30T00:25:30.179 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-30T00:25:30.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:25:30.404 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/78078/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1018-g470100a4', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1018.g470100a4'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-03-29 13:05:59.448245', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240327.051042-reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-30T00:25:30.554 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-rishabh-testing-20240327.051042-reef, sha1 470100a4ccbcf7769052bf09a5413881ccf1b97f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:25:30.556 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/x86_64/flavors/default/cephadm 2024-03-30T00:25:30.556 DEBUG:teuthology.orchestra.run.smithi099:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-30T00:25:31.165 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 Mar 30 00:25 /home/ubuntu/cephtest/cephadm 2024-03-30T00:25:31.165 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-30T00:25:31.604 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 Mar 30 00:25 /home/ubuntu/cephtest/cephadm 2024-03-30T00:25:31.604 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-30T00:25:32.052 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 Mar 30 00:25 /home/ubuntu/cephtest/cephadm 2024-03-30T00:25:32.053 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-30T00:25:32.102 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-30T00:25:32.151 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-30T00:25:32.215 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f on all hosts... 2024-03-30T00:25:32.215 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f pull 2024-03-30T00:25:32.220 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f pull 2024-03-30T00:25:32.223 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f pull 2024-03-30T00:25:33.338 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f... 2024-03-30T00:25:33.563 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f... 2024-03-30T00:25:34.030 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f... 2024-03-30T00:26:00.065 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-30T00:26:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:26:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "e7c444a08cf749270ac19a49d6d49787223f88c26222d2dfc1d77b830f40791d", 2024-03-30T00:26:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-03-30T00:26:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c123be959ad18653301727c9f149d0b2813c48e5bad48157ed6a183a20cecd35" 2024-03-30T00:26:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-03-30T00:26:00.066 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-30T00:26:03.349 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:26:03.350 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:26:03.350 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "e7c444a08cf749270ac19a49d6d49787223f88c26222d2dfc1d77b830f40791d", 2024-03-30T00:26:03.350 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2024-03-30T00:26:03.350 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c123be959ad18653301727c9f149d0b2813c48e5bad48157ed6a183a20cecd35" 2024-03-30T00:26:03.350 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-30T00:26:03.350 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:26:06.530 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-03-30T00:26:06.530 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:26:06.530 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "e7c444a08cf749270ac19a49d6d49787223f88c26222d2dfc1d77b830f40791d", 2024-03-30T00:26:06.530 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-03-30T00:26:06.530 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c123be959ad18653301727c9f149d0b2813c48e5bad48157ed6a183a20cecd35" 2024-03-30T00:26:06.531 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-03-30T00:26:06.531 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-03-30T00:26:06.558 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2024-03-30T00:26:06.634 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-03-30T00:26:06.711 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-03-30T00:26:06.781 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2024-03-30T00:26:06.848 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-03-30T00:26:06.916 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-03-30T00:26:06.980 INFO:tasks.cephadm:Writing seed config... 2024-03-30T00:26:06.996 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-30T00:26:06.996 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-30T00:26:06.996 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-30T00:26:06.996 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-30T00:26:06.996 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-30T00:26:06.996 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-30T00:26:06.996 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-30T00:26:06.996 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-30T00:26:06.996 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-30T00:26:06.996 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-30T00:26:06.997 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-30T00:26:06.997 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-30T00:26:06.997 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-30T00:26:06.997 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-30T00:26:06.997 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-30T00:26:06.997 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-30T00:26:06.997 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-30T00:26:06.997 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-30T00:26:06.997 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-30T00:26:06.997 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-30T00:26:06.997 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-30T00:26:06.998 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-30T00:26:06.999 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-30T00:26:06.999 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-30T00:26:06.999 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-30T00:26:06.999 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-30T00:26:06.999 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-30T00:26:06.999 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-30T00:26:06.999 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-30T00:26:06.999 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-30T00:26:06.999 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-30T00:26:06.999 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-30T00:26:06.999 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-30T00:26:07.000 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:26:07.000 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-30T00:26:07.043 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 03801b1a-ee2c-11ee-b647-cb9ed24678a4 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-30T00:26:07.043 DEBUG:teuthology.orchestra.run.smithi099:mon.a> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.a.service 2024-03-30T00:26:07.087 DEBUG:teuthology.orchestra.run.smithi099:mgr.x> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.x.service 2024-03-30T00:26:07.131 INFO:tasks.cephadm:Bootstrapping... 2024-03-30T00:26:07.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f -v bootstrap --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.99 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-30T00:26:07.151 INFO:journalctl@ceph.mon.a.smithi099.stdout:-- Logs begin at Sat 2024-03-30 00:10:25 UTC. -- 2024-03-30T00:26:07.197 INFO:journalctl@ceph.mgr.x.smithi099.stdout:-- Logs begin at Sat 2024-03-30 00:10:25 UTC. -- 2024-03-30T00:26:07.347 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-03-30T00:26:07.347 INFO:teuthology.orchestra.run.smithi099.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f', '-v', 'bootstrap', '--fsid', '03801b1a-ee2c-11ee-b647-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.99', '--skip-admin-label'] 2024-03-30T00:26:07.394 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 4.6.1 2024-03-30T00:26:07.394 INFO:teuthology.orchestra.run.smithi099.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-30T00:26:07.395 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying podman|docker is present... 2024-03-30T00:26:07.440 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 4.6.1 2024-03-30T00:26:07.440 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying lvm2 is present... 2024-03-30T00:26:07.440 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying time synchronization is in place... 2024-03-30T00:26:07.454 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-30T00:26:07.454 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-30T00:26:07.466 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-30T00:26:07.466 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-30T00:26:07.479 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout enabled 2024-03-30T00:26:07.491 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout active 2024-03-30T00:26:07.491 INFO:teuthology.orchestra.run.smithi099.stdout:Unit chronyd.service is enabled and running 2024-03-30T00:26:07.492 INFO:teuthology.orchestra.run.smithi099.stdout:Repeating the final host check... 2024-03-30T00:26:07.536 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout 4.6.1 2024-03-30T00:26:07.536 INFO:teuthology.orchestra.run.smithi099.stdout:podman (/bin/podman) version 4.6.1 is present 2024-03-30T00:26:07.536 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl is present 2024-03-30T00:26:07.536 INFO:teuthology.orchestra.run.smithi099.stdout:lvcreate is present 2024-03-30T00:26:07.548 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-30T00:26:07.548 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-30T00:26:07.560 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-30T00:26:07.561 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-30T00:26:07.573 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout enabled 2024-03-30T00:26:07.585 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout active 2024-03-30T00:26:07.585 INFO:teuthology.orchestra.run.smithi099.stdout:Unit chronyd.service is enabled and running 2024-03-30T00:26:07.585 INFO:teuthology.orchestra.run.smithi099.stdout:Host looks OK 2024-03-30T00:26:07.585 INFO:teuthology.orchestra.run.smithi099.stdout:Cluster fsid: 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:26:07.586 INFO:teuthology.orchestra.run.smithi099.stdout:Acquiring lock 139680347093480 on /run/cephadm/03801b1a-ee2c-11ee-b647-cb9ed24678a4.lock 2024-03-30T00:26:07.586 INFO:teuthology.orchestra.run.smithi099.stdout:Lock 139680347093480 acquired on /run/cephadm/03801b1a-ee2c-11ee-b647-cb9ed24678a4.lock 2024-03-30T00:26:07.586 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying IP 172.21.15.99 port 3300 ... 2024-03-30T00:26:07.587 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying IP 172.21.15.99 port 6789 ... 2024-03-30T00:26:07.587 INFO:teuthology.orchestra.run.smithi099.stdout:Base mon IP(s) is [172.21.15.99:3300, 172.21.15.99:6789], mon addrv is [v2:172.21.15.99:3300,v1:172.21.15.99:6789] 2024-03-30T00:26:07.595 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2024-03-30T00:26:07.595 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.99 metric 100 2024-03-30T00:26:07.601 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-30T00:26:07.601 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-30T00:26:07.602 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1772sec hoplimit 64 pref medium 2024-03-30T00:26:07.608 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-30T00:26:07.608 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-30T00:26:07.608 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-30T00:26:07.608 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-30T00:26:07.608 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-30T00:26:07.608 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-30T00:26:07.609 INFO:teuthology.orchestra.run.smithi099.stdout:Mon IP `172.21.15.99` is in CIDR network `172.21.0.0/20` 2024-03-30T00:26:07.609 INFO:teuthology.orchestra.run.smithi099.stdout:Mon IP `172.21.15.99` is in CIDR network `172.21.0.0/20` 2024-03-30T00:26:07.609 INFO:teuthology.orchestra.run.smithi099.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-30T00:26:07.610 INFO:teuthology.orchestra.run.smithi099.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-30T00:26:07.611 INFO:teuthology.orchestra.run.smithi099.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f... 2024-03-30T00:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stdout e7c444a08cf749270ac19a49d6d49787223f88c26222d2dfc1d77b830f40791d 2024-03-30T00:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f... 2024-03-30T00:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Getting image source signatures 2024-03-30T00:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying blob sha256:86ca209f044bd2e3705adc9da4fc1145309402ec2e51b9b9b051617a85f31417 2024-03-30T00:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying blob sha256:11da15500eca1f826c328eb3265caa30b0a3730510e2a32023084d6fe9cee20c 2024-03-30T00:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Copying config sha256:e7c444a08cf749270ac19a49d6d49787223f88c26222d2dfc1d77b830f40791d 2024-03-30T00:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-30T00:26:08.738 INFO:teuthology.orchestra.run.smithi099.stdout:ceph: stdout ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable) 2024-03-30T00:26:08.738 INFO:teuthology.orchestra.run.smithi099.stdout:Ceph version: ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable) 2024-03-30T00:26:08.738 INFO:teuthology.orchestra.run.smithi099.stdout:Extracting ceph user uid/gid from container image... 2024-03-30T00:26:09.323 INFO:teuthology.orchestra.run.smithi099.stdout:stat: stdout 167 167 2024-03-30T00:26:09.323 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial keys... 2024-03-30T00:26:10.408 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQAhXAdmjXjfIhAAEcXmiY8JOYHjDOZ/Fjus+A== 2024-03-30T00:26:11.047 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQAiXAdmcr8rKxAAwYKU0TB+fSIvCGeAijZKDA== 2024-03-30T00:26:11.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph-authtool: stdout AQAjXAdmZF1rERAAy8IJvwSEVgK4ySwwHAkWOQ== 2024-03-30T00:26:11.606 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial monmap... 2024-03-30T00:26:12.211 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-30T00:26:12.211 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-30T00:26:12.211 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:26:12.211 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-30T00:26:12.211 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool for a [v2:172.21.15.99:3300,v1:172.21.15.99:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-30T00:26:12.211 INFO:teuthology.orchestra.run.smithi099.stdout:setting min_mon_release = pacific 2024-03-30T00:26:12.211 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: set fsid to 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:26:12.211 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-30T00:26:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:26:12.212 INFO:teuthology.orchestra.run.smithi099.stdout:Creating mon... 2024-03-30T00:26:12.868 INFO:teuthology.orchestra.run.smithi099.stdout:create mon.a on 2024-03-30T00:26:13.102 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-03-30T00:26:13.290 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-30T00:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4.target. 2024-03-30T00:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4.target. 2024-03-30T00:26:13.828 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.a 2024-03-30T00:26:13.828 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to reset failed state of unit ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.a.service: Unit ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.a.service not loaded. 2024-03-30T00:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4.target.wants/ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.a.service → /etc/systemd/system/ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@.service. 2024-03-30T00:26:14.103 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:14 smithi099 systemd[1]: Starting Ceph mon.a for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-30T00:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-03-30T00:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-30T00:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-30T00:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-03-30T00:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to enable service . firewalld.service is not available 2024-03-30T00:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mon to start... 2024-03-30T00:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mon... 2024-03-30T00:26:14.519 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:14 smithi099 bash[37541]: 17c4976c30898ae01bf6c7670cb7b781cecdb33b46640d5553fd442a6451edbf 2024-03-30T00:26:14.519 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:14 smithi099 systemd[1]: Started Ceph mon.a for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:26:14.520 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:14 smithi099 ceph-mon[37580]: mkfs 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:26:14.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:14 smithi099 ceph-mon[37580]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-30T00:26:15.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:15 smithi099 ceph-mon[37580]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-30T00:26:15.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:15 smithi099 ceph-mon[37580]: monmap e1: 1 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T00:26:15.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:15 smithi099 ceph-mon[37580]: fsmap 2024-03-30T00:26:15.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:15 smithi099 ceph-mon[37580]: osdmap e1: 0 total, 0 up, 0 in 2024-03-30T00:26:15.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:15 smithi099 ceph-mon[37580]: mgrmap e1: no daemons active 2024-03-30T00:26:15.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:15 smithi099 ceph-mon[37580]: from='client.? 172.21.15.99:0/2036448233' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-30T00:26:16.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout cluster: 2024-03-30T00:26:16.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout id: 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:26:16.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-30T00:26:16.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:16.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout services: 2024-03-30T00:26:16.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.52388s) 2024-03-30T00:26:16.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-30T00:26:16.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-30T00:26:16.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:16.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout data: 2024-03-30T00:26:16.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-30T00:26:16.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-30T00:26:16.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-30T00:26:16.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout pgs: 2024-03-30T00:26:16.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:16.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.943+0000 7f075f400700 1 Processor -- start 2024-03-30T00:26:16.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.943+0000 7f075f400700 1 -- start start 2024-03-30T00:26:16.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.943+0000 7f075f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f07580791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:16.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.943+0000 7f075f400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07580797c0 con 0x7f075807ad90 2024-03-30T00:26:16.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.944+0000 7f075ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f07580791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:16.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.944+0000 7f075ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f07580791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48518/0 (socket says 172.21.15.99:48518) 2024-03-30T00:26:16.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.944+0000 7f075ca00700 1 -- 172.21.15.99:0/494964188 learned_addr learned my addr 172.21.15.99:0/494964188 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:16.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.945+0000 7f075ca00700 1 -- 172.21.15.99:0/494964188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0758079900 con 0x7f075807ad90 2024-03-30T00:26:16.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.945+0000 7f075ca00700 1 --2- 172.21.15.99:0/494964188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f07580791f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f074c0098a0 tx=0x7f074c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883ade8400e43570 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:16.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.945+0000 7f0757400700 1 -- 172.21.15.99:0/494964188 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f074c004160 con 0x7f075807ad90 2024-03-30T00:26:16.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.945+0000 7f0757400700 1 -- 172.21.15.99:0/494964188 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c0042c0 con 0x7f075807ad90 2024-03-30T00:26:16.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.945+0000 7f0757400700 1 -- 172.21.15.99:0/494964188 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f074c0045a0 con 0x7f075807ad90 2024-03-30T00:26:16.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.945+0000 7f075f400700 1 -- 172.21.15.99:0/494964188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 msgr2=0x7f07580791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.945+0000 7f075f400700 1 --2- 172.21.15.99:0/494964188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f07580791f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f074c0098a0 tx=0x7f074c009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.946+0000 7f075f400700 1 -- 172.21.15.99:0/494964188 shutdown_connections 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.946+0000 7f075f400700 1 --2- 172.21.15.99:0/494964188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f07580791f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.946+0000 7f075f400700 1 -- 172.21.15.99:0/494964188 >> 172.21.15.99:0/494964188 conn(0x7f0758101990 msgr2=0x7f0758103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.946+0000 7f075f400700 1 -- 172.21.15.99:0/494964188 shutdown_connections 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.946+0000 7f075f400700 1 -- 172.21.15.99:0/494964188 wait complete. 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.947+0000 7f075f400700 1 Processor -- start 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.948+0000 7f075f400700 1 -- start start 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.948+0000 7f075f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f075819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.948+0000 7f075f400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075819c090 con 0x7f075807ad90 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.948+0000 7f075ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f075819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:16.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.948+0000 7f075ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f075819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48532/0 (socket says 172.21.15.99:48532) 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.948+0000 7f075ca00700 1 -- 172.21.15.99:0/2036448233 learned_addr learned my addr 172.21.15.99:0/2036448233 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.949+0000 7f075ca00700 1 -- 172.21.15.99:0/2036448233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c009550 con 0x7f075807ad90 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.949+0000 7f075ca00700 1 --2- 172.21.15.99:0/2036448233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f075819bb50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f074c004000 tx=0x7f074c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.949+0000 7f0755600700 1 -- 172.21.15.99:0/2036448233 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f074c011630 con 0x7f075807ad90 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.949+0000 7f0755600700 1 -- 172.21.15.99:0/2036448233 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c011790 con 0x7f075807ad90 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.949+0000 7f075f400700 1 -- 172.21.15.99:0/2036448233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075819c290 con 0x7f075807ad90 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.949+0000 7f0755600700 1 -- 172.21.15.99:0/2036448233 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f074c011a30 con 0x7f075807ad90 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.949+0000 7f075f400700 1 -- 172.21.15.99:0/2036448233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075819c730 con 0x7f075807ad90 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.950+0000 7f0755600700 1 -- 172.21.15.99:0/2036448233 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f074c022070 con 0x7f075807ad90 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.950+0000 7f0755600700 1 -- 172.21.15.99:0/2036448233 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f074c01c450 con 0x7f075807ad90 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.950+0000 7f075f400700 1 -- 172.21.15.99:0/2036448233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07581959d0 con 0x7f075807ad90 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:14.954+0000 7f0755600700 1 -- 172.21.15.99:0/2036448233 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f074c01b450 con 0x7f075807ad90 2024-03-30T00:26:16.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:15.026+0000 7f075f400700 1 -- 172.21.15.99:0/2036448233 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f0758079ea0 con 0x7f075807ad90 2024-03-30T00:26:16.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:15.027+0000 7f0755600700 1 -- 172.21.15.99:0/2036448233 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7f074c01b5f0 con 0x7f075807ad90 2024-03-30T00:26:16.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:15.027+0000 7f075f400700 1 -- 172.21.15.99:0/2036448233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 msgr2=0x7f075819bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:16.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:15.027+0000 7f075f400700 1 --2- 172.21.15.99:0/2036448233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f075819bb50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f074c004000 tx=0x7f074c004750 comp rx=0 tx=0).stop 2024-03-30T00:26:16.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:15.028+0000 7f075f400700 1 -- 172.21.15.99:0/2036448233 shutdown_connections 2024-03-30T00:26:16.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:15.028+0000 7f075f400700 1 --2- 172.21.15.99:0/2036448233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075807ad90 0x7f075819bb50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:16.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:15.028+0000 7f075f400700 1 -- 172.21.15.99:0/2036448233 >> 172.21.15.99:0/2036448233 conn(0x7f0758101990 msgr2=0x7f0758101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:16.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:15.028+0000 7f075f400700 1 -- 172.21.15.99:0/2036448233 shutdown_connections 2024-03-30T00:26:16.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:15.028+0000 7f075f400700 1 -- 172.21.15.99:0/2036448233 wait complete. 2024-03-30T00:26:16.635 INFO:teuthology.orchestra.run.smithi099.stdout:mon is available 2024-03-30T00:26:16.635 INFO:teuthology.orchestra.run.smithi099.stdout:Assimilating anything we can from ceph.conf... 2024-03-30T00:26:17.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:17 smithi099 ceph-mon[37580]: from='client.? 172.21.15.99:0/458201700' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-30T00:26:17.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:17 smithi099 ceph-mon[37580]: from='client.? 172.21.15.99:0/458201700' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [global] 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout fsid = 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.99:3300,v1:172.21.15.99:6789] 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:17.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [osd] 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.068+0000 7fdde1a00700 1 Processor -- start 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.069+0000 7fdde1a00700 1 -- start start 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.069+0000 7fdde1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc0791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.069+0000 7fdde1a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0797a0 con 0x7fdddc07ad70 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.070+0000 7fdddbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc0791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.070+0000 7fdddbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc0791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48542/0 (socket says 172.21.15.99:48542) 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.070+0000 7fdddbe00700 1 -- 172.21.15.99:0/1257854062 learned_addr learned my addr 172.21.15.99:0/1257854062 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.070+0000 7fdddbe00700 1 -- 172.21.15.99:0/1257854062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0798e0 con 0x7fdddc07ad70 2024-03-30T00:26:17.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.070+0000 7fdddbe00700 1 --2- 172.21.15.99:0/1257854062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc0791d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0098a0 tx=0x7fddcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c984e3c540fb0524 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.071+0000 7fdddb400700 1 -- 172.21.15.99:0/1257854062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddcc004160 con 0x7fdddc07ad70 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.071+0000 7fdddb400700 1 -- 172.21.15.99:0/1257854062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc0042c0 con 0x7fdddc07ad70 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.071+0000 7fdddb400700 1 -- 172.21.15.99:0/1257854062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddcc0045a0 con 0x7fdddc07ad70 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.071+0000 7fdde1a00700 1 -- 172.21.15.99:0/1257854062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 msgr2=0x7fdddc0791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.071+0000 7fdde1a00700 1 --2- 172.21.15.99:0/1257854062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc0791d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0098a0 tx=0x7fddcc009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.071+0000 7fdde1a00700 1 -- 172.21.15.99:0/1257854062 shutdown_connections 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.071+0000 7fdde1a00700 1 --2- 172.21.15.99:0/1257854062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc0791d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.071+0000 7fdde1a00700 1 -- 172.21.15.99:0/1257854062 >> 172.21.15.99:0/1257854062 conn(0x7fdddc1017c0 msgr2=0x7fdddc103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.071+0000 7fdde1a00700 1 -- 172.21.15.99:0/1257854062 shutdown_connections 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.071+0000 7fdde1a00700 1 -- 172.21.15.99:0/1257854062 wait complete. 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.072+0000 7fdde1a00700 1 Processor -- start 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.073+0000 7fdde1a00700 1 -- start start 2024-03-30T00:26:17.512 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.073+0000 7fdde1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.073+0000 7fdde1a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc19bfa0 con 0x7fdddc07ad70 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.073+0000 7fdddbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.073+0000 7fdddbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48558/0 (socket says 172.21.15.99:48558) 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.073+0000 7fdddbe00700 1 -- 172.21.15.99:0/458201700 learned_addr learned my addr 172.21.15.99:0/458201700 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.073+0000 7fdddbe00700 1 -- 172.21.15.99:0/458201700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc009550 con 0x7fdddc07ad70 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.074+0000 7fdddbe00700 1 --2- 172.21.15.99:0/458201700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc19ba60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fddcc004000 tx=0x7fddcc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.074+0000 7fddd9600700 1 -- 172.21.15.99:0/458201700 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddcc011630 con 0x7fdddc07ad70 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.074+0000 7fddd9600700 1 -- 172.21.15.99:0/458201700 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc011790 con 0x7fdddc07ad70 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.074+0000 7fdde1a00700 1 -- 172.21.15.99:0/458201700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc19c1a0 con 0x7fdddc07ad70 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.074+0000 7fddd9600700 1 -- 172.21.15.99:0/458201700 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddcc011a30 con 0x7fdddc07ad70 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.074+0000 7fdde1a00700 1 -- 172.21.15.99:0/458201700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc19c640 con 0x7fdddc07ad70 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.074+0000 7fddd9600700 1 -- 172.21.15.99:0/458201700 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fddcc021070 con 0x7fdddc07ad70 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.074+0000 7fddd9600700 1 -- 172.21.15.99:0/458201700 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fddcc01b440 con 0x7fdddc07ad70 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.075+0000 7fdde1a00700 1 -- 172.21.15.99:0/458201700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc195790 con 0x7fdddc07ad70 2024-03-30T00:26:17.513 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.078+0000 7fddd9600700 1 -- 172.21.15.99:0/458201700 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fddcc01a460 con 0x7fdddc07ad70 2024-03-30T00:26:17.514 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.145+0000 7fdde1a00700 1 -- 172.21.15.99:0/458201700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fdddc079e80 con 0x7fdddc07ad70 2024-03-30T00:26:17.514 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.150+0000 7fddd9600700 1 -- 172.21.15.99:0/458201700 <== mon.0 v2:172.21.15.99:3300/0 7 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fddcc01a600 con 0x7fdddc07ad70 2024-03-30T00:26:17.514 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.150+0000 7fddd9600700 1 -- 172.21.15.99:0/458201700 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fddcc035590 con 0x7fdddc07ad70 2024-03-30T00:26:17.514 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.151+0000 7fdde1a00700 1 -- 172.21.15.99:0/458201700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 msgr2=0x7fdddc19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:17.514 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.151+0000 7fdde1a00700 1 --2- 172.21.15.99:0/458201700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc19ba60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fddcc004000 tx=0x7fddcc004750 comp rx=0 tx=0).stop 2024-03-30T00:26:17.514 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.151+0000 7fdde1a00700 1 -- 172.21.15.99:0/458201700 shutdown_connections 2024-03-30T00:26:17.514 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.151+0000 7fdde1a00700 1 --2- 172.21.15.99:0/458201700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdddc07ad70 0x7fdddc19ba60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:17.514 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.151+0000 7fdde1a00700 1 -- 172.21.15.99:0/458201700 >> 172.21.15.99:0/458201700 conn(0x7fdddc1017c0 msgr2=0x7fdddc102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:17.514 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.151+0000 7fdde1a00700 1 -- 172.21.15.99:0/458201700 shutdown_connections 2024-03-30T00:26:17.514 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.151+0000 7fdde1a00700 1 -- 172.21.15.99:0/458201700 wait complete. 2024-03-30T00:26:17.514 INFO:teuthology.orchestra.run.smithi099.stdout:Generating new minimal ceph.conf... 2024-03-30T00:26:18.300 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:18 smithi099 ceph-mon[37580]: from='client.? 172.21.15.99:0/2713147712' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.893+0000 7f838f800700 1 Processor -- start 2024-03-30T00:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.894+0000 7f838f800700 1 -- start start 2024-03-30T00:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.894+0000 7f838f800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f8388106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.894+0000 7f838f800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8388106e20 con 0x7f8388106430 2024-03-30T00:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.894+0000 7f838ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f8388106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.894+0000 7f838ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f8388106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48568/0 (socket says 172.21.15.99:48568) 2024-03-30T00:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.894+0000 7f838ce00700 1 -- 172.21.15.99:0/2861772432 learned_addr learned my addr 172.21.15.99:0/2861772432 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.894+0000 7f838ce00700 1 -- 172.21.15.99:0/2861772432 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8388107630 con 0x7f8388106430 2024-03-30T00:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.895+0000 7f838ce00700 1 --2- 172.21.15.99:0/2861772432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f8388106850 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f837c0098a0 tx=0x7f837c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b4817542705db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.895+0000 7f8387400700 1 -- 172.21.15.99:0/2861772432 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f837c004160 con 0x7f8388106430 2024-03-30T00:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.895+0000 7f8387400700 1 -- 172.21.15.99:0/2861772432 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f837c0042c0 con 0x7f8388106430 2024-03-30T00:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.895+0000 7f8387400700 1 -- 172.21.15.99:0/2861772432 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f837c011940 con 0x7f8388106430 2024-03-30T00:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.895+0000 7f838f800700 1 -- 172.21.15.99:0/2861772432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 msgr2=0x7f8388106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.895+0000 7f838f800700 1 --2- 172.21.15.99:0/2861772432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f8388106850 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f837c0098a0 tx=0x7f837c009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.896+0000 7f838f800700 1 -- 172.21.15.99:0/2861772432 shutdown_connections 2024-03-30T00:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.896+0000 7f838f800700 1 --2- 172.21.15.99:0/2861772432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f8388106850 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.896+0000 7f838f800700 1 -- 172.21.15.99:0/2861772432 >> 172.21.15.99:0/2861772432 conn(0x7f8388101990 msgr2=0x7f8388103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.896+0000 7f838f800700 1 -- 172.21.15.99:0/2861772432 shutdown_connections 2024-03-30T00:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.896+0000 7f838f800700 1 -- 172.21.15.99:0/2861772432 wait complete. 2024-03-30T00:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.897+0000 7f838f800700 1 Processor -- start 2024-03-30T00:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.897+0000 7f838f800700 1 -- start start 2024-03-30T00:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.897+0000 7f838f800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f838807a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.897+0000 7f838f800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838807af30 con 0x7f8388106430 2024-03-30T00:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.897+0000 7f838ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f838807a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.897+0000 7f838ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f838807a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48578/0 (socket says 172.21.15.99:48578) 2024-03-30T00:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.897+0000 7f838ce00700 1 -- 172.21.15.99:0/2713147712 learned_addr learned my addr 172.21.15.99:0/2713147712 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.898+0000 7f838ce00700 1 -- 172.21.15.99:0/2713147712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c009550 con 0x7f8388106430 2024-03-30T00:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.898+0000 7f838ce00700 1 --2- 172.21.15.99:0/2713147712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f838807a9f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f837c003f20 tx=0x7f837c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.898+0000 7f8385600700 1 -- 172.21.15.99:0/2713147712 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f837c0036a0 con 0x7f8388106430 2024-03-30T00:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.898+0000 7f8385600700 1 -- 172.21.15.99:0/2713147712 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f837c01b440 con 0x7f8388106430 2024-03-30T00:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.898+0000 7f8385600700 1 -- 172.21.15.99:0/2713147712 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f837c023400 con 0x7f8388106430 2024-03-30T00:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.898+0000 7f838f800700 1 -- 172.21.15.99:0/2713147712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8388079170 con 0x7f8388106430 2024-03-30T00:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.898+0000 7f838f800700 1 -- 172.21.15.99:0/2713147712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8388079610 con 0x7f8388106430 2024-03-30T00:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.899+0000 7f8385600700 1 -- 172.21.15.99:0/2713147712 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f837c023970 con 0x7f8388106430 2024-03-30T00:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.899+0000 7f8385600700 1 -- 172.21.15.99:0/2713147712 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f837c02cdf0 con 0x7f8388106430 2024-03-30T00:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.900+0000 7f838f800700 1 -- 172.21.15.99:0/2713147712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83880623e0 con 0x7f8388106430 2024-03-30T00:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.902+0000 7f8385600700 1 -- 172.21.15.99:0/2713147712 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f837c023d60 con 0x7f8388106430 2024-03-30T00:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.971+0000 7f838f800700 1 -- 172.21.15.99:0/2713147712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f8388079850 con 0x7f8388106430 2024-03-30T00:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.972+0000 7f8385600700 1 -- 172.21.15.99:0/2713147712 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f837c01b5b0 con 0x7f8388106430 2024-03-30T00:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.973+0000 7f838f800700 1 -- 172.21.15.99:0/2713147712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 msgr2=0x7f838807a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.973+0000 7f838f800700 1 --2- 172.21.15.99:0/2713147712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f838807a9f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f837c003f20 tx=0x7f837c011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.973+0000 7f838f800700 1 -- 172.21.15.99:0/2713147712 shutdown_connections 2024-03-30T00:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.973+0000 7f838f800700 1 --2- 172.21.15.99:0/2713147712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388106430 0x7f838807a9f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.973+0000 7f838f800700 1 -- 172.21.15.99:0/2713147712 >> 172.21.15.99:0/2713147712 conn(0x7f8388101990 msgr2=0x7f83881024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.973+0000 7f838f800700 1 -- 172.21.15.99:0/2713147712 shutdown_connections 2024-03-30T00:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:17.973+0000 7f838f800700 1 -- 172.21.15.99:0/2713147712 wait complete. 2024-03-30T00:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout:Restarting the monitor... 2024-03-30T00:26:18.584 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:18 smithi099 systemd[1]: Stopping Ceph mon.a for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:26:18.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:18 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-a[37557]: 2024-03-30T00:26:18.575+0000 7fb248200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:26:18.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:18 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-a[37557]: 2024-03-30T00:26:18.575+0000 7fb248200700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-30T00:26:19.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:18 smithi099 bash[37916]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-a 2024-03-30T00:26:19.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 systemd[1]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.a.service: Succeeded. 2024-03-30T00:26:19.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 systemd[1]: Stopped Ceph mon.a for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:26:19.667 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 systemd[1]: Starting Ceph mon.a for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:26:19.741 INFO:teuthology.orchestra.run.smithi099.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-30T00:26:19.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 bash[38041]: aa969bc775f153c31704d0d1325926e90deb52c0ccdcc662a5ce123206f27fbb 2024-03-30T00:26:19.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 systemd[1]: Started Ceph mon.a for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:26:19.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: set uid:gid to 167:167 (ceph:ceph) 2024-03-30T00:26:19.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable), process ceph-mon, pid 7 2024-03-30T00:26:19.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: pidfile_write: ignore empty --pid-file 2024-03-30T00:26:19.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: load: jerasure load: lrc 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: RocksDB version: 7.9.2 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Git sha 0 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Compile date 2024-03-29 11:49:16 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: DB SUMMARY 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: DB Session ID: CYDJAPODNTDUQUQAYCYD 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: CURRENT file: CURRENT 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: IDENTITY file: IDENTITY 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99177 ; 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.error_if_exists: 0 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.create_if_missing: 0 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.paranoid_checks: 1 2024-03-30T00:26:19.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.env: 0x560bc85d1720 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.fs: PosixFileSystem 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.info_log: 0x560bcb1cf520 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_file_opening_threads: 16 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.statistics: (nil) 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.use_fsync: 0 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_log_file_size: 0 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.keep_log_file_num: 1000 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.recycle_log_file_num: 0 2024-03-30T00:26:19.993 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.allow_fallocate: 1 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.allow_mmap_reads: 0 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.allow_mmap_writes: 0 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.use_direct_reads: 0 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.create_missing_column_families: 0 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.db_log_dir: 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.wal_dir: 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-30T00:26:19.994 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.advise_random_on_open: 1 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.db_write_buffer_size: 0 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.write_buffer_manager: 0x560bca466320 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.rate_limiter: (nil) 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.wal_recovery_mode: 2 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.enable_thread_tracking: 0 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.enable_pipelined_write: 0 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.unordered_write: 0 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-30T00:26:19.995 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.row_cache: None 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.wal_filter: None 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.allow_ingest_behind: 0 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.two_write_queues: 0 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.manual_wal_flush: 0 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.wal_compression: 0 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.atomic_flush: 0 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.log_readahead_size: 0 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.best_efforts_recovery: 0 2024-03-30T00:26:19.996 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.allow_data_in_errors: 0 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.db_host_id: __hostname__ 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_background_jobs: 2 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_background_compactions: -1 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_subcompactions: 1 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_total_wal_size: 0 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-30T00:26:19.997 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_open_files: -1 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bytes_per_sync: 0 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_readahead_size: 0 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_background_flushes: -1 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Compression algorithms supported: 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: kZSTD supported: 0 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: kXpressCompression supported: 0 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: kLZ4HCCompression supported: 1 2024-03-30T00:26:19.998 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: kZlibCompression supported: 1 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: kSnappyCompression supported: 1 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: kLZ4Compression supported: 1 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: kBZip2Compression supported: 0 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.merge_operator: 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_filter: None 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_filter_factory: None 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.sst_partitioner_factory: None 2024-03-30T00:26:19.999 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560bcb1cf620) 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: cache_index_and_filter_blocks: 1 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: pin_top_level_index_and_filter: 1 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_type: 0 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: data_block_index_type: 0 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_shortening: 1 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: checksum: 4 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: no_block_cache: 0 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache: 0x560bca4e7350 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_name: BinnedLRUCache 2024-03-30T00:26:20.000 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_options: 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: capacity : 536870912 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: num_shard_bits : 4 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: strict_capacity_limit : 0 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: high_pri_pool_ratio: 0.000 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_cache_compressed: (nil) 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: persistent_cache: (nil) 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_size: 4096 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_size_deviation: 10 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_restart_interval: 16 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: index_block_restart_interval: 1 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: metadata_block_size: 4096 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: partition_filters: 0 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: use_delta_encoding: 1 2024-03-30T00:26:20.001 INFO:journalctl@ceph.mon.a.smithi099.stdout: filter_policy: bloomfilter 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: whole_key_filtering: 1 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: verify_compression: 0 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: read_amp_bytes_per_bit: 0 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: format_version: 5 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: enable_index_compression: 1 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: block_align: 0 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: max_auto_readahead_size: 262144 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: prepopulate_block_cache: 0 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: initial_auto_readahead_size: 8192 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout: num_file_reads_for_auto_readahead: 2 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.write_buffer_size: 33554432 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_write_buffer_number: 2 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compression: NoCompression 2024-03-30T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bottommost_compression: Disabled 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.prefix_extractor: nullptr 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.num_levels: 7 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-30T00:26:20.003 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compression_opts.level: 32767 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compression_opts.strategy: 0 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compression_opts.enabled: false 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.target_file_size_base: 67108864 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-30T00:26:20.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.arena_block_size: 1048576 2024-03-30T00:26:20.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.disable_auto_compactions: 0 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-30T00:26:20.006 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.table_properties_collectors: 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.inplace_update_support: 0 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.bloom_locality: 0 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.max_successive_merges: 0 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.paranoid_file_checks: 0 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.force_consistency_checks: 1 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.report_bg_io_stats: 0 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.ttl: 2592000 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-30T00:26:20.007 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.enable_blob_files: false 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.min_blob_size: 0 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.blob_file_size: 268435456 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.blob_file_starting_level: 0 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 38f7c9e3-2341-45bb-993a-d1ef5905066b 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711758379749618, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-30T00:26:20.008 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711758379751326, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92394, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13745, "raw_average_key_size": 49, "raw_value_size": 84893, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711758379, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "38f7c9e3-2341-45bb-993a-d1ef5905066b", "db_session_id": "CYDJAPODNTDUQUQAYCYD", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711758379751455, "job": 1, "event": "recovery_finished"} 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560bca584000 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: DB pointer 0x560bca572000 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** DB Stats ** 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-30T00:26:20.009 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: L0 2/0 94.02 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: Sum 2/0 94.02 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** Compaction Stats [default] ** 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 86.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:26:20.010 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: Cumulative compaction: 0.00 GB write, 13.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: Interval compaction: 0.00 GB write, 13.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: Block cache BinnedLRUCache@0x560bca4e7350#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-30T00:26:20.011 INFO:journalctl@ceph.mon.a.smithi099.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: starting mon.a rank 0 at public addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] at bind addrs [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: mon.a@-1(???) e1 preinit fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: mon.a@-1(???).mds e1 new map 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: mon.a@-1(???).mds e1 print_map 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout: e1 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout: legacy client fscid: -1 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout: 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout: No filesystems configured 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T00:26:20.012 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-30T00:26:20.013 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-30T00:26:20.013 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: monmap e1: 1 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T00:26:20.013 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: fsmap 2024-03-30T00:26:20.013 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: osdmap e1: 0 total, 0 up, 0 in 2024-03-30T00:26:20.013 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:19 smithi099 ceph-mon[38079]: mgrmap e1: no daemons active 2024-03-30T00:26:20.546 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.138+0000 7f9499600700 1 Processor -- start 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.139+0000 7f9499600700 1 -- start start 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.139+0000 7f9499600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f94940791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.139+0000 7f9499600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940797a0 con 0x7f949407ad70 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.140+0000 7f9492a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f94940791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.140+0000 7f9492a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f94940791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48590/0 (socket says 172.21.15.99:48590) 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.140+0000 7f9492a00700 1 -- 172.21.15.99:0/3692366287 learned_addr learned my addr 172.21.15.99:0/3692366287 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.140+0000 7f9492a00700 1 -- 172.21.15.99:0/3692366287 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940798e0 con 0x7f949407ad70 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.140+0000 7f9492a00700 1 --2- 172.21.15.99:0/3692366287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f94940791d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f94840098a0 tx=0x7f9484009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9e24f2509f8a27 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.140+0000 7f9492000700 1 -- 172.21.15.99:0/3692366287 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9484004160 con 0x7f949407ad70 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.140+0000 7f9492000700 1 -- 172.21.15.99:0/3692366287 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f94840042c0 con 0x7f949407ad70 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.141+0000 7f9492000700 1 -- 172.21.15.99:0/3692366287 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9484011940 con 0x7f949407ad70 2024-03-30T00:26:20.547 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.141+0000 7f9499600700 1 -- 172.21.15.99:0/3692366287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 msgr2=0x7f94940791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.141+0000 7f9499600700 1 --2- 172.21.15.99:0/3692366287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f94940791d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f94840098a0 tx=0x7f9484009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.141+0000 7f9499600700 1 -- 172.21.15.99:0/3692366287 shutdown_connections 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.141+0000 7f9499600700 1 --2- 172.21.15.99:0/3692366287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f94940791d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.141+0000 7f9499600700 1 -- 172.21.15.99:0/3692366287 >> 172.21.15.99:0/3692366287 conn(0x7f94941019a0 msgr2=0x7f9494103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.141+0000 7f9499600700 1 -- 172.21.15.99:0/3692366287 shutdown_connections 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.141+0000 7f9499600700 1 -- 172.21.15.99:0/3692366287 wait complete. 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.142+0000 7f9499600700 1 Processor -- start 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.142+0000 7f9499600700 1 -- start start 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.143+0000 7f9499600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f949419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.143+0000 7f9499600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949419c870 con 0x7f949407ad70 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.143+0000 7f9492a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f949419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.143+0000 7f9492a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f949419c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48598/0 (socket says 172.21.15.99:48598) 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.143+0000 7f9492a00700 1 -- 172.21.15.99:0/3043159560 learned_addr learned my addr 172.21.15.99:0/3043159560 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:20.548 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.143+0000 7f9492a00700 1 -- 172.21.15.99:0/3043159560 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9484009550 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.143+0000 7f9492a00700 1 --2- 172.21.15.99:0/3043159560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f949419c330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9484003f20 tx=0x7f9484011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.144+0000 7f948b400700 1 -- 172.21.15.99:0/3043159560 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94840036a0 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.144+0000 7f948b400700 1 -- 172.21.15.99:0/3043159560 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f948401b440 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.144+0000 7f948b400700 1 -- 172.21.15.99:0/3043159560 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9484023400 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.144+0000 7f9499600700 1 -- 172.21.15.99:0/3043159560 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949419ca70 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.144+0000 7f9499600700 1 -- 172.21.15.99:0/3043159560 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949419cf10 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.144+0000 7f948b400700 1 -- 172.21.15.99:0/3043159560 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9484023970 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.144+0000 7f948b400700 1 -- 172.21.15.99:0/3043159560 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f948402cab0 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.145+0000 7f9499600700 1 -- 172.21.15.99:0/3043159560 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949410c790 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.148+0000 7f948b400700 1 -- 172.21.15.99:0/3043159560 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f948401b5b0 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.215+0000 7f9499600700 1 -- 172.21.15.99:0/3043159560 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f949419f7a0 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.221+0000 7f948b400700 1 -- 172.21.15.99:0/3043159560 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9484023d60 con 0x7f949407ad70 2024-03-30T00:26:20.549 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.222+0000 7f9499600700 1 -- 172.21.15.99:0/3043159560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 msgr2=0x7f949419c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:20.550 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.222+0000 7f9499600700 1 --2- 172.21.15.99:0/3043159560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f949419c330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9484003f20 tx=0x7f9484011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:20.550 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.222+0000 7f9499600700 1 -- 172.21.15.99:0/3043159560 shutdown_connections 2024-03-30T00:26:20.550 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.222+0000 7f9499600700 1 --2- 172.21.15.99:0/3043159560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f949407ad70 0x7f949419c330 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:20.550 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.222+0000 7f9499600700 1 -- 172.21.15.99:0/3043159560 >> 172.21.15.99:0/3043159560 conn(0x7f94941019a0 msgr2=0x7f9494102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:20.550 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.222+0000 7f9499600700 1 -- 172.21.15.99:0/3043159560 shutdown_connections 2024-03-30T00:26:20.550 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:20.222+0000 7f9499600700 1 -- 172.21.15.99:0/3043159560 wait complete. 2024-03-30T00:26:20.550 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-30T00:26:20.580 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-30T00:26:20.581 INFO:teuthology.orchestra.run.smithi099.stdout:Creating mgr... 2024-03-30T00:26:20.581 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-30T00:26:20.581 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-30T00:26:20.900 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.x 2024-03-30T00:26:20.900 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Failed to reset failed state of unit ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.x.service: Unit ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.x.service not loaded. 2024-03-30T00:26:21.087 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4.target.wants/ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.x.service → /etc/systemd/system/ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@.service. 2024-03-30T00:26:21.228 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:21 smithi099 systemd[1]: Starting Ceph mgr.x for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:26:21.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:21 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3043159560' entity='client.admin' 2024-03-30T00:26:21.613 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-30T00:26:21.613 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-03-30T00:26:21.629 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-30T00:26:21.629 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-30T00:26:21.629 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-03-30T00:26:21.629 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to enable service . firewalld.service is not available 2024-03-30T00:26:21.646 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-30T00:26:21.646 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-03-30T00:26:21.661 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-30T00:26:21.662 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-30T00:26:21.662 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-03-30T00:26:21.662 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-30T00:26:21.662 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr to start... 2024-03-30T00:26:21.662 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr... 2024-03-30T00:26:21.835 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:21 smithi099 bash[38317]: 417f140602d0e0fda9c4cdbf133e38a765cf8d8b1cc2e0e1d1be5488b79bf546 2024-03-30T00:26:21.835 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:21 smithi099 systemd[1]: Started Ceph mgr.x for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:26:22.440 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:22.441 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-30T00:26:22.441 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "03801b1a-ee2c-11ee-b647-cb9ed24678a4", 2024-03-30T00:26:22.441 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-03-30T00:26:22.441 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-30T00:26:22.441 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-30T00:26:22.441 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:22 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3230898127' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-30T00:26:22.442 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-30T00:26:22.442 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:22.442 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:22 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:22.128+0000 7fd846c7f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T00:26:22.442 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:22 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:22.260+0000 7fd846c7f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T00:26:22.442 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:22 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:22.377+0000 7fd846c7f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T00:26:22.453 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-30T00:26:22.453 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-30T00:26:22.453 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-03-30T00:26:22.453 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-30T00:26:22.453 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-30T00:26:22.453 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-30T00:26:22.454 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-30T00:26:22.455 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:22.456 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-30T00:26:22.456 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-30T00:26:22.456 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:22.456 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-30T00:26:22.456 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-30T00:26:22.456 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-03-30T00:26:14.506370+0000", 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:22.457 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.060+0000 7f61b2a00700 1 Processor -- start 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.060+0000 7f61b2a00700 1 -- start start 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.060+0000 7f61b2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac106860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.060+0000 7f61b2a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac106e30 con 0x7f61ac106440 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.061+0000 7f61abe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac106860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.061+0000 7f61abe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac106860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48634/0 (socket says 172.21.15.99:48634) 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.061+0000 7f61abe00700 1 -- 172.21.15.99:0/4247288197 learned_addr learned my addr 172.21.15.99:0/4247288197 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.061+0000 7f61abe00700 1 -- 172.21.15.99:0/4247288197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac107640 con 0x7f61ac106440 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.061+0000 7f61abe00700 1 --2- 172.21.15.99:0/4247288197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac106860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f619c0098a0 tx=0x7f619c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10044506508e9976 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.062+0000 7f61aaa00700 1 -- 172.21.15.99:0/4247288197 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f619c004160 con 0x7f61ac106440 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.062+0000 7f61aaa00700 1 -- 172.21.15.99:0/4247288197 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f619c0042c0 con 0x7f61ac106440 2024-03-30T00:26:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.062+0000 7f61aaa00700 1 -- 172.21.15.99:0/4247288197 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f619c011940 con 0x7f61ac106440 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.062+0000 7f61b2a00700 1 -- 172.21.15.99:0/4247288197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 msgr2=0x7f61ac106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.062+0000 7f61b2a00700 1 --2- 172.21.15.99:0/4247288197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac106860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f619c0098a0 tx=0x7f619c009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.063+0000 7f61b2a00700 1 -- 172.21.15.99:0/4247288197 shutdown_connections 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.063+0000 7f61b2a00700 1 --2- 172.21.15.99:0/4247288197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac106860 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.063+0000 7f61b2a00700 1 -- 172.21.15.99:0/4247288197 >> 172.21.15.99:0/4247288197 conn(0x7f61ac1019c0 msgr2=0x7f61ac103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.063+0000 7f61b2a00700 1 -- 172.21.15.99:0/4247288197 shutdown_connections 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.063+0000 7f61b2a00700 1 -- 172.21.15.99:0/4247288197 wait complete. 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.064+0000 7f61b2a00700 1 Processor -- start 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.064+0000 7f61b2a00700 1 -- start start 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.065+0000 7f61b2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.065+0000 7f61b2a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac19c800 con 0x7f61ac106440 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.065+0000 7f61abe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.065+0000 7f61abe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac19c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48640/0 (socket says 172.21.15.99:48640) 2024-03-30T00:26:22.459 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.065+0000 7f61abe00700 1 -- 172.21.15.99:0/3230898127 learned_addr learned my addr 172.21.15.99:0/3230898127 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.065+0000 7f61abe00700 1 -- 172.21.15.99:0/3230898127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c009550 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.066+0000 7f61abe00700 1 --2- 172.21.15.99:0/3230898127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac19c2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f619c003f20 tx=0x7f619c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.066+0000 7f61a8c00700 1 -- 172.21.15.99:0/3230898127 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f619c0036a0 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.066+0000 7f61a8c00700 1 -- 172.21.15.99:0/3230898127 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f619c01b440 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.066+0000 7f61a8c00700 1 -- 172.21.15.99:0/3230898127 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f619c023400 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.066+0000 7f61b2a00700 1 -- 172.21.15.99:0/3230898127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac19ca00 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.066+0000 7f61b2a00700 1 -- 172.21.15.99:0/3230898127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac19cea0 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.066+0000 7f61a8c00700 1 -- 172.21.15.99:0/3230898127 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f619c023970 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.066+0000 7f61a8c00700 1 -- 172.21.15.99:0/3230898127 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f619c02cdf0 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.067+0000 7f61b2a00700 1 -- 172.21.15.99:0/3230898127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac0623e0 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.070+0000 7f61a8c00700 1 -- 172.21.15.99:0/3230898127 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f619c023d60 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.144+0000 7f61b2a00700 1 -- 172.21.15.99:0/3230898127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f61ac10a870 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.145+0000 7f61a8c00700 1 -- 172.21.15.99:0/3230898127 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f619c01b5b0 con 0x7f61ac106440 2024-03-30T00:26:22.460 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.146+0000 7f61b2a00700 1 -- 172.21.15.99:0/3230898127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 msgr2=0x7f61ac19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:22.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.146+0000 7f61b2a00700 1 --2- 172.21.15.99:0/3230898127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac19c2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f619c003f20 tx=0x7f619c011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:22.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.146+0000 7f61b2a00700 1 -- 172.21.15.99:0/3230898127 shutdown_connections 2024-03-30T00:26:22.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.146+0000 7f61b2a00700 1 --2- 172.21.15.99:0/3230898127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61ac106440 0x7f61ac19c2c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:22.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.146+0000 7f61b2a00700 1 -- 172.21.15.99:0/3230898127 >> 172.21.15.99:0/3230898127 conn(0x7f61ac1019c0 msgr2=0x7f61ac102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:22.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.146+0000 7f61b2a00700 1 -- 172.21.15.99:0/3230898127 shutdown_connections 2024-03-30T00:26:22.461 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:22.146+0000 7f61b2a00700 1 -- 172.21.15.99:0/3230898127 wait complete. 2024-03-30T00:26:22.461 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (1/15)... 2024-03-30T00:26:22.738 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:22 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:22.492+0000 7fd846c7f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T00:26:23.646 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:23 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:23.332+0000 7fd846c7f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:26:23.988 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:23 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:23.637+0000 7fd846c7f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T00:26:23.989 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:23 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:23.745+0000 7fd846c7f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T00:26:23.989 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:23 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:23.881+0000 7fd846c7f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T00:26:24.355 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:24 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:24.074+0000 7fd846c7f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T00:26:24.355 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:24 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:24.238+0000 7fd846c7f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T00:26:24.610 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:24 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:24.347+0000 7fd846c7f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T00:26:25.124 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:24 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:24.856+0000 7fd846c7f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T00:26:25.268 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:25.268 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-30T00:26:25.268 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "03801b1a-ee2c-11ee-b647-cb9ed24678a4", 2024-03-30T00:26:25.268 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-03-30T00:26:25.268 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-30T00:26:25.268 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-30T00:26:25.268 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-30T00:26:25.268 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:25.268 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:25.269 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-30T00:26:25.270 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-30T00:26:25.270 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-30T00:26:25.278 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-30T00:26:25.278 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-30T00:26:25.278 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-30T00:26:25.278 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:25.278 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-30T00:26:25.278 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:25.279 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-03-30T00:26:14.506370+0000", 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T00:26:25.280 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.854+0000 7f4447000700 1 Processor -- start 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.854+0000 7f4447000700 1 -- start start 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.854+0000 7f4447000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f4440106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.854+0000 7f4447000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4440106e20 con 0x7f4440106430 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.855+0000 7f443fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f4440106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.855+0000 7f443fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f4440106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48652/0 (socket says 172.21.15.99:48652) 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.855+0000 7f443fe00700 1 -- 172.21.15.99:0/2748217805 learned_addr learned my addr 172.21.15.99:0/2748217805 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.855+0000 7f443fe00700 1 -- 172.21.15.99:0/2748217805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4440107630 con 0x7f4440106430 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.855+0000 7f443fe00700 1 --2- 172.21.15.99:0/2748217805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f4440106850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f44300098a0 tx=0x7f4430009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2e208455fa8784 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.856+0000 7f443ea00700 1 -- 172.21.15.99:0/2748217805 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4430004160 con 0x7f4440106430 2024-03-30T00:26:25.281 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.856+0000 7f443ea00700 1 -- 172.21.15.99:0/2748217805 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f44300042c0 con 0x7f4440106430 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.856+0000 7f443ea00700 1 -- 172.21.15.99:0/2748217805 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4430011940 con 0x7f4440106430 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.857+0000 7f4447000700 1 -- 172.21.15.99:0/2748217805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 msgr2=0x7f4440106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.857+0000 7f4447000700 1 --2- 172.21.15.99:0/2748217805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f4440106850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f44300098a0 tx=0x7f4430009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.857+0000 7f4447000700 1 -- 172.21.15.99:0/2748217805 shutdown_connections 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.857+0000 7f4447000700 1 --2- 172.21.15.99:0/2748217805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f4440106850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.857+0000 7f4447000700 1 -- 172.21.15.99:0/2748217805 >> 172.21.15.99:0/2748217805 conn(0x7f4440101990 msgr2=0x7f4440103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.857+0000 7f4447000700 1 -- 172.21.15.99:0/2748217805 shutdown_connections 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.857+0000 7f4447000700 1 -- 172.21.15.99:0/2748217805 wait complete. 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.858+0000 7f4447000700 1 Processor -- start 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.858+0000 7f4447000700 1 -- start start 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.859+0000 7f4447000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f444019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.859+0000 7f4447000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444019a630 con 0x7f4440106430 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.859+0000 7f443fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f444019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:25.282 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.859+0000 7f443fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f444019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48668/0 (socket says 172.21.15.99:48668) 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.859+0000 7f443fe00700 1 -- 172.21.15.99:0/3693488790 learned_addr learned my addr 172.21.15.99:0/3693488790 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.859+0000 7f443fe00700 1 -- 172.21.15.99:0/3693488790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4430009550 con 0x7f4440106430 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.860+0000 7f443fe00700 1 --2- 172.21.15.99:0/3693488790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f444019a0f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4430003f20 tx=0x7f4430011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.860+0000 7f443cc00700 1 -- 172.21.15.99:0/3693488790 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44300036a0 con 0x7f4440106430 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.860+0000 7f443cc00700 1 -- 172.21.15.99:0/3693488790 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f443001b440 con 0x7f4440106430 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.860+0000 7f4447000700 1 -- 172.21.15.99:0/3693488790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444019a830 con 0x7f4440106430 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.860+0000 7f443cc00700 1 -- 172.21.15.99:0/3693488790 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4430023400 con 0x7f4440106430 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.860+0000 7f4447000700 1 -- 172.21.15.99:0/3693488790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444019acd0 con 0x7f4440106430 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.860+0000 7f443cc00700 1 -- 172.21.15.99:0/3693488790 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f44300239c0 con 0x7f4440106430 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.860+0000 7f443cc00700 1 -- 172.21.15.99:0/3693488790 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f443002ce40 con 0x7f4440106430 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.861+0000 7f4447000700 1 -- 172.21.15.99:0/3693488790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44400623e0 con 0x7f4440106430 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.864+0000 7f443cc00700 1 -- 172.21.15.99:0/3693488790 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f4430023d60 con 0x7f4440106430 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.938+0000 7f4447000700 1 -- 172.21.15.99:0/3693488790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f444010a860 con 0x7f4440106430 2024-03-30T00:26:25.283 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.939+0000 7f443cc00700 1 -- 172.21.15.99:0/3693488790 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f443001b5b0 con 0x7f4440106430 2024-03-30T00:26:25.284 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.940+0000 7f4447000700 1 -- 172.21.15.99:0/3693488790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 msgr2=0x7f444019a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:25.284 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.940+0000 7f4447000700 1 --2- 172.21.15.99:0/3693488790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f444019a0f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4430003f20 tx=0x7f4430011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:25.284 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.940+0000 7f4447000700 1 -- 172.21.15.99:0/3693488790 shutdown_connections 2024-03-30T00:26:25.284 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.940+0000 7f4447000700 1 --2- 172.21.15.99:0/3693488790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4440106430 0x7f444019a0f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:25.284 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.940+0000 7f4447000700 1 -- 172.21.15.99:0/3693488790 >> 172.21.15.99:0/3693488790 conn(0x7f4440101990 msgr2=0x7f44401024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:25.284 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.940+0000 7f4447000700 1 -- 172.21.15.99:0/3693488790 shutdown_connections 2024-03-30T00:26:25.284 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:24.940+0000 7f4447000700 1 -- 172.21.15.99:0/3693488790 wait complete. 2024-03-30T00:26:25.284 INFO:teuthology.orchestra.run.smithi099.stdout:mgr not available, waiting (2/15)... 2024-03-30T00:26:25.488 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:25 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:25.363+0000 7fd846c7f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T00:26:25.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:25 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3693488790' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-30T00:26:25.989 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:25 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:25.612+0000 7fd846c7f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T00:26:25.989 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:25 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:25.733+0000 7fd846c7f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T00:26:26.988 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:26 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:26.687+0000 7fd846c7f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T00:26:26.988 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:26 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:26.815+0000 7fd846c7f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T00:26:27.488 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:27 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:27.005+0000 7fd846c7f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T00:26:28.238 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:27 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:27.821+0000 7fd846c7f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T00:26:28.239 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:28 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:28.057+0000 7fd846c7f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T00:26:28.738 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:28 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:28.315+0000 7fd846c7f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T00:26:29.738 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:29 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:29.454+0000 7fd846c7f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T00:26:30.238 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:29 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:29.825+0000 7fd846c7f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:26:30.238 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:29 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:29.942+0000 7fd846c7f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T00:26:30.515 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:30.192+0000 7fd846c7f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T00:26:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: Activating manager daemon x 2024-03-30T00:26:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: mgrmap e2: x(active, starting, since 0.00398191s) 2024-03-30T00:26:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: from='mgr.14100 172.21.15.99:0/1735328517' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-30T00:26:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: from='mgr.14100 172.21.15.99:0/1735328517' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-30T00:26:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: from='mgr.14100 172.21.15.99:0/1735328517' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-30T00:26:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: from='mgr.14100 172.21.15.99:0/1735328517' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:26:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: from='mgr.14100 172.21.15.99:0/1735328517' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-30T00:26:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: Manager daemon x is now available 2024-03-30T00:26:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: from='mgr.14100 172.21.15.99:0/1735328517' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-30T00:26:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: from='mgr.14100 172.21.15.99:0/1735328517' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-30T00:26:31.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: from='mgr.14100 172.21.15.99:0/1735328517' entity='mgr.x' 2024-03-30T00:26:31.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: from='mgr.14100 172.21.15.99:0/1735328517' entity='mgr.x' 2024-03-30T00:26:31.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:30 smithi099 ceph-mon[38079]: from='mgr.14100 172.21.15.99:0/1735328517' entity='mgr.x' 2024-03-30T00:26:32.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:32 smithi099 ceph-mon[38079]: mgrmap e3: x(active, since 1.00935s) 2024-03-30T00:26:32.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:32 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2283797060' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-30T00:26:33.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:33 smithi099 ceph-mon[38079]: mgrmap e4: x(active, since 2s) 2024-03-30T00:26:34.641 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:34.653 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-30T00:26:34.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsid": "03801b1a-ee2c-11ee-b647-cb9ed24678a4", 2024-03-30T00:26:34.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "health": { 2024-03-30T00:26:34.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-30T00:26:34.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-30T00:26:34.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-30T00:26:34.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:34.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-30T00:26:34.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-30T00:26:34.654 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 0 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "a" 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:34.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-30T00:26:34.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-30T00:26:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "restful" 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ], 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "modified": "2024-03-30T00:26:14.506370+0000", 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout }, 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-30T00:26:34.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.639+0000 7f32a1400700 1 Processor -- start 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.639+0000 7f32a1400700 1 -- start start 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.640+0000 7f32a1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c106800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.640+0000 7f32a1400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c106dd0 con 0x7f329c1063e0 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.640+0000 7f329a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c106800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.640+0000 7f329a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c106800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47672/0 (socket says 172.21.15.99:47672) 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.640+0000 7f329a800700 1 -- 172.21.15.99:0/3215149281 learned_addr learned my addr 172.21.15.99:0/3215149281 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.640+0000 7f329a800700 1 -- 172.21.15.99:0/3215149281 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c1075e0 con 0x7f329c1063e0 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.641+0000 7f329a800700 1 --2- 172.21.15.99:0/3215149281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c106800 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f328c0098a0 tx=0x7f328c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528d6067b5342fb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.641+0000 7f3299400700 1 -- 172.21.15.99:0/3215149281 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f328c004160 con 0x7f329c1063e0 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.641+0000 7f3299400700 1 -- 172.21.15.99:0/3215149281 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f328c0042c0 con 0x7f329c1063e0 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.641+0000 7f3299400700 1 -- 172.21.15.99:0/3215149281 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f328c011940 con 0x7f329c1063e0 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.641+0000 7f32a1400700 1 -- 172.21.15.99:0/3215149281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 msgr2=0x7f329c106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:34.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.641+0000 7f32a1400700 1 --2- 172.21.15.99:0/3215149281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c106800 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f328c0098a0 tx=0x7f328c009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.641+0000 7f32a1400700 1 -- 172.21.15.99:0/3215149281 shutdown_connections 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.641+0000 7f32a1400700 1 --2- 172.21.15.99:0/3215149281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c106800 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.641+0000 7f32a1400700 1 -- 172.21.15.99:0/3215149281 >> 172.21.15.99:0/3215149281 conn(0x7f329c101960 msgr2=0x7f329c103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.642+0000 7f32a1400700 1 -- 172.21.15.99:0/3215149281 shutdown_connections 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.642+0000 7f32a1400700 1 -- 172.21.15.99:0/3215149281 wait complete. 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.642+0000 7f32a1400700 1 Processor -- start 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.643+0000 7f32a1400700 1 -- start start 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.643+0000 7f32a1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.643+0000 7f32a1400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c19a620 con 0x7f329c1063e0 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.643+0000 7f329a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.643+0000 7f329a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c19a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47680/0 (socket says 172.21.15.99:47680) 2024-03-30T00:26:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.643+0000 7f329a800700 1 -- 172.21.15.99:0/2283797060 learned_addr learned my addr 172.21.15.99:0/2283797060 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.643+0000 7f329a800700 1 -- 172.21.15.99:0/2283797060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c009550 con 0x7f329c1063e0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.644+0000 7f329a800700 1 --2- 172.21.15.99:0/2283797060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c19a0e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f328c003f20 tx=0x7f328c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.644+0000 7f3293400700 1 -- 172.21.15.99:0/2283797060 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f328c0036a0 con 0x7f329c1063e0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.644+0000 7f3293400700 1 -- 172.21.15.99:0/2283797060 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f328c003c70 con 0x7f329c1063e0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.644+0000 7f32a1400700 1 -- 172.21.15.99:0/2283797060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c19a820 con 0x7f329c1063e0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.644+0000 7f3293400700 1 -- 172.21.15.99:0/2283797060 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f328c01be80 con 0x7f329c1063e0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.644+0000 7f32a1400700 1 -- 172.21.15.99:0/2283797060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c19acc0 con 0x7f329c1063e0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.645+0000 7f3293400700 1 -- 172.21.15.99:0/2283797060 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 3) v1 ==== 45416+0+0 (secure 0 0 0) 0x7f328c003800 con 0x7f329c1063e0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.645+0000 7f32a1400700 1 -- 172.21.15.99:0/2283797060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c10a810 con 0x7f329c1063e0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.645+0000 7f3293400700 1 --2- 172.21.15.99:0/2283797060 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f32800385d0 0x7f328003aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.645+0000 7f3293400700 1 -- 172.21.15.99:0/2283797060 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f328c04c350 con 0x7f329c1063e0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.646+0000 7f3299e00700 1 --2- 172.21.15.99:0/2283797060 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f32800385d0 0x7f328003aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.646+0000 7f3299e00700 1 --2- 172.21.15.99:0/2283797060 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f32800385d0 0x7f328003aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3288006fd0 tx=0x7f3288006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:34.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.652+0000 7f3293400700 1 -- 172.21.15.99:0/2283797060 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f328c00ba30 con 0x7f329c1063e0 2024-03-30T00:26:34.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.938+0000 7f32a1400700 1 -- 172.21.15.99:0/2283797060 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f329c19d550 con 0x7f329c1063e0 2024-03-30T00:26:34.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.939+0000 7f3293400700 1 -- 172.21.15.99:0/2283797060 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f328c019a90 con 0x7f329c1063e0 2024-03-30T00:26:34.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.941+0000 7f32a1400700 1 -- 172.21.15.99:0/2283797060 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f32800385d0 msgr2=0x7f328003aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:34.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.941+0000 7f32a1400700 1 --2- 172.21.15.99:0/2283797060 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f32800385d0 0x7f328003aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3288006fd0 tx=0x7f3288006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:34.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.941+0000 7f32a1400700 1 -- 172.21.15.99:0/2283797060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 msgr2=0x7f329c19a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:34.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.941+0000 7f32a1400700 1 --2- 172.21.15.99:0/2283797060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c19a0e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f328c003f20 tx=0x7f328c011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:34.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.941+0000 7f32a1400700 1 -- 172.21.15.99:0/2283797060 shutdown_connections 2024-03-30T00:26:34.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.941+0000 7f32a1400700 1 --2- 172.21.15.99:0/2283797060 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f32800385d0 0x7f328003aa90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:34.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.941+0000 7f32a1400700 1 --2- 172.21.15.99:0/2283797060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329c1063e0 0x7f329c19a0e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:34.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.941+0000 7f32a1400700 1 -- 172.21.15.99:0/2283797060 >> 172.21.15.99:0/2283797060 conn(0x7f329c101960 msgr2=0x7f329c102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:34.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.941+0000 7f32a1400700 1 -- 172.21.15.99:0/2283797060 shutdown_connections 2024-03-30T00:26:34.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:31.941+0000 7f32a1400700 1 -- 172.21.15.99:0/2283797060 wait complete. 2024-03-30T00:26:34.663 INFO:teuthology.orchestra.run.smithi099.stdout:mgr is available 2024-03-30T00:26:35.674 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [global] 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout fsid = 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout [osd] 2024-03-30T00:26:35.675 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.023+0000 7f11a7200700 1 Processor -- start 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.023+0000 7f11a7200700 1 -- start start 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.023+0000 7f11a7200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a00791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.023+0000 7f11a7200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a0079780 con 0x7f11a007ad50 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.024+0000 7f119fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a00791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.024+0000 7f119fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a00791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49582/0 (socket says 172.21.15.99:49582) 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.024+0000 7f119fe00700 1 -- 172.21.15.99:0/1456995547 learned_addr learned my addr 172.21.15.99:0/1456995547 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.024+0000 7f119fe00700 1 -- 172.21.15.99:0/1456995547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a00798c0 con 0x7f11a007ad50 2024-03-30T00:26:35.676 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.024+0000 7f119fe00700 1 --2- 172.21.15.99:0/1456995547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a00791b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f11900098a0 tx=0x7f1190009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4217688d15776a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.024+0000 7f119ea00700 1 -- 172.21.15.99:0/1456995547 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1190004160 con 0x7f11a007ad50 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.025+0000 7f119ea00700 1 -- 172.21.15.99:0/1456995547 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f11900042c0 con 0x7f11a007ad50 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.025+0000 7f119ea00700 1 -- 172.21.15.99:0/1456995547 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1190011940 con 0x7f11a007ad50 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.025+0000 7f11a7200700 1 -- 172.21.15.99:0/1456995547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 msgr2=0x7f11a00791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.025+0000 7f11a7200700 1 --2- 172.21.15.99:0/1456995547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a00791b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f11900098a0 tx=0x7f1190009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.025+0000 7f11a7200700 1 -- 172.21.15.99:0/1456995547 shutdown_connections 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.025+0000 7f11a7200700 1 --2- 172.21.15.99:0/1456995547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a00791b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.025+0000 7f11a7200700 1 -- 172.21.15.99:0/1456995547 >> 172.21.15.99:0/1456995547 conn(0x7f11a0101990 msgr2=0x7f11a0103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.025+0000 7f11a7200700 1 -- 172.21.15.99:0/1456995547 shutdown_connections 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.025+0000 7f11a7200700 1 -- 172.21.15.99:0/1456995547 wait complete. 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.026+0000 7f11a7200700 1 Processor -- start 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.026+0000 7f11a7200700 1 -- start start 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.027+0000 7f11a7200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:35.677 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.027+0000 7f11a7200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a019c7e0 con 0x7f11a007ad50 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.027+0000 7f119fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.027+0000 7f119fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49592/0 (socket says 172.21.15.99:49592) 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.027+0000 7f119fe00700 1 -- 172.21.15.99:0/4090515024 learned_addr learned my addr 172.21.15.99:0/4090515024 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.027+0000 7f119fe00700 1 -- 172.21.15.99:0/4090515024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1190009550 con 0x7f11a007ad50 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.027+0000 7f119fe00700 1 --2- 172.21.15.99:0/4090515024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a019c2a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1190003f20 tx=0x7f1190011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.028+0000 7f119cc00700 1 -- 172.21.15.99:0/4090515024 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11900037d0 con 0x7f11a007ad50 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.028+0000 7f119cc00700 1 -- 172.21.15.99:0/4090515024 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f119001b440 con 0x7f11a007ad50 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.028+0000 7f11a7200700 1 -- 172.21.15.99:0/4090515024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a019c9e0 con 0x7f11a007ad50 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.028+0000 7f119cc00700 1 -- 172.21.15.99:0/4090515024 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1190023590 con 0x7f11a007ad50 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.028+0000 7f11a7200700 1 -- 172.21.15.99:0/4090515024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a019ce80 con 0x7f11a007ad50 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.029+0000 7f119cc00700 1 -- 172.21.15.99:0/4090515024 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 45522+0+0 (secure 0 0 0) 0x7f119001b5b0 con 0x7f11a007ad50 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.029+0000 7f11a7200700 1 -- 172.21.15.99:0/4090515024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a010c7c0 con 0x7f11a007ad50 2024-03-30T00:26:35.678 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.029+0000 7f119cc00700 1 --2- 172.21.15.99:0/4090515024 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f1184038700 0x7f118403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.029+0000 7f119cc00700 1 -- 172.21.15.99:0/4090515024 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f119004c620 con 0x7f11a007ad50 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.030+0000 7f119f400700 1 --2- 172.21.15.99:0/4090515024 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f1184038700 0x7f118403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.030+0000 7f119f400700 1 --2- 172.21.15.99:0/4090515024 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f1184038700 0x7f118403abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f118c006fd0 tx=0x7f118c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.035+0000 7f119cc00700 1 -- 172.21.15.99:0/4090515024 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f119001b920 con 0x7f11a007ad50 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.232+0000 7f11a7200700 1 -- 172.21.15.99:0/4090515024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f11a010ca00 con 0x7f11a007ad50 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.233+0000 7f119cc00700 1 -- 172.21.15.99:0/4090515024 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f119000bbd0 con 0x7f11a007ad50 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.235+0000 7f11a7200700 1 -- 172.21.15.99:0/4090515024 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f1184038700 msgr2=0x7f118403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.235+0000 7f11a7200700 1 --2- 172.21.15.99:0/4090515024 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f1184038700 0x7f118403abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f118c006fd0 tx=0x7f118c006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.235+0000 7f11a7200700 1 -- 172.21.15.99:0/4090515024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 msgr2=0x7f11a019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.235+0000 7f11a7200700 1 --2- 172.21.15.99:0/4090515024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a019c2a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1190003f20 tx=0x7f1190011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.235+0000 7f11a7200700 1 -- 172.21.15.99:0/4090515024 shutdown_connections 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.235+0000 7f11a7200700 1 --2- 172.21.15.99:0/4090515024 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f1184038700 0x7f118403abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.235+0000 7f11a7200700 1 --2- 172.21.15.99:0/4090515024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11a007ad50 0x7f11a019c2a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:35.679 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.235+0000 7f11a7200700 1 -- 172.21.15.99:0/4090515024 >> 172.21.15.99:0/4090515024 conn(0x7f11a0101990 msgr2=0x7f11a0102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:35.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.236+0000 7f11a7200700 1 -- 172.21.15.99:0/4090515024 shutdown_connections 2024-03-30T00:26:35.680 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:35.236+0000 7f11a7200700 1 -- 172.21.15.99:0/4090515024 wait complete. 2024-03-30T00:26:35.680 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling cephadm module... 2024-03-30T00:26:35.792 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:35 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4090515024' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-30T00:26:37.106 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:36 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3061079739' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-30T00:26:37.106 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:36 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: ignoring --setuser ceph since I am not root 2024-03-30T00:26:37.106 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:36 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: ignoring --setgroup ceph since I am not root 2024-03-30T00:26:37.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.032+0000 7f6c91200700 1 Processor -- start 2024-03-30T00:26:37.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.033+0000 7f6c91200700 1 -- start start 2024-03-30T00:26:37.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.033+0000 7f6c91200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c0791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:37.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.033+0000 7f6c91200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c0797a0 con 0x7f6c8c07ad70 2024-03-30T00:26:37.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.033+0000 7f6c89e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c0791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:37.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.033+0000 7f6c89e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c0791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49598/0 (socket says 172.21.15.99:49598) 2024-03-30T00:26:37.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.033+0000 7f6c89e00700 1 -- 172.21.15.99:0/4141370944 learned_addr learned my addr 172.21.15.99:0/4141370944 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:37.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.034+0000 7f6c89e00700 1 -- 172.21.15.99:0/4141370944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c0798e0 con 0x7f6c8c07ad70 2024-03-30T00:26:37.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.034+0000 7f6c89e00700 1 --2- 172.21.15.99:0/4141370944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c0791d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0098a0 tx=0x7f6c7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51944c8678080b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:37.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.034+0000 7f6c89400700 1 -- 172.21.15.99:0/4141370944 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c004160 con 0x7f6c8c07ad70 2024-03-30T00:26:37.112 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.034+0000 7f6c89400700 1 -- 172.21.15.99:0/4141370944 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f6c7c0042c0 con 0x7f6c8c07ad70 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.034+0000 7f6c89400700 1 -- 172.21.15.99:0/4141370944 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c011940 con 0x7f6c8c07ad70 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.035+0000 7f6c91200700 1 -- 172.21.15.99:0/4141370944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 msgr2=0x7f6c8c0791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.035+0000 7f6c91200700 1 --2- 172.21.15.99:0/4141370944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c0791d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0098a0 tx=0x7f6c7c009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.035+0000 7f6c91200700 1 -- 172.21.15.99:0/4141370944 shutdown_connections 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.035+0000 7f6c91200700 1 --2- 172.21.15.99:0/4141370944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c0791d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.035+0000 7f6c91200700 1 -- 172.21.15.99:0/4141370944 >> 172.21.15.99:0/4141370944 conn(0x7f6c8c1019a0 msgr2=0x7f6c8c103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.035+0000 7f6c91200700 1 -- 172.21.15.99:0/4141370944 shutdown_connections 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.035+0000 7f6c91200700 1 -- 172.21.15.99:0/4141370944 wait complete. 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.036+0000 7f6c91200700 1 Processor -- start 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.036+0000 7f6c91200700 1 -- start start 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.036+0000 7f6c91200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.036+0000 7f6c91200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c19c870 con 0x7f6c8c07ad70 2024-03-30T00:26:37.113 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.036+0000 7f6c89e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.037+0000 7f6c89e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c19c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49614/0 (socket says 172.21.15.99:49614) 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.037+0000 7f6c89e00700 1 -- 172.21.15.99:0/3061079739 learned_addr learned my addr 172.21.15.99:0/3061079739 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.037+0000 7f6c89e00700 1 -- 172.21.15.99:0/3061079739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c009550 con 0x7f6c8c07ad70 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.037+0000 7f6c89e00700 1 --2- 172.21.15.99:0/3061079739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c19c330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c003f20 tx=0x7f6c7c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.037+0000 7f6c82a00700 1 -- 172.21.15.99:0/3061079739 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c0036a0 con 0x7f6c8c07ad70 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.037+0000 7f6c91200700 1 -- 172.21.15.99:0/3061079739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8c19ca70 con 0x7f6c8c07ad70 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.037+0000 7f6c82a00700 1 -- 172.21.15.99:0/3061079739 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f6c7c01b440 con 0x7f6c8c07ad70 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.037+0000 7f6c82a00700 1 -- 172.21.15.99:0/3061079739 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c023400 con 0x7f6c8c07ad70 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.037+0000 7f6c91200700 1 -- 172.21.15.99:0/3061079739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c19cf10 con 0x7f6c8c07ad70 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.038+0000 7f6c82a00700 1 -- 172.21.15.99:0/3061079739 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 45522+0+0 (secure 0 0 0) 0x7f6c7c003800 con 0x7f6c8c07ad70 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.038+0000 7f6c91200700 1 -- 172.21.15.99:0/3061079739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8c10c790 con 0x7f6c8c07ad70 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.039+0000 7f6c82a00700 1 --2- 172.21.15.99:0/3061079739 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f6c700386b0 0x7f6c7003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:37.114 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.039+0000 7f6c82a00700 1 -- 172.21.15.99:0/3061079739 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6c7c04c670 con 0x7f6c8c07ad70 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.039+0000 7f6c83e00700 1 --2- 172.21.15.99:0/3061079739 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f6c700386b0 0x7f6c7003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.040+0000 7f6c83e00700 1 --2- 172.21.15.99:0/3061079739 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f6c700386b0 0x7f6c7003ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c74006fd0 tx=0x7f6c74006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.044+0000 7f6c82a00700 1 -- 172.21.15.99:0/3061079739 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6c7c00ba30 con 0x7f6c8c07ad70 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.299+0000 7f6c91200700 1 -- 172.21.15.99:0/3061079739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f6c8c10c9d0 con 0x7f6c8c07ad70 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.767+0000 7f6c82a00700 1 -- 172.21.15.99:0/3061079739 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f6c7c023760 con 0x7f6c8c07ad70 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.768+0000 7f6c82a00700 1 -- 172.21.15.99:0/3061079739 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f6c7c019a90 con 0x7f6c8c07ad70 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.771+0000 7f6c91200700 1 -- 172.21.15.99:0/3061079739 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f6c700386b0 msgr2=0x7f6c7003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.771+0000 7f6c91200700 1 --2- 172.21.15.99:0/3061079739 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f6c700386b0 0x7f6c7003ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c74006fd0 tx=0x7f6c74006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.771+0000 7f6c91200700 1 -- 172.21.15.99:0/3061079739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 msgr2=0x7f6c8c19c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.771+0000 7f6c91200700 1 --2- 172.21.15.99:0/3061079739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c19c330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c003f20 tx=0x7f6c7c011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.771+0000 7f6c91200700 1 -- 172.21.15.99:0/3061079739 shutdown_connections 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.771+0000 7f6c91200700 1 --2- 172.21.15.99:0/3061079739 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7f6c700386b0 0x7f6c7003ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.771+0000 7f6c91200700 1 --2- 172.21.15.99:0/3061079739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c8c07ad70 0x7f6c8c19c330 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:37.115 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.771+0000 7f6c91200700 1 -- 172.21.15.99:0/3061079739 >> 172.21.15.99:0/3061079739 conn(0x7f6c8c1019a0 msgr2=0x7f6c8c102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:37.116 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.771+0000 7f6c91200700 1 -- 172.21.15.99:0/3061079739 shutdown_connections 2024-03-30T00:26:37.116 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:36.771+0000 7f6c91200700 1 -- 172.21.15.99:0/3061079739 wait complete. 2024-03-30T00:26:37.738 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:37 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:37.392+0000 7f3aa7b6b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T00:26:37.738 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:37 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:37.515+0000 7f3aa7b6b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T00:26:37.738 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:37 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:37.635+0000 7f3aa7b6b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T00:26:38.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:37 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3061079739' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-30T00:26:38.010 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:37 smithi099 ceph-mon[38079]: mgrmap e5: x(active, since 6s) 2024-03-30T00:26:38.010 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:37 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:37.747+0000 7f3aa7b6b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T00:26:38.153 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-30T00:26:38.153 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-30T00:26:38.153 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-30T00:26:38.153 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.525+0000 7fb4eb000700 1 Processor -- start 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.525+0000 7fb4eb000700 1 -- start start 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.526+0000 7fb4eb000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e4106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.526+0000 7fb4eb000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e4106e20 con 0x7fb4e4106430 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.526+0000 7fb4e3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e4106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.526+0000 7fb4e3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e4106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49654/0 (socket says 172.21.15.99:49654) 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.526+0000 7fb4e3e00700 1 -- 172.21.15.99:0/3733493896 learned_addr learned my addr 172.21.15.99:0/3733493896 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.527+0000 7fb4e3e00700 1 -- 172.21.15.99:0/3733493896 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e4107630 con 0x7fb4e4106430 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.527+0000 7fb4e3e00700 1 --2- 172.21.15.99:0/3733493896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e4106850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40098a0 tx=0x7fb4d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e0f69fff3d8a67 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.527+0000 7fb4e2a00700 1 -- 172.21.15.99:0/3733493896 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4d4004160 con 0x7fb4e4106430 2024-03-30T00:26:38.154 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.527+0000 7fb4e2a00700 1 -- 172.21.15.99:0/3733493896 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb4d40042c0 con 0x7fb4e4106430 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.527+0000 7fb4e2a00700 1 -- 172.21.15.99:0/3733493896 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4d4011940 con 0x7fb4e4106430 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.528+0000 7fb4eb000700 1 -- 172.21.15.99:0/3733493896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 msgr2=0x7fb4e4106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.528+0000 7fb4eb000700 1 --2- 172.21.15.99:0/3733493896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e4106850 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40098a0 tx=0x7fb4d4009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.528+0000 7fb4eb000700 1 -- 172.21.15.99:0/3733493896 shutdown_connections 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.528+0000 7fb4eb000700 1 --2- 172.21.15.99:0/3733493896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e4106850 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.528+0000 7fb4eb000700 1 -- 172.21.15.99:0/3733493896 >> 172.21.15.99:0/3733493896 conn(0x7fb4e4101990 msgr2=0x7fb4e4103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.528+0000 7fb4eb000700 1 -- 172.21.15.99:0/3733493896 shutdown_connections 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.528+0000 7fb4eb000700 1 -- 172.21.15.99:0/3733493896 wait complete. 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.529+0000 7fb4eb000700 1 Processor -- start 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.529+0000 7fb4eb000700 1 -- start start 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.529+0000 7fb4eb000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e407a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.529+0000 7fb4eb000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e407af30 con 0x7fb4e4106430 2024-03-30T00:26:38.155 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.530+0000 7fb4e3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e407a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.530+0000 7fb4e3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e407a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49670/0 (socket says 172.21.15.99:49670) 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.530+0000 7fb4e3e00700 1 -- 172.21.15.99:0/3579077737 learned_addr learned my addr 172.21.15.99:0/3579077737 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.530+0000 7fb4e3e00700 1 -- 172.21.15.99:0/3579077737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d4009550 con 0x7fb4e4106430 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.530+0000 7fb4e3e00700 1 --2- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e407a9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4003f20 tx=0x7fb4d4011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.530+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3579077737 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4d40036a0 con 0x7fb4e4106430 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.530+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3579077737 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb4d401b440 con 0x7fb4e4106430 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.530+0000 7fb4eb000700 1 -- 172.21.15.99:0/3579077737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e4079170 con 0x7fb4e4106430 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.530+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3579077737 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4d4023400 con 0x7fb4e4106430 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.531+0000 7fb4eb000700 1 -- 172.21.15.99:0/3579077737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e4079610 con 0x7fb4e4106430 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.532+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3579077737 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fb4d4023670 con 0x7fb4e4106430 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.532+0000 7fb4eb000700 1 -- 172.21.15.99:0/3579077737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e40623e0 con 0x7fb4e4106430 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.532+0000 7fb4e0c00700 1 --2- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fb4c8038750 0x7fb4c803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.532+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3579077737 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb4d404d590 con 0x7fb4e4106430 2024-03-30T00:26:38.156 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.532+0000 7fb4e3400700 1 -- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fb4c8038750 msgr2=0x7fb4c803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1133935143 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.532+0000 7fb4e3400700 1 --2- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fb4c8038750 0x7fb4c803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.541+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3579077737 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb4d4023920 con 0x7fb4e4106430 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.733+0000 7fb4e3400700 1 -- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fb4c8038750 msgr2=0x7fb4c803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1133935143 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.733+0000 7fb4e3400700 1 --2- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fb4c8038750 0x7fb4c803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.807+0000 7fb4eb000700 1 -- 172.21.15.99:0/3579077737 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb4e4079d90 con 0x7fb4e4106430 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.808+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3579077737 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fb4d402bb40 con 0x7fb4e4106430 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.810+0000 7fb4eb000700 1 -- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fb4c8038750 msgr2=0x7fb4c803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.810+0000 7fb4eb000700 1 --2- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fb4c8038750 0x7fb4c803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.810+0000 7fb4eb000700 1 -- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 msgr2=0x7fb4e407a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.810+0000 7fb4eb000700 1 --2- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e407a9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4003f20 tx=0x7fb4d4011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.810+0000 7fb4eb000700 1 -- 172.21.15.99:0/3579077737 shutdown_connections 2024-03-30T00:26:38.157 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.810+0000 7fb4eb000700 1 --2- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fb4c8038750 0x7fb4c803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:38.158 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.810+0000 7fb4eb000700 1 --2- 172.21.15.99:0/3579077737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4106430 0x7fb4e407a9f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:38.158 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.810+0000 7fb4eb000700 1 -- 172.21.15.99:0/3579077737 >> 172.21.15.99:0/3579077737 conn(0x7fb4e4101990 msgr2=0x7fb4e41024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:38.158 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.811+0000 7fb4eb000700 1 -- 172.21.15.99:0/3579077737 shutdown_connections 2024-03-30T00:26:38.158 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:37.811+0000 7fb4eb000700 1 -- 172.21.15.99:0/3579077737 wait complete. 2024-03-30T00:26:38.158 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for the mgr to restart... 2024-03-30T00:26:38.158 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr epoch 5... 2024-03-30T00:26:38.926 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:38 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3579077737' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-30T00:26:38.926 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:38 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:38.606+0000 7f3aa7b6b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:26:38.926 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:38 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:38.917+0000 7f3aa7b6b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T00:26:39.238 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:39 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:39.026+0000 7f3aa7b6b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T00:26:39.238 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:39 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:39.161+0000 7f3aa7b6b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T00:26:39.634 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:39 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:39.355+0000 7f3aa7b6b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T00:26:39.634 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:39 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:39.518+0000 7f3aa7b6b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T00:26:39.988 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:39 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:39.626+0000 7f3aa7b6b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T00:26:40.488 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:40 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:40.083+0000 7f3aa7b6b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T00:26:40.934 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:40 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:40.562+0000 7f3aa7b6b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T00:26:40.935 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:40 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:40.807+0000 7f3aa7b6b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T00:26:40.935 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:40 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:40.926+0000 7f3aa7b6b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T00:26:42.174 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:41 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:41.855+0000 7f3aa7b6b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T00:26:42.175 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:41 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:41.977+0000 7f3aa7b6b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T00:26:42.175 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:42 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:42.165+0000 7f3aa7b6b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T00:26:43.238 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:42 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:42.955+0000 7f3aa7b6b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T00:26:43.239 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:43 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:43.189+0000 7f3aa7b6b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T00:26:43.738 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:43 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:43.436+0000 7f3aa7b6b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T00:26:44.940 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:44 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:44.559+0000 7f3aa7b6b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T00:26:44.940 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:44 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:44.931+0000 7f3aa7b6b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:26:45.238 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:45.047+0000 7f3aa7b6b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T00:26:45.738 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:26:45.297+0000 7f3aa7b6b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T00:26:46.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: Active manager daemon x restarted 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: Activating manager daemon x 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: osdmap e2: 0 total, 0 up, 0 in 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: mgrmap e6: x(active, starting, since 0.00416245s) 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: Manager daemon x is now available 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:26:46.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:26:46.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-30T00:26:46.240 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:45 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-30T00:26:46.644 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-30T00:26:46.644 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-30T00:26:46.644 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-30T00:26:46.644 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-30T00:26:46.644 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.567+0000 7fdff0a00700 1 Processor -- start 2024-03-30T00:26:46.644 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.568+0000 7fdff0a00700 1 -- start start 2024-03-30T00:26:46.644 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.568+0000 7fdff0a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec0791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.568+0000 7fdff0a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec079780 con 0x7fdfec07ad50 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.568+0000 7fdfe9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec0791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.568+0000 7fdfe9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec0791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49676/0 (socket says 172.21.15.99:49676) 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.568+0000 7fdfe9e00700 1 -- 172.21.15.99:0/1577004813 learned_addr learned my addr 172.21.15.99:0/1577004813 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.569+0000 7fdfe9e00700 1 -- 172.21.15.99:0/1577004813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec0798c0 con 0x7fdfec07ad50 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.569+0000 7fdfe9e00700 1 --2- 172.21.15.99:0/1577004813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec0791b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0098a0 tx=0x7fdfdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df2232bf0004d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.569+0000 7fdfe8a00700 1 -- 172.21.15.99:0/1577004813 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfdc004160 con 0x7fdfec07ad50 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.569+0000 7fdfe8a00700 1 -- 172.21.15.99:0/1577004813 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fdfdc0042c0 con 0x7fdfec07ad50 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.569+0000 7fdfe8a00700 1 -- 172.21.15.99:0/1577004813 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfdc011940 con 0x7fdfec07ad50 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.570+0000 7fdff0a00700 1 -- 172.21.15.99:0/1577004813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 msgr2=0x7fdfec0791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.570+0000 7fdff0a00700 1 --2- 172.21.15.99:0/1577004813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec0791b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0098a0 tx=0x7fdfdc009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.570+0000 7fdff0a00700 1 -- 172.21.15.99:0/1577004813 shutdown_connections 2024-03-30T00:26:46.645 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.570+0000 7fdff0a00700 1 --2- 172.21.15.99:0/1577004813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec0791b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.570+0000 7fdff0a00700 1 -- 172.21.15.99:0/1577004813 >> 172.21.15.99:0/1577004813 conn(0x7fdfec1019c0 msgr2=0x7fdfec103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.570+0000 7fdff0a00700 1 -- 172.21.15.99:0/1577004813 shutdown_connections 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.570+0000 7fdff0a00700 1 -- 172.21.15.99:0/1577004813 wait complete. 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.571+0000 7fdff0a00700 1 Processor -- start 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.571+0000 7fdff0a00700 1 -- start start 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.571+0000 7fdff0a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.571+0000 7fdff0a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec19c790 con 0x7fdfec07ad50 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.572+0000 7fdfe9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.572+0000 7fdfe9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec19c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49680/0 (socket says 172.21.15.99:49680) 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.572+0000 7fdfe9e00700 1 -- 172.21.15.99:0/1113093562 learned_addr learned my addr 172.21.15.99:0/1113093562 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.572+0000 7fdfe9e00700 1 -- 172.21.15.99:0/1113093562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc009550 con 0x7fdfec07ad50 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.572+0000 7fdfe9e00700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec19c250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc003f20 tx=0x7fdfdc011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.572+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfdc0037d0 con 0x7fdfec07ad50 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.572+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fdfdc01b440 con 0x7fdfec07ad50 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.573+0000 7fdff0a00700 1 -- 172.21.15.99:0/1113093562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec19c990 con 0x7fdfec07ad50 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.573+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfdc023590 con 0x7fdfec07ad50 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.573+0000 7fdff0a00700 1 -- 172.21.15.99:0/1113093562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec19ce30 con 0x7fdfec07ad50 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.574+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fdfdc003930 con 0x7fdfec07ad50 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.574+0000 7fdfe2a00700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 0x7fdfd003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.574+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 --> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfd003b2d0 con 0x7fdfd0038700 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.574+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdfdc04c690 con 0x7fdfec07ad50 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.574+0000 7fdfe9400700 1 -- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 msgr2=0x7fdfd003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1133935143 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.574+0000 7fdfe9400700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 0x7fdfd003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.775+0000 7fdfe9400700 1 -- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 msgr2=0x7fdfd003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1133935143 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:38.775+0000 7fdfe9400700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 0x7fdfd003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:39.176+0000 7fdfe9400700 1 -- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 msgr2=0x7fdfd003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1133935143 2024-03-30T00:26:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:39.176+0000 7fdfe9400700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 0x7fdfd003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:39.977+0000 7fdfe9400700 1 -- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 msgr2=0x7fdfd003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1133935143 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:39.977+0000 7fdfe9400700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 0x7fdfd003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:41.578+0000 7fdfe9400700 1 -- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 msgr2=0x7fdfd003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1133935143 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:41.578+0000 7fdfe9400700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 0x7fdfd003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:44.781+0000 7fdfe9400700 1 -- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 msgr2=0x7fdfd003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1133935143 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:44.781+0000 7fdfe9400700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 0x7fdfd003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:45.302+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mgrmap(e 6) v1 ==== 45300+0+0 (secure 0 0 0) 0x7fdfdc0238f0 con 0x7fdfec07ad50 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:45.302+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 msgr2=0x7fdfd003abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:45.302+0000 7fdfe2a00700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 0x7fdfd003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.303+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 7) v1 ==== 45427+0+0 (secure 0 0 0) 0x7fdfdc04e120 con 0x7fdfec07ad50 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.304+0000 7fdfe2a00700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdfd003bfe0 0x7fdfd003e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.304+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 --> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfd003b2d0 con 0x7fdfd003bfe0 2024-03-30T00:26:46.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.304+0000 7fdfe9400700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdfd003bfe0 0x7fdfd003e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.305+0000 7fdfe9400700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdfd003bfe0 0x7fdfd003e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8003a10 tx=0x7fdfd8009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.306+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 <== mgr.14118 v2:172.21.15.99:6800/1748928762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fdfd003b2d0 con 0x7fdfd003bfe0 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.313+0000 7fdfd7e00700 1 -- 172.21.15.99:0/1113093562 --> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fdfc4002600 con 0x7fdfd003bfe0 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.313+0000 7fdfe2a00700 1 -- 172.21.15.99:0/1113093562 <== mgr.14118 v2:172.21.15.99:6800/1748928762 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fdfc4002600 con 0x7fdfd003bfe0 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.314+0000 7fdfd7e00700 1 -- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdfd003bfe0 msgr2=0x7fdfd003e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.314+0000 7fdfd7e00700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdfd003bfe0 0x7fdfd003e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8003a10 tx=0x7fdfd8009290 comp rx=0 tx=0).stop 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.314+0000 7fdfd7e00700 1 -- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 msgr2=0x7fdfec19c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.314+0000 7fdfd7e00700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec19c250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc003f20 tx=0x7fdfdc011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.314+0000 7fdfd7e00700 1 -- 172.21.15.99:0/1113093562 shutdown_connections 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.314+0000 7fdfd7e00700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdfd003bfe0 0x7fdfd003e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.314+0000 7fdfd7e00700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfec07ad50 0x7fdfec19c250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:46.649 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.314+0000 7fdfd7e00700 1 --2- 172.21.15.99:0/1113093562 >> [v2:172.21.15.99:6800/1133935143,v1:172.21.15.99:6801/1133935143] conn(0x7fdfd0038700 0x7fdfd003abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:46.650 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.314+0000 7fdfd7e00700 1 -- 172.21.15.99:0/1113093562 >> 172.21.15.99:0/1113093562 conn(0x7fdfec1019c0 msgr2=0x7fdfec102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:46.650 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.314+0000 7fdfd7e00700 1 -- 172.21.15.99:0/1113093562 shutdown_connections 2024-03-30T00:26:46.650 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:46.314+0000 7fdfd7e00700 1 -- 172.21.15.99:0/1113093562 wait complete. 2024-03-30T00:26:46.650 INFO:teuthology.orchestra.run.smithi099.stdout:mgr epoch 5 is available 2024-03-30T00:26:46.650 INFO:teuthology.orchestra.run.smithi099.stdout:Setting orchestrator backend to cephadm... 2024-03-30T00:26:46.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:46 smithi099 ceph-mon[38079]: Found migration_current of "None". Setting to last migration. 2024-03-30T00:26:46.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:46 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:46.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:46 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:46.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:46 smithi099 ceph-mon[38079]: mgrmap e7: x(active, since 1.00625s) 2024-03-30T00:26:47.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.030+0000 7f3ec4e00700 1 Processor -- start 2024-03-30T00:26:47.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.030+0000 7f3ec4e00700 1 -- start start 2024-03-30T00:26:47.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.031+0000 7f3ec4e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec0106710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:47.602 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.031+0000 7f3ec4e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec0106ce0 con 0x7f3ec01062f0 2024-03-30T00:26:47.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.031+0000 7f3ebf400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec0106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:47.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.031+0000 7f3ebf400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec0106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48270/0 (socket says 172.21.15.99:48270) 2024-03-30T00:26:47.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.031+0000 7f3ebf400700 1 -- 172.21.15.99:0/1328766536 learned_addr learned my addr 172.21.15.99:0/1328766536 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:47.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.031+0000 7f3ebf400700 1 -- 172.21.15.99:0/1328766536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec01074a0 con 0x7f3ec01062f0 2024-03-30T00:26:47.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.031+0000 7f3ebf400700 1 --2- 172.21.15.99:0/1328766536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec0106710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00098a0 tx=0x7f3eb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12f96c291e9dea9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:47.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.032+0000 7f3ebe000700 1 -- 172.21.15.99:0/1328766536 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb0004160 con 0x7f3ec01062f0 2024-03-30T00:26:47.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.032+0000 7f3ebe000700 1 -- 172.21.15.99:0/1328766536 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3eb00042c0 con 0x7f3ec01062f0 2024-03-30T00:26:47.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.032+0000 7f3ebe000700 1 -- 172.21.15.99:0/1328766536 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb0011940 con 0x7f3ec01062f0 2024-03-30T00:26:47.603 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.032+0000 7f3ec4e00700 1 -- 172.21.15.99:0/1328766536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 msgr2=0x7f3ec0106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:47.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.032+0000 7f3ec4e00700 1 --2- 172.21.15.99:0/1328766536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec0106710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00098a0 tx=0x7f3eb0009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:47.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.032+0000 7f3ec4e00700 1 -- 172.21.15.99:0/1328766536 shutdown_connections 2024-03-30T00:26:47.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.032+0000 7f3ec4e00700 1 --2- 172.21.15.99:0/1328766536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec0106710 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:47.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.032+0000 7f3ec4e00700 1 -- 172.21.15.99:0/1328766536 >> 172.21.15.99:0/1328766536 conn(0x7f3ec01018d0 msgr2=0x7f3ec0103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:47.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.033+0000 7f3ec4e00700 1 -- 172.21.15.99:0/1328766536 shutdown_connections 2024-03-30T00:26:47.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.033+0000 7f3ec4e00700 1 -- 172.21.15.99:0/1328766536 wait complete. 2024-03-30T00:26:47.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.033+0000 7f3ec4e00700 1 Processor -- start 2024-03-30T00:26:47.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.034+0000 7f3ec4e00700 1 -- start start 2024-03-30T00:26:47.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.034+0000 7f3ec4e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec019c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:47.604 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.034+0000 7f3ec4e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec019c680 con 0x7f3ec01062f0 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.034+0000 7f3ebf400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec019c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.034+0000 7f3ebf400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec019c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48278/0 (socket says 172.21.15.99:48278) 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.034+0000 7f3ebf400700 1 -- 172.21.15.99:0/2769056151 learned_addr learned my addr 172.21.15.99:0/2769056151 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.034+0000 7f3ebf400700 1 -- 172.21.15.99:0/2769056151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb0009550 con 0x7f3ec01062f0 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.035+0000 7f3ebf400700 1 --2- 172.21.15.99:0/2769056151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec019c140 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0003f20 tx=0x7f3eb0011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.035+0000 7f3eb7e00700 1 -- 172.21.15.99:0/2769056151 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb00036a0 con 0x7f3ec01062f0 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.035+0000 7f3eb7e00700 1 -- 172.21.15.99:0/2769056151 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3eb0003c70 con 0x7f3ec01062f0 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.035+0000 7f3ec4e00700 1 -- 172.21.15.99:0/2769056151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec019c880 con 0x7f3ec01062f0 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.035+0000 7f3eb7e00700 1 -- 172.21.15.99:0/2769056151 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3eb001be80 con 0x7f3ec01062f0 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.035+0000 7f3ec4e00700 1 -- 172.21.15.99:0/2769056151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec019cd20 con 0x7f3ec01062f0 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.036+0000 7f3eb7e00700 1 -- 172.21.15.99:0/2769056151 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 7) v1 ==== 45427+0+0 (secure 0 0 0) 0x7f3eb001b440 con 0x7f3ec01062f0 2024-03-30T00:26:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.036+0000 7f3ec4e00700 1 -- 172.21.15.99:0/2769056151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec010a6d0 con 0x7f3ec01062f0 2024-03-30T00:26:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.036+0000 7f3eb7e00700 1 --2- 172.21.15.99:0/2769056151 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3ea4038620 0x7f3ea403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.036+0000 7f3eb7e00700 1 -- 172.21.15.99:0/2769056151 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3eb004c480 con 0x7f3ec01062f0 2024-03-30T00:26:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.037+0000 7f3ebea00700 1 --2- 172.21.15.99:0/2769056151 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3ea4038620 0x7f3ea403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.037+0000 7f3ebea00700 1 --2- 172.21.15.99:0/2769056151 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3ea4038620 0x7f3ea403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006fd0 tx=0x7f3eac006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.042+0000 7f3eb7e00700 1 -- 172.21.15.99:0/2769056151 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3eb001b900 con 0x7f3ec01062f0 2024-03-30T00:26:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.259+0000 7f3ec4e00700 1 -- 172.21.15.99:0/2769056151 --> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ec019f5b0 con 0x7f3ea4038620 2024-03-30T00:26:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.268+0000 7f3eb7e00700 1 -- 172.21.15.99:0/2769056151 <== mgr.14118 v2:172.21.15.99:6800/1748928762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3ec019f5b0 con 0x7f3ea4038620 2024-03-30T00:26:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.271+0000 7f3ec4e00700 1 -- 172.21.15.99:0/2769056151 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3ea4038620 msgr2=0x7f3ea403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.271+0000 7f3ec4e00700 1 --2- 172.21.15.99:0/2769056151 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3ea4038620 0x7f3ea403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006fd0 tx=0x7f3eac006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.271+0000 7f3ec4e00700 1 -- 172.21.15.99:0/2769056151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 msgr2=0x7f3ec019c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:47.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.271+0000 7f3ec4e00700 1 --2- 172.21.15.99:0/2769056151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec019c140 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0003f20 tx=0x7f3eb0011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:47.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.271+0000 7f3ec4e00700 1 -- 172.21.15.99:0/2769056151 shutdown_connections 2024-03-30T00:26:47.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.271+0000 7f3ec4e00700 1 --2- 172.21.15.99:0/2769056151 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3ea4038620 0x7f3ea403aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:47.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.271+0000 7f3ec4e00700 1 --2- 172.21.15.99:0/2769056151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ec01062f0 0x7f3ec019c140 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:47.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.271+0000 7f3ec4e00700 1 -- 172.21.15.99:0/2769056151 >> 172.21.15.99:0/2769056151 conn(0x7f3ec01018d0 msgr2=0x7f3ec0102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:47.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.271+0000 7f3ec4e00700 1 -- 172.21.15.99:0/2769056151 shutdown_connections 2024-03-30T00:26:47.607 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.271+0000 7f3ec4e00700 1 -- 172.21.15.99:0/2769056151 wait complete. 2024-03-30T00:26:47.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:47 smithi099 ceph-mon[38079]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-30T00:26:47.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:47 smithi099 ceph-mon[38079]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-30T00:26:47.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:47 smithi099 ceph-mon[38079]: [30/Mar/2024:00:26:46] ENGINE Bus STARTING 2024-03-30T00:26:47.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:47 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:26:47.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:47 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:47.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:47 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:26:48.518 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-30T00:26:48.518 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.970+0000 7f5f05000700 1 Processor -- start 2024-03-30T00:26:48.518 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.970+0000 7f5f05000700 1 -- start start 2024-03-30T00:26:48.518 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.970+0000 7f5f05000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f000791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:48.518 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.970+0000 7f5f05000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000797a0 con 0x7f5f0007ad70 2024-03-30T00:26:48.518 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.971+0000 7f5efde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f000791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.971+0000 7f5efde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f000791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48286/0 (socket says 172.21.15.99:48286) 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.971+0000 7f5efde00700 1 -- 172.21.15.99:0/653754378 learned_addr learned my addr 172.21.15.99:0/653754378 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.971+0000 7f5efde00700 1 -- 172.21.15.99:0/653754378 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f000798e0 con 0x7f5f0007ad70 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.971+0000 7f5efde00700 1 --2- 172.21.15.99:0/653754378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f000791d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00098a0 tx=0x7f5ef0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2265ac417889f604 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.971+0000 7f5efca00700 1 -- 172.21.15.99:0/653754378 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef0004160 con 0x7f5f0007ad70 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.972+0000 7f5efca00700 1 -- 172.21.15.99:0/653754378 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5ef00042c0 con 0x7f5f0007ad70 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.972+0000 7f5efca00700 1 -- 172.21.15.99:0/653754378 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef0011940 con 0x7f5f0007ad70 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.972+0000 7f5f05000700 1 -- 172.21.15.99:0/653754378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 msgr2=0x7f5f000791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.972+0000 7f5f05000700 1 --2- 172.21.15.99:0/653754378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f000791d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00098a0 tx=0x7f5ef0009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.972+0000 7f5f05000700 1 -- 172.21.15.99:0/653754378 shutdown_connections 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.972+0000 7f5f05000700 1 --2- 172.21.15.99:0/653754378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f000791d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.972+0000 7f5f05000700 1 -- 172.21.15.99:0/653754378 >> 172.21.15.99:0/653754378 conn(0x7f5f00101990 msgr2=0x7f5f00103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.972+0000 7f5f05000700 1 -- 172.21.15.99:0/653754378 shutdown_connections 2024-03-30T00:26:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.972+0000 7f5f05000700 1 -- 172.21.15.99:0/653754378 wait complete. 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.974+0000 7f5f05000700 1 Processor -- start 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.974+0000 7f5f05000700 1 -- start start 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.974+0000 7f5f05000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f0019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.974+0000 7f5f05000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0019c840 con 0x7f5f0007ad70 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.974+0000 7f5efde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f0019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.974+0000 7f5efde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f0019c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48294/0 (socket says 172.21.15.99:48294) 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.974+0000 7f5efde00700 1 -- 172.21.15.99:0/3537070785 learned_addr learned my addr 172.21.15.99:0/3537070785 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.975+0000 7f5efde00700 1 -- 172.21.15.99:0/3537070785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef0009550 con 0x7f5f0007ad70 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.975+0000 7f5efde00700 1 --2- 172.21.15.99:0/3537070785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f0019c300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0003f20 tx=0x7f5ef0011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.975+0000 7f5ef6a00700 1 -- 172.21.15.99:0/3537070785 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef00036a0 con 0x7f5f0007ad70 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.975+0000 7f5ef6a00700 1 -- 172.21.15.99:0/3537070785 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5ef001b440 con 0x7f5f0007ad70 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.975+0000 7f5f05000700 1 -- 172.21.15.99:0/3537070785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0019ca40 con 0x7f5f0007ad70 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.975+0000 7f5ef6a00700 1 -- 172.21.15.99:0/3537070785 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef0023400 con 0x7f5f0007ad70 2024-03-30T00:26:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.975+0000 7f5f05000700 1 -- 172.21.15.99:0/3537070785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0019cee0 con 0x7f5f0007ad70 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.977+0000 7f5f05000700 1 -- 172.21.15.99:0/3537070785 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0010c7a0 con 0x7f5f0007ad70 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.977+0000 7f5ef6a00700 1 -- 172.21.15.99:0/3537070785 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 7) v1 ==== 45427+0+0 (secure 0 0 0) 0x7f5ef001b5b0 con 0x7f5f0007ad70 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.977+0000 7f5ef6a00700 1 --2- 172.21.15.99:0/3537070785 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5ee40385d0 0x7f5ee403aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.977+0000 7f5ef6a00700 1 -- 172.21.15.99:0/3537070785 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5ef004c4d0 con 0x7f5f0007ad70 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.978+0000 7f5efd400700 1 --2- 172.21.15.99:0/3537070785 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5ee40385d0 0x7f5ee403aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.978+0000 7f5efd400700 1 --2- 172.21.15.99:0/3537070785 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5ee40385d0 0x7f5ee403aa90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5eec006fd0 tx=0x7f5eec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:47.982+0000 7f5ef6a00700 1 -- 172.21.15.99:0/3537070785 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5ef001b920 con 0x7f5f0007ad70 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.181+0000 7f5f05000700 1 -- 172.21.15.99:0/3537070785 --> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f00102c40 con 0x7f5ee40385d0 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.182+0000 7f5ef6a00700 1 -- 172.21.15.99:0/3537070785 <== mgr.14118 v2:172.21.15.99:6800/1748928762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f5f00102c40 con 0x7f5ee40385d0 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.184+0000 7f5f05000700 1 -- 172.21.15.99:0/3537070785 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5ee40385d0 msgr2=0x7f5ee403aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.184+0000 7f5f05000700 1 --2- 172.21.15.99:0/3537070785 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5ee40385d0 0x7f5ee403aa90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5eec006fd0 tx=0x7f5eec006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.184+0000 7f5f05000700 1 -- 172.21.15.99:0/3537070785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 msgr2=0x7f5f0019c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:48.521 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.184+0000 7f5f05000700 1 --2- 172.21.15.99:0/3537070785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f0019c300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0003f20 tx=0x7f5ef0011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:48.522 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.184+0000 7f5f05000700 1 -- 172.21.15.99:0/3537070785 shutdown_connections 2024-03-30T00:26:48.522 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.184+0000 7f5f05000700 1 --2- 172.21.15.99:0/3537070785 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5ee40385d0 0x7f5ee403aa90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:48.522 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.184+0000 7f5f05000700 1 --2- 172.21.15.99:0/3537070785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f0007ad70 0x7f5f0019c300 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:48.522 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.184+0000 7f5f05000700 1 -- 172.21.15.99:0/3537070785 >> 172.21.15.99:0/3537070785 conn(0x7f5f00101990 msgr2=0x7f5f00102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:48.522 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.184+0000 7f5f05000700 1 -- 172.21.15.99:0/3537070785 shutdown_connections 2024-03-30T00:26:48.522 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.184+0000 7f5f05000700 1 -- 172.21.15.99:0/3537070785 wait complete. 2024-03-30T00:26:48.522 INFO:teuthology.orchestra.run.smithi099.stdout:Generating ssh key... 2024-03-30T00:26:49.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:48 smithi099 ceph-mon[38079]: [30/Mar/2024:00:26:46] ENGINE Serving on http://172.21.15.99:8765 2024-03-30T00:26:49.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:48 smithi099 ceph-mon[38079]: [30/Mar/2024:00:26:47] ENGINE Serving on https://172.21.15.99:7150 2024-03-30T00:26:49.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:48 smithi099 ceph-mon[38079]: [30/Mar/2024:00:26:47] ENGINE Bus STARTED 2024-03-30T00:26:49.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:48 smithi099 ceph-mon[38079]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:26:49.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:48 smithi099 ceph-mon[38079]: mgrmap e8: x(active, since 2s) 2024-03-30T00:26:49.709 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.894+0000 7efc27e00700 1 Processor -- start 2024-03-30T00:26:49.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.894+0000 7efc27e00700 1 -- start start 2024-03-30T00:26:49.710 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: Generating public/private rsa key pair. 2024-03-30T00:26:49.710 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: Your identification has been saved in /tmp/tmpzzvd0iph/key. 2024-03-30T00:26:49.710 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: Your public key has been saved in /tmp/tmpzzvd0iph/key.pub. 2024-03-30T00:26:49.710 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: The key fingerprint is: 2024-03-30T00:26:49.710 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: SHA256:O+xRUa/NZ1HQz+6fkcQL8qOcOshcfNrj8RYso+P7S08 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:26:49.710 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: The key's randomart image is: 2024-03-30T00:26:49.710 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: +---[RSA 3072]----+ 2024-03-30T00:26:49.711 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: | . .o.| 2024-03-30T00:26:49.711 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: | . . o| 2024-03-30T00:26:49.711 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: | . .o.| 2024-03-30T00:26:49.711 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: | . +. +| 2024-03-30T00:26:49.711 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: | S ..o.o+o| 2024-03-30T00:26:49.711 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: | . = +oooo+| 2024-03-30T00:26:49.711 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: | o B =ooE.= | 2024-03-30T00:26:49.711 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: | = BooB.. +| 2024-03-30T00:26:49.711 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: | o+BB+o .o| 2024-03-30T00:26:49.711 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: +----[SHA256]-----+ 2024-03-30T00:26:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.894+0000 7efc27e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc28079190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.894+0000 7efc27e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc28079760 con 0x7efc2807ad30 2024-03-30T00:26:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.894+0000 7efc26a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc28079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:49.716 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.895+0000 7efc26a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc28079190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48310/0 (socket says 172.21.15.99:48310) 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.895+0000 7efc26a00700 1 -- 172.21.15.99:0/2880775403 learned_addr learned my addr 172.21.15.99:0/2880775403 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.895+0000 7efc26a00700 1 -- 172.21.15.99:0/2880775403 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc280798a0 con 0x7efc2807ad30 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.895+0000 7efc26a00700 1 --2- 172.21.15.99:0/2880775403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc28079190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efc180098a0 tx=0x7efc18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4692ce0985dfba3c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.895+0000 7efc25600700 1 -- 172.21.15.99:0/2880775403 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc18004160 con 0x7efc2807ad30 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.895+0000 7efc25600700 1 -- 172.21.15.99:0/2880775403 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7efc180042c0 con 0x7efc2807ad30 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.895+0000 7efc25600700 1 -- 172.21.15.99:0/2880775403 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc18011940 con 0x7efc2807ad30 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.896+0000 7efc27e00700 1 -- 172.21.15.99:0/2880775403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 msgr2=0x7efc28079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.896+0000 7efc27e00700 1 --2- 172.21.15.99:0/2880775403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc28079190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efc180098a0 tx=0x7efc18009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.896+0000 7efc27e00700 1 -- 172.21.15.99:0/2880775403 shutdown_connections 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.896+0000 7efc27e00700 1 --2- 172.21.15.99:0/2880775403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc28079190 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.896+0000 7efc27e00700 1 -- 172.21.15.99:0/2880775403 >> 172.21.15.99:0/2880775403 conn(0x7efc281017c0 msgr2=0x7efc28103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.896+0000 7efc27e00700 1 -- 172.21.15.99:0/2880775403 shutdown_connections 2024-03-30T00:26:49.717 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.896+0000 7efc27e00700 1 -- 172.21.15.99:0/2880775403 wait complete. 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.897+0000 7efc27e00700 1 Processor -- start 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.897+0000 7efc27e00700 1 -- start start 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.897+0000 7efc27e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc2819c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.897+0000 7efc27e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2819c660 con 0x7efc2807ad30 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.898+0000 7efc26a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc2819c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.898+0000 7efc26a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc2819c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48316/0 (socket says 172.21.15.99:48316) 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.898+0000 7efc26a00700 1 -- 172.21.15.99:0/712680687 learned_addr learned my addr 172.21.15.99:0/712680687 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.898+0000 7efc26a00700 1 -- 172.21.15.99:0/712680687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc18009550 con 0x7efc2807ad30 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.898+0000 7efc26a00700 1 --2- 172.21.15.99:0/712680687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc2819c120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efc18003f20 tx=0x7efc18011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.898+0000 7efc1f400700 1 -- 172.21.15.99:0/712680687 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc180037d0 con 0x7efc2807ad30 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.898+0000 7efc1f400700 1 -- 172.21.15.99:0/712680687 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7efc1801b440 con 0x7efc2807ad30 2024-03-30T00:26:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.898+0000 7efc1f400700 1 -- 172.21.15.99:0/712680687 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc18023590 con 0x7efc2807ad30 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.898+0000 7efc27e00700 1 -- 172.21.15.99:0/712680687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2819c860 con 0x7efc2807ad30 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.899+0000 7efc27e00700 1 -- 172.21.15.99:0/712680687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2819cd00 con 0x7efc2807ad30 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.900+0000 7efc1f400700 1 -- 172.21.15.99:0/712680687 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7efc1801b5b0 con 0x7efc2807ad30 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.900+0000 7efc27e00700 1 -- 172.21.15.99:0/712680687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2810c5b0 con 0x7efc2807ad30 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.900+0000 7efc1f400700 1 --2- 172.21.15.99:0/712680687 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7efc0c038750 0x7efc0c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.900+0000 7efc1f400700 1 -- 172.21.15.99:0/712680687 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efc1804c6a0 con 0x7efc2807ad30 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.900+0000 7efc26000700 1 --2- 172.21.15.99:0/712680687 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7efc0c038750 0x7efc0c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.901+0000 7efc26000700 1 --2- 172.21.15.99:0/712680687 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7efc0c038750 0x7efc0c03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efc14006fd0 tx=0x7efc14006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:48.905+0000 7efc1f400700 1 -- 172.21.15.99:0/712680687 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efc1801b920 con 0x7efc2807ad30 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.107+0000 7efc27e00700 1 -- 172.21.15.99:0/712680687 --> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7efc28102b10 con 0x7efc0c038750 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.382+0000 7efc1f400700 1 -- 172.21.15.99:0/712680687 <== mgr.14118 v2:172.21.15.99:6800/1748928762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efc28102b10 con 0x7efc0c038750 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.385+0000 7efc27e00700 1 -- 172.21.15.99:0/712680687 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7efc0c038750 msgr2=0x7efc0c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.385+0000 7efc27e00700 1 --2- 172.21.15.99:0/712680687 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7efc0c038750 0x7efc0c03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efc14006fd0 tx=0x7efc14006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:49.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.385+0000 7efc27e00700 1 -- 172.21.15.99:0/712680687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 msgr2=0x7efc2819c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.385+0000 7efc27e00700 1 --2- 172.21.15.99:0/712680687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc2819c120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efc18003f20 tx=0x7efc18011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.385+0000 7efc27e00700 1 -- 172.21.15.99:0/712680687 shutdown_connections 2024-03-30T00:26:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.385+0000 7efc27e00700 1 --2- 172.21.15.99:0/712680687 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7efc0c038750 0x7efc0c03ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.385+0000 7efc27e00700 1 --2- 172.21.15.99:0/712680687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc2807ad30 0x7efc2819c120 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.385+0000 7efc27e00700 1 -- 172.21.15.99:0/712680687 >> 172.21.15.99:0/712680687 conn(0x7efc281017c0 msgr2=0x7efc28102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.385+0000 7efc27e00700 1 -- 172.21.15.99:0/712680687 shutdown_connections 2024-03-30T00:26:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:49.385+0000 7efc27e00700 1 -- 172.21.15.99:0/712680687 wait complete. 2024-03-30T00:26:49.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-mon[38079]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:26:49.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:49.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:49 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:50.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+Cv5uDS5VlZ9DBDzW/S6j9tw8vu3vM0xFZIFF/UBQccJgQnvp7vcttKmuUnu0uWKeTmXIAXqN+oGrCXw004pkj7M5+/0pujdALwbihqEc+Lx9takTObQbmb8iD4UYs5k8y6+eNqsZixCKA3EQpd++/i3ecO14mvsqrQ15p5yEmo2/gYxIDpsSux+CTeb3MTJC0gLaHQWMRvRRGaxnXqIbJAFAX1i9R0lR6z4+/pudmM7vTAP6Tvz6kz5tTplGjYUR5Bi9tW4aml+7/wBdDl5Ri9fz4by9Gw5ZNylZsGRrUEUPad1gIXo8v8RTUgk5XRWJoY55NqjACSlPRZnGnDVyESYGZdh2PniworL9Gg2cnX1j3mNlFaSCpb4N8Fv1AyofpZ7bB3W48ABu2/UGNqCBJP/Hr8LhFMQPW23k4NdtEVjp97BrAAGJoYcz/pI7BZXwqTEImoQM9sUI6daQgvrTUL0Yz15z9lcGFB5uBo+tTOut0OB9saTDM6IGDy09qUc= ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:26:50.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.076+0000 7f3baae00700 1 Processor -- start 2024-03-30T00:26:50.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.077+0000 7f3baae00700 1 -- start start 2024-03-30T00:26:50.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.077+0000 7f3baae00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba40791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:50.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.077+0000 7f3baae00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba4079780 con 0x7f3ba407ad50 2024-03-30T00:26:50.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.077+0000 7f3ba3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba40791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:50.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.077+0000 7f3ba3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba40791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48324/0 (socket says 172.21.15.99:48324) 2024-03-30T00:26:50.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.077+0000 7f3ba3e00700 1 -- 172.21.15.99:0/2637930778 learned_addr learned my addr 172.21.15.99:0/2637930778 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:50.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.078+0000 7f3ba3e00700 1 -- 172.21.15.99:0/2637930778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba40798c0 con 0x7f3ba407ad50 2024-03-30T00:26:50.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.078+0000 7f3ba3e00700 1 --2- 172.21.15.99:0/2637930778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba40791b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3b940098a0 tx=0x7f3b94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1091616daee184cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:50.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.078+0000 7f3ba3400700 1 -- 172.21.15.99:0/2637930778 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b94004160 con 0x7f3ba407ad50 2024-03-30T00:26:50.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.078+0000 7f3ba3400700 1 -- 172.21.15.99:0/2637930778 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3b940042c0 con 0x7f3ba407ad50 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.078+0000 7f3ba3400700 1 -- 172.21.15.99:0/2637930778 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b94011940 con 0x7f3ba407ad50 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.079+0000 7f3baae00700 1 -- 172.21.15.99:0/2637930778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 msgr2=0x7f3ba40791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.079+0000 7f3baae00700 1 --2- 172.21.15.99:0/2637930778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba40791b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3b940098a0 tx=0x7f3b94009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.079+0000 7f3baae00700 1 -- 172.21.15.99:0/2637930778 shutdown_connections 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.079+0000 7f3baae00700 1 --2- 172.21.15.99:0/2637930778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba40791b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.079+0000 7f3baae00700 1 -- 172.21.15.99:0/2637930778 >> 172.21.15.99:0/2637930778 conn(0x7f3ba4101990 msgr2=0x7f3ba4103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.079+0000 7f3baae00700 1 -- 172.21.15.99:0/2637930778 shutdown_connections 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.079+0000 7f3baae00700 1 -- 172.21.15.99:0/2637930778 wait complete. 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.080+0000 7f3baae00700 1 Processor -- start 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.080+0000 7f3baae00700 1 -- start start 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.080+0000 7f3baae00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.080+0000 7f3baae00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba419c7e0 con 0x7f3ba407ad50 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.080+0000 7f3ba3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.081+0000 7f3ba3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba419c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48332/0 (socket says 172.21.15.99:48332) 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.081+0000 7f3ba3e00700 1 -- 172.21.15.99:0/1895866110 learned_addr learned my addr 172.21.15.99:0/1895866110 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.081+0000 7f3ba3e00700 1 -- 172.21.15.99:0/1895866110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b94009550 con 0x7f3ba407ad50 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.081+0000 7f3ba3e00700 1 --2- 172.21.15.99:0/1895866110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba419c2a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b94003f20 tx=0x7f3b94011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.081+0000 7f3ba1600700 1 -- 172.21.15.99:0/1895866110 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b940037d0 con 0x7f3ba407ad50 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.081+0000 7f3ba1600700 1 -- 172.21.15.99:0/1895866110 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3b9401b440 con 0x7f3ba407ad50 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.081+0000 7f3baae00700 1 -- 172.21.15.99:0/1895866110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba419c9e0 con 0x7f3ba407ad50 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.081+0000 7f3ba1600700 1 -- 172.21.15.99:0/1895866110 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b94023590 con 0x7f3ba407ad50 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.081+0000 7f3baae00700 1 -- 172.21.15.99:0/1895866110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba419ce80 con 0x7f3ba407ad50 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.082+0000 7f3ba1600700 1 -- 172.21.15.99:0/1895866110 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f3b94003930 con 0x7f3ba407ad50 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.082+0000 7f3baae00700 1 -- 172.21.15.99:0/1895866110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba410c7c0 con 0x7f3ba407ad50 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.083+0000 7f3ba1600700 1 --2- 172.21.15.99:0/1895866110 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3b88038750 0x7f3b8803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.083+0000 7f3ba1600700 1 -- 172.21.15.99:0/1895866110 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3b9404c700 con 0x7f3ba407ad50 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.083+0000 7f3b9be00700 1 --2- 172.21.15.99:0/1895866110 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3b88038750 0x7f3b8803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:50.660 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.084+0000 7f3b9be00700 1 --2- 172.21.15.99:0/1895866110 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3b88038750 0x7f3b8803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c006fd0 tx=0x7f3b8c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.089+0000 7f3ba1600700 1 -- 172.21.15.99:0/1895866110 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3b9400bbd0 con 0x7f3ba407ad50 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.288+0000 7f3baae00700 1 -- 172.21.15.99:0/1895866110 --> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ba4102c70 con 0x7f3b88038750 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.289+0000 7f3ba1600700 1 -- 172.21.15.99:0/1895866110 <== mgr.14118 v2:172.21.15.99:6800/1748928762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f3ba4102c70 con 0x7f3b88038750 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.291+0000 7f3baae00700 1 -- 172.21.15.99:0/1895866110 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3b88038750 msgr2=0x7f3b8803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.291+0000 7f3baae00700 1 --2- 172.21.15.99:0/1895866110 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3b88038750 0x7f3b8803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c006fd0 tx=0x7f3b8c006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.291+0000 7f3baae00700 1 -- 172.21.15.99:0/1895866110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 msgr2=0x7f3ba419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.291+0000 7f3baae00700 1 --2- 172.21.15.99:0/1895866110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba419c2a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b94003f20 tx=0x7f3b94011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.291+0000 7f3baae00700 1 -- 172.21.15.99:0/1895866110 shutdown_connections 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.291+0000 7f3baae00700 1 --2- 172.21.15.99:0/1895866110 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f3b88038750 0x7f3b8803ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.291+0000 7f3baae00700 1 --2- 172.21.15.99:0/1895866110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ba407ad50 0x7f3ba419c2a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.291+0000 7f3baae00700 1 -- 172.21.15.99:0/1895866110 >> 172.21.15.99:0/1895866110 conn(0x7f3ba4101990 msgr2=0x7f3ba4102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.291+0000 7f3baae00700 1 -- 172.21.15.99:0/1895866110 shutdown_connections 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:50.291+0000 7f3baae00700 1 -- 172.21.15.99:0/1895866110 wait complete. 2024-03-30T00:26:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-30T00:26:50.662 INFO:teuthology.orchestra.run.smithi099.stdout:Adding key to root@localhost authorized_keys... 2024-03-30T00:26:50.662 INFO:teuthology.orchestra.run.smithi099.stdout:Adding host smithi099... 2024-03-30T00:26:50.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:50 smithi099 ceph-mon[38079]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:26:50.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:50 smithi099 ceph-mon[38079]: Generating ssh key... 2024-03-30T00:26:51.918 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:51 smithi099 ceph-mon[38079]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:26:53.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:52 smithi099 ceph-mon[38079]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "addr": "172.21.15.99", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:26:53.808 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:53 smithi099 ceph-mon[38079]: Deploying cephadm binary to smithi099 2024-03-30T00:26:54.585 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Added host 'smithi099' with addr '172.21.15.99' 2024-03-30T00:26:54.585 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.046+0000 7f8cd2000700 1 Processor -- start 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.046+0000 7f8cd2000700 1 -- start start 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.047+0000 7f8cd2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc0791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.047+0000 7f8cd2000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc079780 con 0x7f8ccc07ad50 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.047+0000 7f8ccb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc0791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.047+0000 7f8ccb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc0791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48342/0 (socket says 172.21.15.99:48342) 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.047+0000 7f8ccb400700 1 -- 172.21.15.99:0/2326447398 learned_addr learned my addr 172.21.15.99:0/2326447398 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.047+0000 7f8ccb400700 1 -- 172.21.15.99:0/2326447398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc0798c0 con 0x7f8ccc07ad50 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.048+0000 7f8ccb400700 1 --2- 172.21.15.99:0/2326447398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc0791b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0098a0 tx=0x7f8cbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c69e57c275a46 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.048+0000 7f8ccaa00700 1 -- 172.21.15.99:0/2326447398 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc004160 con 0x7f8ccc07ad50 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.048+0000 7f8ccaa00700 1 -- 172.21.15.99:0/2326447398 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8cbc0042c0 con 0x7f8ccc07ad50 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.048+0000 7f8ccaa00700 1 -- 172.21.15.99:0/2326447398 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc011940 con 0x7f8ccc07ad50 2024-03-30T00:26:54.586 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.048+0000 7f8cd2000700 1 -- 172.21.15.99:0/2326447398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 msgr2=0x7f8ccc0791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.048+0000 7f8cd2000700 1 --2- 172.21.15.99:0/2326447398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc0791b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0098a0 tx=0x7f8cbc009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.049+0000 7f8cd2000700 1 -- 172.21.15.99:0/2326447398 shutdown_connections 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.049+0000 7f8cd2000700 1 --2- 172.21.15.99:0/2326447398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc0791b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.049+0000 7f8cd2000700 1 -- 172.21.15.99:0/2326447398 >> 172.21.15.99:0/2326447398 conn(0x7f8ccc1019a0 msgr2=0x7f8ccc103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.049+0000 7f8cd2000700 1 -- 172.21.15.99:0/2326447398 shutdown_connections 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.049+0000 7f8cd2000700 1 -- 172.21.15.99:0/2326447398 wait complete. 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.050+0000 7f8cd2000700 1 Processor -- start 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.050+0000 7f8cd2000700 1 -- start start 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.050+0000 7f8cd2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.050+0000 7f8cd2000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc19c7d0 con 0x7f8ccc07ad50 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.050+0000 7f8ccb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.050+0000 7f8ccb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc19c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48354/0 (socket says 172.21.15.99:48354) 2024-03-30T00:26:54.587 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.050+0000 7f8ccb400700 1 -- 172.21.15.99:0/2473159990 learned_addr learned my addr 172.21.15.99:0/2473159990 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.051+0000 7f8ccb400700 1 -- 172.21.15.99:0/2473159990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc009550 con 0x7f8ccc07ad50 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.051+0000 7f8ccb400700 1 --2- 172.21.15.99:0/2473159990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc19c290 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc003f20 tx=0x7f8cbc011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.051+0000 7f8cc8c00700 1 -- 172.21.15.99:0/2473159990 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc0037d0 con 0x7f8ccc07ad50 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.051+0000 7f8cc8c00700 1 -- 172.21.15.99:0/2473159990 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8cbc01b440 con 0x7f8ccc07ad50 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.051+0000 7f8cd2000700 1 -- 172.21.15.99:0/2473159990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc19c9d0 con 0x7f8ccc07ad50 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.051+0000 7f8cc8c00700 1 -- 172.21.15.99:0/2473159990 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc023590 con 0x7f8ccc07ad50 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.051+0000 7f8cd2000700 1 -- 172.21.15.99:0/2473159990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc19ce70 con 0x7f8ccc07ad50 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.052+0000 7f8cc8c00700 1 -- 172.21.15.99:0/2473159990 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f8cbc003930 con 0x7f8ccc07ad50 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.052+0000 7f8cd2000700 1 -- 172.21.15.99:0/2473159990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc10c7b0 con 0x7f8ccc07ad50 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.053+0000 7f8cc8c00700 1 --2- 172.21.15.99:0/2473159990 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f8cb0038750 0x7f8cb003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.053+0000 7f8cc8c00700 1 -- 172.21.15.99:0/2473159990 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8cbc04c720 con 0x7f8ccc07ad50 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.053+0000 7f8cc3e00700 1 --2- 172.21.15.99:0/2473159990 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f8cb0038750 0x7f8cb003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:54.588 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.054+0000 7f8cc3e00700 1 --2- 172.21.15.99:0/2473159990 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f8cb0038750 0x7f8cb003ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4006fd0 tx=0x7f8cb4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.058+0000 7f8cc8c00700 1 -- 172.21.15.99:0/2473159990 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8cbc00bbd0 con 0x7f8ccc07ad50 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:51.261+0000 7f8cd2000700 1 -- 172.21.15.99:0/2473159990 --> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi099", "addr": "172.21.15.99", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ccc102cb0 con 0x7f8cb0038750 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:54.227+0000 7f8cc8c00700 1 -- 172.21.15.99:0/2473159990 <== mgr.14118 v2:172.21.15.99:6800/1748928762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f8ccc102cb0 con 0x7f8cb0038750 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:54.230+0000 7f8cd2000700 1 -- 172.21.15.99:0/2473159990 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f8cb0038750 msgr2=0x7f8cb003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:54.230+0000 7f8cd2000700 1 --2- 172.21.15.99:0/2473159990 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f8cb0038750 0x7f8cb003ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4006fd0 tx=0x7f8cb4006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:54.230+0000 7f8cd2000700 1 -- 172.21.15.99:0/2473159990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 msgr2=0x7f8ccc19c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:54.230+0000 7f8cd2000700 1 --2- 172.21.15.99:0/2473159990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc19c290 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc003f20 tx=0x7f8cbc011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:54.230+0000 7f8cd2000700 1 -- 172.21.15.99:0/2473159990 shutdown_connections 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:54.230+0000 7f8cd2000700 1 --2- 172.21.15.99:0/2473159990 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f8cb0038750 0x7f8cb003ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:54.230+0000 7f8cd2000700 1 --2- 172.21.15.99:0/2473159990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ccc07ad50 0x7f8ccc19c290 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:54.230+0000 7f8cd2000700 1 -- 172.21.15.99:0/2473159990 >> 172.21.15.99:0/2473159990 conn(0x7f8ccc1019a0 msgr2=0x7f8ccc1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:54.231+0000 7f8cd2000700 1 -- 172.21.15.99:0/2473159990 shutdown_connections 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:54.231+0000 7f8cd2000700 1 -- 172.21.15.99:0/2473159990 wait complete. 2024-03-30T00:26:54.589 INFO:teuthology.orchestra.run.smithi099.stdout:Deploying unmanaged mon service... 2024-03-30T00:26:55.418 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:55 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:55.418 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:55 smithi099 ceph-mon[38079]: Added host smithi099 2024-03-30T00:26:55.418 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:55 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:26:55.690 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-30T00:26:55.690 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.021+0000 7fdcd8800700 1 Processor -- start 2024-03-30T00:26:55.690 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.021+0000 7fdcd8800700 1 -- start start 2024-03-30T00:26:55.690 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.021+0000 7fdcd8800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd00791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:55.690 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.022+0000 7fdcd8800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd0079780 con 0x7fdcd007ad50 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.022+0000 7fdcd5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd00791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.022+0000 7fdcd5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd00791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48710/0 (socket says 172.21.15.99:48710) 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.022+0000 7fdcd5e00700 1 -- 172.21.15.99:0/660903496 learned_addr learned my addr 172.21.15.99:0/660903496 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.022+0000 7fdcd5e00700 1 -- 172.21.15.99:0/660903496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd00798c0 con 0x7fdcd007ad50 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.023+0000 7fdcd5e00700 1 --2- 172.21.15.99:0/660903496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd00791b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40098a0 tx=0x7fdcc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389babef85fbff18 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.023+0000 7fdcd4a00700 1 -- 172.21.15.99:0/660903496 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcc4004160 con 0x7fdcd007ad50 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.023+0000 7fdcd4a00700 1 -- 172.21.15.99:0/660903496 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fdcc40042c0 con 0x7fdcd007ad50 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.023+0000 7fdcd4a00700 1 -- 172.21.15.99:0/660903496 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcc4011940 con 0x7fdcd007ad50 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.023+0000 7fdcd8800700 1 -- 172.21.15.99:0/660903496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 msgr2=0x7fdcd00791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.023+0000 7fdcd8800700 1 --2- 172.21.15.99:0/660903496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd00791b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40098a0 tx=0x7fdcc4009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.024+0000 7fdcd8800700 1 -- 172.21.15.99:0/660903496 shutdown_connections 2024-03-30T00:26:55.691 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.024+0000 7fdcd8800700 1 --2- 172.21.15.99:0/660903496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd00791b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.024+0000 7fdcd8800700 1 -- 172.21.15.99:0/660903496 >> 172.21.15.99:0/660903496 conn(0x7fdcd01019a0 msgr2=0x7fdcd0103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.024+0000 7fdcd8800700 1 -- 172.21.15.99:0/660903496 shutdown_connections 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.024+0000 7fdcd8800700 1 -- 172.21.15.99:0/660903496 wait complete. 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.025+0000 7fdcd8800700 1 Processor -- start 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.025+0000 7fdcd8800700 1 -- start start 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.026+0000 7fdcd8800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.026+0000 7fdcd8800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019c7d0 con 0x7fdcd007ad50 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.026+0000 7fdcd5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.026+0000 7fdcd5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48718/0 (socket says 172.21.15.99:48718) 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.026+0000 7fdcd5e00700 1 -- 172.21.15.99:0/3223956909 learned_addr learned my addr 172.21.15.99:0/3223956909 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.026+0000 7fdcd5e00700 1 -- 172.21.15.99:0/3223956909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4009550 con 0x7fdcd007ad50 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.027+0000 7fdcd5e00700 1 --2- 172.21.15.99:0/3223956909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd019c290 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4003f20 tx=0x7fdcc4011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.027+0000 7fdccaa00700 1 -- 172.21.15.99:0/3223956909 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcc40037d0 con 0x7fdcd007ad50 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.027+0000 7fdccaa00700 1 -- 172.21.15.99:0/3223956909 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fdcc401b440 con 0x7fdcd007ad50 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.027+0000 7fdcd8800700 1 -- 172.21.15.99:0/3223956909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd019c9d0 con 0x7fdcd007ad50 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.027+0000 7fdccaa00700 1 -- 172.21.15.99:0/3223956909 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcc4023590 con 0x7fdcd007ad50 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.027+0000 7fdcd8800700 1 -- 172.21.15.99:0/3223956909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd019ce70 con 0x7fdcd007ad50 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.028+0000 7fdcd8800700 1 -- 172.21.15.99:0/3223956909 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd010c7b0 con 0x7fdcd007ad50 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.029+0000 7fdccaa00700 1 -- 172.21.15.99:0/3223956909 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fdcc401b5b0 con 0x7fdcd007ad50 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.030+0000 7fdccaa00700 1 --2- 172.21.15.99:0/3223956909 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdcb8038750 0x7fdcb803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.030+0000 7fdccaa00700 1 -- 172.21.15.99:0/3223956909 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdcc404ccf0 con 0x7fdcd007ad50 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.030+0000 7fdcd5400700 1 --2- 172.21.15.99:0/3223956909 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdcb8038750 0x7fdcb803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.030+0000 7fdcd5400700 1 --2- 172.21.15.99:0/3223956909 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdcb8038750 0x7fdcb803ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0006fd0 tx=0x7fdcc0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.035+0000 7fdccaa00700 1 -- 172.21.15.99:0/3223956909 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdcc401b920 con 0x7fdcd007ad50 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.263+0000 7fdcd8800700 1 -- 172.21.15.99:0/3223956909 --> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fdcd0102cb0 con 0x7fdcb8038750 2024-03-30T00:26:55.693 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.269+0000 7fdccaa00700 1 -- 172.21.15.99:0/3223956909 <== mgr.14118 v2:172.21.15.99:6800/1748928762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdcd0102cb0 con 0x7fdcb8038750 2024-03-30T00:26:55.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.272+0000 7fdcd8800700 1 -- 172.21.15.99:0/3223956909 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdcb8038750 msgr2=0x7fdcb803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:55.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.272+0000 7fdcd8800700 1 --2- 172.21.15.99:0/3223956909 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdcb8038750 0x7fdcb803ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0006fd0 tx=0x7fdcc0006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:55.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.272+0000 7fdcd8800700 1 -- 172.21.15.99:0/3223956909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 msgr2=0x7fdcd019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:55.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.272+0000 7fdcd8800700 1 --2- 172.21.15.99:0/3223956909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd019c290 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4003f20 tx=0x7fdcc4011f00 comp rx=0 tx=0).stop 2024-03-30T00:26:55.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.272+0000 7fdcd8800700 1 -- 172.21.15.99:0/3223956909 shutdown_connections 2024-03-30T00:26:55.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.272+0000 7fdcd8800700 1 --2- 172.21.15.99:0/3223956909 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fdcb8038750 0x7fdcb803ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:55.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.272+0000 7fdcd8800700 1 --2- 172.21.15.99:0/3223956909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcd007ad50 0x7fdcd019c290 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:55.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.272+0000 7fdcd8800700 1 -- 172.21.15.99:0/3223956909 >> 172.21.15.99:0/3223956909 conn(0x7fdcd01019a0 msgr2=0x7fdcd01025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:55.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.272+0000 7fdcd8800700 1 -- 172.21.15.99:0/3223956909 shutdown_connections 2024-03-30T00:26:55.694 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:55.272+0000 7fdcd8800700 1 -- 172.21.15.99:0/3223956909 wait complete. 2024-03-30T00:26:55.694 INFO:teuthology.orchestra.run.smithi099.stdout:Deploying unmanaged mgr service... 2024-03-30T00:26:56.529 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:56 smithi099 ceph-mon[38079]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:26:56.529 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:56 smithi099 ceph-mon[38079]: Saving service mon spec with placement count:5 2024-03-30T00:26:56.529 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:56 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:56.529 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:56 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:56.529 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:56 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-30T00:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.060+0000 7f5c75a00700 1 Processor -- start 2024-03-30T00:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.060+0000 7f5c75a00700 1 -- start start 2024-03-30T00:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.060+0000 7f5c75a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c700791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.060+0000 7f5c75a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c70079780 con 0x7f5c7007ad50 2024-03-30T00:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.060+0000 7f5c6ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c700791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.061+0000 7f5c6ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c700791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48722/0 (socket says 172.21.15.99:48722) 2024-03-30T00:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.061+0000 7f5c6ea00700 1 -- 172.21.15.99:0/1036503611 learned_addr learned my addr 172.21.15.99:0/1036503611 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.061+0000 7f5c6ea00700 1 -- 172.21.15.99:0/1036503611 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700798c0 con 0x7f5c7007ad50 2024-03-30T00:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.061+0000 7f5c6ea00700 1 --2- 172.21.15.99:0/1036503611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c700791b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c600098a0 tx=0x7f5c60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c19cd55139a4c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.061+0000 7f5c6e000700 1 -- 172.21.15.99:0/1036503611 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c60004160 con 0x7f5c7007ad50 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.062+0000 7f5c6e000700 1 -- 172.21.15.99:0/1036503611 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c600042c0 con 0x7f5c7007ad50 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.062+0000 7f5c6e000700 1 -- 172.21.15.99:0/1036503611 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c600118e0 con 0x7f5c7007ad50 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.062+0000 7f5c75a00700 1 -- 172.21.15.99:0/1036503611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 msgr2=0x7f5c700791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.062+0000 7f5c75a00700 1 --2- 172.21.15.99:0/1036503611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c700791b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c600098a0 tx=0x7f5c60009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.062+0000 7f5c75a00700 1 -- 172.21.15.99:0/1036503611 shutdown_connections 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.062+0000 7f5c75a00700 1 --2- 172.21.15.99:0/1036503611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c700791b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.062+0000 7f5c75a00700 1 -- 172.21.15.99:0/1036503611 >> 172.21.15.99:0/1036503611 conn(0x7f5c701019a0 msgr2=0x7f5c70103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.062+0000 7f5c75a00700 1 -- 172.21.15.99:0/1036503611 shutdown_connections 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.062+0000 7f5c75a00700 1 -- 172.21.15.99:0/1036503611 wait complete. 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.063+0000 7f5c75a00700 1 Processor -- start 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.064+0000 7f5c75a00700 1 -- start start 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.064+0000 7f5c75a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c7019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.064+0000 7f5c75a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7019c890 con 0x7f5c7007ad50 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.064+0000 7f5c6ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c7019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.064+0000 7f5c6ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c7019c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48734/0 (socket says 172.21.15.99:48734) 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.064+0000 7f5c6ea00700 1 -- 172.21.15.99:0/877674712 learned_addr learned my addr 172.21.15.99:0/877674712 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.064+0000 7f5c6ea00700 1 -- 172.21.15.99:0/877674712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c60009550 con 0x7f5c7007ad50 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.065+0000 7f5c6ea00700 1 --2- 172.21.15.99:0/877674712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c7019c350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5c60003f70 tx=0x7f5c600119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.065+0000 7f5c67400700 1 -- 172.21.15.99:0/877674712 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c600036a0 con 0x7f5c7007ad50 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.065+0000 7f5c67400700 1 -- 172.21.15.99:0/877674712 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c6001b440 con 0x7f5c7007ad50 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.065+0000 7f5c67400700 1 -- 172.21.15.99:0/877674712 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c60023400 con 0x7f5c7007ad50 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.065+0000 7f5c75a00700 1 -- 172.21.15.99:0/877674712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7019ca90 con 0x7f5c7007ad50 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.065+0000 7f5c75a00700 1 -- 172.21.15.99:0/877674712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7019cf30 con 0x7f5c7007ad50 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.066+0000 7f5c67400700 1 -- 172.21.15.99:0/877674712 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f5c60023630 con 0x7f5c7007ad50 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.067+0000 7f5c75a00700 1 -- 172.21.15.99:0/877674712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7010c800 con 0x7f5c7007ad50 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.067+0000 7f5c67400700 1 --2- 172.21.15.99:0/877674712 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5c54038750 0x7f5c5403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.067+0000 7f5c67400700 1 -- 172.21.15.99:0/877674712 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5c6004c810 con 0x7f5c7007ad50 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.067+0000 7f5c67e00700 1 --2- 172.21.15.99:0/877674712 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5c54038750 0x7f5c5403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.067+0000 7f5c67e00700 1 --2- 172.21.15.99:0/877674712 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5c54038750 0x7f5c5403ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5c58006fd0 tx=0x7f5c58006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.072+0000 7f5c67400700 1 -- 172.21.15.99:0/877674712 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5c6002c3f0 con 0x7f5c7007ad50 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.310+0000 7f5c75a00700 1 -- 172.21.15.99:0/877674712 --> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5c7019f7c0 con 0x7f5c54038750 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.315+0000 7f5c67400700 1 -- 172.21.15.99:0/877674712 <== mgr.14118 v2:172.21.15.99:6800/1748928762 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5c7019f7c0 con 0x7f5c54038750 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.318+0000 7f5c75a00700 1 -- 172.21.15.99:0/877674712 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5c54038750 msgr2=0x7f5c5403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.318+0000 7f5c75a00700 1 --2- 172.21.15.99:0/877674712 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5c54038750 0x7f5c5403ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5c58006fd0 tx=0x7f5c58006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.318+0000 7f5c75a00700 1 -- 172.21.15.99:0/877674712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 msgr2=0x7f5c7019c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.318+0000 7f5c75a00700 1 --2- 172.21.15.99:0/877674712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c7019c350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5c60003f70 tx=0x7f5c600119a0 comp rx=0 tx=0).stop 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.318+0000 7f5c75a00700 1 -- 172.21.15.99:0/877674712 shutdown_connections 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.318+0000 7f5c75a00700 1 --2- 172.21.15.99:0/877674712 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f5c54038750 0x7f5c5403ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.318+0000 7f5c75a00700 1 --2- 172.21.15.99:0/877674712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7007ad50 0x7f5c7019c350 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.318+0000 7f5c75a00700 1 -- 172.21.15.99:0/877674712 >> 172.21.15.99:0/877674712 conn(0x7f5c701019a0 msgr2=0x7f5c701025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.318+0000 7f5c75a00700 1 -- 172.21.15.99:0/877674712 shutdown_connections 2024-03-30T00:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:56.318+0000 7f5c75a00700 1 -- 172.21.15.99:0/877674712 wait complete. 2024-03-30T00:26:57.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.037+0000 7fbc49600700 1 Processor -- start 2024-03-30T00:26:57.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.037+0000 7fbc49600700 1 -- start start 2024-03-30T00:26:57.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.037+0000 7fbc49600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc440791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.037+0000 7fbc49600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc44079780 con 0x7fbc4407ad50 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.038+0000 7fbc42a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc440791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.038+0000 7fbc42a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc440791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48748/0 (socket says 172.21.15.99:48748) 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.038+0000 7fbc42a00700 1 -- 172.21.15.99:0/3222494705 learned_addr learned my addr 172.21.15.99:0/3222494705 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.038+0000 7fbc42a00700 1 -- 172.21.15.99:0/3222494705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc440798c0 con 0x7fbc4407ad50 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.038+0000 7fbc42a00700 1 --2- 172.21.15.99:0/3222494705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc440791b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc340098a0 tx=0x7fbc34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234b8471d89b04cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.039+0000 7fbc42000700 1 -- 172.21.15.99:0/3222494705 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc34004160 con 0x7fbc4407ad50 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.039+0000 7fbc42000700 1 -- 172.21.15.99:0/3222494705 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc340042c0 con 0x7fbc4407ad50 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.039+0000 7fbc42000700 1 -- 172.21.15.99:0/3222494705 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc340118e0 con 0x7fbc4407ad50 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.039+0000 7fbc49600700 1 -- 172.21.15.99:0/3222494705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 msgr2=0x7fbc440791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.039+0000 7fbc49600700 1 --2- 172.21.15.99:0/3222494705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc440791b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc340098a0 tx=0x7fbc34009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.039+0000 7fbc49600700 1 -- 172.21.15.99:0/3222494705 shutdown_connections 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.039+0000 7fbc49600700 1 --2- 172.21.15.99:0/3222494705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc440791b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.039+0000 7fbc49600700 1 -- 172.21.15.99:0/3222494705 >> 172.21.15.99:0/3222494705 conn(0x7fbc44101990 msgr2=0x7fbc44103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.039+0000 7fbc49600700 1 -- 172.21.15.99:0/3222494705 shutdown_connections 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.040+0000 7fbc49600700 1 -- 172.21.15.99:0/3222494705 wait complete. 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.040+0000 7fbc49600700 1 Processor -- start 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.041+0000 7fbc49600700 1 -- start start 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.041+0000 7fbc49600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc4419c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.041+0000 7fbc49600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4419c880 con 0x7fbc4407ad50 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.041+0000 7fbc42a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc4419c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.041+0000 7fbc42a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc4419c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48762/0 (socket says 172.21.15.99:48762) 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.041+0000 7fbc42a00700 1 -- 172.21.15.99:0/3289204086 learned_addr learned my addr 172.21.15.99:0/3289204086 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.041+0000 7fbc42a00700 1 -- 172.21.15.99:0/3289204086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc34009550 con 0x7fbc4407ad50 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.042+0000 7fbc42a00700 1 --2- 172.21.15.99:0/3289204086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc4419c340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc34003f70 tx=0x7fbc340119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:57.635 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.042+0000 7fbc3b400700 1 -- 172.21.15.99:0/3289204086 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc340036a0 con 0x7fbc4407ad50 2024-03-30T00:26:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.042+0000 7fbc3b400700 1 -- 172.21.15.99:0/3289204086 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc3401b440 con 0x7fbc4407ad50 2024-03-30T00:26:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.042+0000 7fbc49600700 1 -- 172.21.15.99:0/3289204086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4419ca80 con 0x7fbc4407ad50 2024-03-30T00:26:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.042+0000 7fbc3b400700 1 -- 172.21.15.99:0/3289204086 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc34023400 con 0x7fbc4407ad50 2024-03-30T00:26:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.043+0000 7fbc49600700 1 -- 172.21.15.99:0/3289204086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4419cf20 con 0x7fbc4407ad50 2024-03-30T00:26:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.044+0000 7fbc49600700 1 -- 172.21.15.99:0/3289204086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4410c810 con 0x7fbc4407ad50 2024-03-30T00:26:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.044+0000 7fbc3b400700 1 -- 172.21.15.99:0/3289204086 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fbc3401b8b0 con 0x7fbc4407ad50 2024-03-30T00:26:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.045+0000 7fbc3b400700 1 --2- 172.21.15.99:0/3289204086 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fbc28038750 0x7fbc2803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.045+0000 7fbc3b400700 1 -- 172.21.15.99:0/3289204086 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbc3404c790 con 0x7fbc4407ad50 2024-03-30T00:26:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.045+0000 7fbc3be00700 1 --2- 172.21.15.99:0/3289204086 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fbc28038750 0x7fbc2803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.046+0000 7fbc3be00700 1 --2- 172.21.15.99:0/3289204086 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fbc28038750 0x7fbc2803ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c006fd0 tx=0x7fbc2c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:57.636 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.050+0000 7fbc3b400700 1 -- 172.21.15.99:0/3289204086 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbc34018760 con 0x7fbc4407ad50 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.269+0000 7fbc49600700 1 -- 172.21.15.99:0/3289204086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fbc4410ca50 con 0x7fbc4407ad50 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.273+0000 7fbc3b400700 1 -- 172.21.15.99:0/3289204086 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fbc3400b3e0 con 0x7fbc4407ad50 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.276+0000 7fbc49600700 1 -- 172.21.15.99:0/3289204086 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fbc28038750 msgr2=0x7fbc2803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.276+0000 7fbc49600700 1 --2- 172.21.15.99:0/3289204086 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fbc28038750 0x7fbc2803ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c006fd0 tx=0x7fbc2c006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.276+0000 7fbc49600700 1 -- 172.21.15.99:0/3289204086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 msgr2=0x7fbc4419c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.276+0000 7fbc49600700 1 --2- 172.21.15.99:0/3289204086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc4419c340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc34003f70 tx=0x7fbc340119a0 comp rx=0 tx=0).stop 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.277+0000 7fbc49600700 1 -- 172.21.15.99:0/3289204086 shutdown_connections 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.277+0000 7fbc49600700 1 --2- 172.21.15.99:0/3289204086 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fbc28038750 0x7fbc2803ac10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.277+0000 7fbc49600700 1 --2- 172.21.15.99:0/3289204086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc4407ad50 0x7fbc4419c340 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.277+0000 7fbc49600700 1 -- 172.21.15.99:0/3289204086 >> 172.21.15.99:0/3289204086 conn(0x7fbc44101990 msgr2=0x7fbc44102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.277+0000 7fbc49600700 1 -- 172.21.15.99:0/3289204086 shutdown_connections 2024-03-30T00:26:57.637 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:57.277+0000 7fbc49600700 1 -- 172.21.15.99:0/3289204086 wait complete. 2024-03-30T00:26:57.637 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:57 smithi099 ceph-mon[38079]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:26:57.638 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:57 smithi099 ceph-mon[38079]: Saving service mgr spec with placement count:2 2024-03-30T00:26:57.638 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:57 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:57.638 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:57 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3289204086' entity='client.admin' 2024-03-30T00:26:57.638 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:57 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:26:58.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.046+0000 7f26b4e00700 1 Processor -- start 2024-03-30T00:26:58.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.046+0000 7f26b4e00700 1 -- start start 2024-03-30T00:26:58.666 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.047+0000 7f26b4e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b00791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.047+0000 7f26b4e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b00797a0 con 0x7f26b007ad70 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.047+0000 7f26ade00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b00791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.047+0000 7f26ade00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b00791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48778/0 (socket says 172.21.15.99:48778) 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.047+0000 7f26ade00700 1 -- 172.21.15.99:0/4152655387 learned_addr learned my addr 172.21.15.99:0/4152655387 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.047+0000 7f26ade00700 1 -- 172.21.15.99:0/4152655387 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b00798e0 con 0x7f26b007ad70 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.048+0000 7f26ade00700 1 --2- 172.21.15.99:0/4152655387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b00791d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f26a00098a0 tx=0x7f26a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296ed6abb1e47ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.048+0000 7f26aca00700 1 -- 172.21.15.99:0/4152655387 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26a0004160 con 0x7f26b007ad70 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.048+0000 7f26aca00700 1 -- 172.21.15.99:0/4152655387 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26a00042c0 con 0x7f26b007ad70 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.048+0000 7f26aca00700 1 -- 172.21.15.99:0/4152655387 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26a00118e0 con 0x7f26b007ad70 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.048+0000 7f26b4e00700 1 -- 172.21.15.99:0/4152655387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 msgr2=0x7f26b00791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.048+0000 7f26b4e00700 1 --2- 172.21.15.99:0/4152655387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b00791d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f26a00098a0 tx=0x7f26a0009bb0 comp rx=0 tx=0).stop 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.049+0000 7f26b4e00700 1 -- 172.21.15.99:0/4152655387 shutdown_connections 2024-03-30T00:26:58.667 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.049+0000 7f26b4e00700 1 --2- 172.21.15.99:0/4152655387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b00791d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:58.668 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.049+0000 7f26b4e00700 1 -- 172.21.15.99:0/4152655387 >> 172.21.15.99:0/4152655387 conn(0x7f26b01019a0 msgr2=0x7f26b0103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:58.668 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.049+0000 7f26b4e00700 1 -- 172.21.15.99:0/4152655387 shutdown_connections 2024-03-30T00:26:58.668 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.049+0000 7f26b4e00700 1 -- 172.21.15.99:0/4152655387 wait complete. 2024-03-30T00:26:58.668 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.050+0000 7f26b4e00700 1 Processor -- start 2024-03-30T00:26:58.668 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.050+0000 7f26b4e00700 1 -- start start 2024-03-30T00:26:58.668 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.050+0000 7f26b4e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:58.668 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.050+0000 7f26b4e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b019c920 con 0x7f26b007ad70 2024-03-30T00:26:58.668 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.050+0000 7f26ade00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:58.668 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.050+0000 7f26ade00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48782/0 (socket says 172.21.15.99:48782) 2024-03-30T00:26:58.668 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.050+0000 7f26ade00700 1 -- 172.21.15.99:0/3418389240 learned_addr learned my addr 172.21.15.99:0/3418389240 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.051+0000 7f26ade00700 1 -- 172.21.15.99:0/3418389240 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a0009550 con 0x7f26b007ad70 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.051+0000 7f26ade00700 1 --2- 172.21.15.99:0/3418389240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b019c3e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26a0003f70 tx=0x7f26a00119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.051+0000 7f26a6a00700 1 -- 172.21.15.99:0/3418389240 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26a00036a0 con 0x7f26b007ad70 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.051+0000 7f26a6a00700 1 -- 172.21.15.99:0/3418389240 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26a001b440 con 0x7f26b007ad70 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.051+0000 7f26b4e00700 1 -- 172.21.15.99:0/3418389240 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b019cb20 con 0x7f26b007ad70 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.051+0000 7f26a6a00700 1 -- 172.21.15.99:0/3418389240 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f26a0023400 con 0x7f26b007ad70 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.051+0000 7f26b4e00700 1 -- 172.21.15.99:0/3418389240 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b019cfc0 con 0x7f26b007ad70 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.052+0000 7f26b4e00700 1 -- 172.21.15.99:0/3418389240 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b010c7e0 con 0x7f26b007ad70 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.052+0000 7f26a6a00700 1 -- 172.21.15.99:0/3418389240 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f26a001b5b0 con 0x7f26b007ad70 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.053+0000 7f26a6a00700 1 --2- 172.21.15.99:0/3418389240 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f2694038750 0x7f269403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.053+0000 7f26a6a00700 1 -- 172.21.15.99:0/3418389240 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f26a004c6b0 con 0x7f26b007ad70 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.053+0000 7f26ad400700 1 --2- 172.21.15.99:0/3418389240 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f2694038750 0x7f269403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:26:58.669 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.054+0000 7f26ad400700 1 --2- 172.21.15.99:0/3418389240 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f2694038750 0x7f269403ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f269c006fd0 tx=0x7f269c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:26:58.670 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.059+0000 7f26a6a00700 1 -- 172.21.15.99:0/3418389240 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f26a001ba90 con 0x7f26b007ad70 2024-03-30T00:26:58.670 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.300+0000 7f26b4e00700 1 -- 172.21.15.99:0/3418389240 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f26b010ca20 con 0x7f26b007ad70 2024-03-30T00:26:58.670 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.304+0000 7f26a6a00700 1 -- 172.21.15.99:0/3418389240 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f26a000b3e0 con 0x7f26b007ad70 2024-03-30T00:26:58.670 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.306+0000 7f26b4e00700 1 -- 172.21.15.99:0/3418389240 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f2694038750 msgr2=0x7f269403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:58.670 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.306+0000 7f26b4e00700 1 --2- 172.21.15.99:0/3418389240 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f2694038750 0x7f269403ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f269c006fd0 tx=0x7f269c006e40 comp rx=0 tx=0).stop 2024-03-30T00:26:58.670 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.306+0000 7f26b4e00700 1 -- 172.21.15.99:0/3418389240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 msgr2=0x7f26b019c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:26:58.670 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.306+0000 7f26b4e00700 1 --2- 172.21.15.99:0/3418389240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b019c3e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26a0003f70 tx=0x7f26a00119a0 comp rx=0 tx=0).stop 2024-03-30T00:26:58.670 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.307+0000 7f26b4e00700 1 -- 172.21.15.99:0/3418389240 shutdown_connections 2024-03-30T00:26:58.670 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.307+0000 7f26b4e00700 1 --2- 172.21.15.99:0/3418389240 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7f2694038750 0x7f269403ac10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:58.670 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.307+0000 7f26b4e00700 1 --2- 172.21.15.99:0/3418389240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26b007ad70 0x7f26b019c3e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:26:58.670 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.307+0000 7f26b4e00700 1 -- 172.21.15.99:0/3418389240 >> 172.21.15.99:0/3418389240 conn(0x7f26b01019a0 msgr2=0x7f26b01026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:26:58.671 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.307+0000 7f26b4e00700 1 -- 172.21.15.99:0/3418389240 shutdown_connections 2024-03-30T00:26:58.671 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:58.307+0000 7f26b4e00700 1 -- 172.21.15.99:0/3418389240 wait complete. 2024-03-30T00:26:58.671 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling the dashboard module... 2024-03-30T00:26:59.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:59 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3418389240' entity='client.admin' 2024-03-30T00:26:59.657 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:26:59 smithi099 ceph-mon[38079]: from='mgr.14118 172.21.15.99:0/1199735907' entity='mgr.x' 2024-03-30T00:27:00.624 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:00 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3954213101' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-30T00:27:00.624 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:00 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: ignoring --setuser ceph since I am not root 2024-03-30T00:27:00.625 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:00 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: ignoring --setgroup ceph since I am not root 2024-03-30T00:27:00.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.038+0000 7fec6cc00700 1 Processor -- start 2024-03-30T00:27:00.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.039+0000 7fec6cc00700 1 -- start start 2024-03-30T00:27:00.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.039+0000 7fec6cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec68109370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:00.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.039+0000 7fec6cc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec68062010 con 0x7fec68106f80 2024-03-30T00:27:00.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.039+0000 7fec65e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec68109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:00.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.039+0000 7fec65e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec68109370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48798/0 (socket says 172.21.15.99:48798) 2024-03-30T00:27:00.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.039+0000 7fec65e00700 1 -- 172.21.15.99:0/2320890254 learned_addr learned my addr 172.21.15.99:0/2320890254 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:00.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.040+0000 7fec65e00700 1 -- 172.21.15.99:0/2320890254 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec68062150 con 0x7fec68106f80 2024-03-30T00:27:00.630 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.040+0000 7fec65e00700 1 --2- 172.21.15.99:0/2320890254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec68109370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec580098a0 tx=0x7fec58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818c5bc3c589e170 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.040+0000 7fec64a00700 1 -- 172.21.15.99:0/2320890254 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec58004160 con 0x7fec68106f80 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.040+0000 7fec64a00700 1 -- 172.21.15.99:0/2320890254 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec580042c0 con 0x7fec68106f80 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.040+0000 7fec64a00700 1 -- 172.21.15.99:0/2320890254 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec580118e0 con 0x7fec68106f80 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.041+0000 7fec6cc00700 1 -- 172.21.15.99:0/2320890254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 msgr2=0x7fec68109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.041+0000 7fec6cc00700 1 --2- 172.21.15.99:0/2320890254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec68109370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec580098a0 tx=0x7fec58009bb0 comp rx=0 tx=0).stop 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.041+0000 7fec6cc00700 1 -- 172.21.15.99:0/2320890254 shutdown_connections 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.041+0000 7fec6cc00700 1 --2- 172.21.15.99:0/2320890254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec68109370 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.041+0000 7fec6cc00700 1 -- 172.21.15.99:0/2320890254 >> 172.21.15.99:0/2320890254 conn(0x7fec681008f0 msgr2=0x7fec68102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.041+0000 7fec6cc00700 1 -- 172.21.15.99:0/2320890254 shutdown_connections 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.041+0000 7fec6cc00700 1 -- 172.21.15.99:0/2320890254 wait complete. 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.042+0000 7fec6cc00700 1 Processor -- start 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.042+0000 7fec6cc00700 1 -- start start 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.043+0000 7fec6cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec6819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:00.631 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.043+0000 7fec6cc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6819c8f0 con 0x7fec68106f80 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.043+0000 7fec65e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec6819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.043+0000 7fec65e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec6819c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48814/0 (socket says 172.21.15.99:48814) 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.043+0000 7fec65e00700 1 -- 172.21.15.99:0/3954213101 learned_addr learned my addr 172.21.15.99:0/3954213101 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.043+0000 7fec65e00700 1 -- 172.21.15.99:0/3954213101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec58009550 con 0x7fec68106f80 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.044+0000 7fec65e00700 1 --2- 172.21.15.99:0/3954213101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec6819c3b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fec580046e0 tx=0x7fec580119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.044+0000 7fec5ea00700 1 -- 172.21.15.99:0/3954213101 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec580037d0 con 0x7fec68106f80 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.044+0000 7fec5ea00700 1 -- 172.21.15.99:0/3954213101 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec5801b440 con 0x7fec68106f80 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.044+0000 7fec6cc00700 1 -- 172.21.15.99:0/3954213101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6819caf0 con 0x7fec68106f80 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.044+0000 7fec5ea00700 1 -- 172.21.15.99:0/3954213101 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec58023400 con 0x7fec68106f80 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.044+0000 7fec6cc00700 1 -- 172.21.15.99:0/3954213101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6819cf90 con 0x7fec68106f80 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.045+0000 7fec6cc00700 1 -- 172.21.15.99:0/3954213101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6810a5c0 con 0x7fec68106f80 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.045+0000 7fec5ea00700 1 -- 172.21.15.99:0/3954213101 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fec5801b840 con 0x7fec68106f80 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.046+0000 7fec5ea00700 1 --2- 172.21.15.99:0/3954213101 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fec4c038750 0x7fec4c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:00.632 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.046+0000 7fec5ea00700 1 -- 172.21.15.99:0/3954213101 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fec5804c6c0 con 0x7fec68106f80 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.046+0000 7fec65400700 1 --2- 172.21.15.99:0/3954213101 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fec4c038750 0x7fec4c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.046+0000 7fec65400700 1 --2- 172.21.15.99:0/3954213101 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fec4c038750 0x7fec4c03ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fec54006fd0 tx=0x7fec54006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.050+0000 7fec5ea00700 1 -- 172.21.15.99:0/3954213101 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fec5802b760 con 0x7fec68106f80 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:26:59.342+0000 7fec6cc00700 1 -- 172.21.15.99:0/3954213101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fec6810a830 con 0x7fec68106f80 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.311+0000 7fec5ea00700 1 -- 172.21.15.99:0/3954213101 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fec58023630 con 0x7fec68106f80 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.313+0000 7fec5ea00700 1 -- 172.21.15.99:0/3954213101 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fec5800b5a0 con 0x7fec68106f80 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.316+0000 7fec6cc00700 1 -- 172.21.15.99:0/3954213101 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fec4c038750 msgr2=0x7fec4c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.316+0000 7fec6cc00700 1 --2- 172.21.15.99:0/3954213101 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fec4c038750 0x7fec4c03ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fec54006fd0 tx=0x7fec54006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.316+0000 7fec6cc00700 1 -- 172.21.15.99:0/3954213101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 msgr2=0x7fec6819c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.316+0000 7fec6cc00700 1 --2- 172.21.15.99:0/3954213101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec6819c3b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fec580046e0 tx=0x7fec580119a0 comp rx=0 tx=0).stop 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.316+0000 7fec6cc00700 1 -- 172.21.15.99:0/3954213101 shutdown_connections 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.316+0000 7fec6cc00700 1 --2- 172.21.15.99:0/3954213101 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fec4c038750 0x7fec4c03ac10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.316+0000 7fec6cc00700 1 --2- 172.21.15.99:0/3954213101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec68106f80 0x7fec6819c3b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:00.633 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.316+0000 7fec6cc00700 1 -- 172.21.15.99:0/3954213101 >> 172.21.15.99:0/3954213101 conn(0x7fec681008f0 msgr2=0x7fec68101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:00.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.316+0000 7fec6cc00700 1 -- 172.21.15.99:0/3954213101 shutdown_connections 2024-03-30T00:27:00.634 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:00.316+0000 7fec6cc00700 1 -- 172.21.15.99:0/3954213101 wait complete. 2024-03-30T00:27:01.238 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:00 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:00.937+0000 7f2eb3134200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T00:27:01.239 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:01 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:01.063+0000 7f2eb3134200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T00:27:01.239 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:01 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:01.179+0000 7f2eb3134200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T00:27:01.648 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-30T00:27:01.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:01 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3954213101' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-30T00:27:01.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:01 smithi099 ceph-mon[38079]: mgrmap e9: x(active, since 15s) 2024-03-30T00:27:01.650 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:01 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/174480962' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-30T00:27:01.650 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:01 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:01.291+0000 7f2eb3134200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T00:27:01.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-30T00:27:01.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-30T00:27:01.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-30T00:27:01.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-30T00:27:01.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-30T00:27:01.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.023+0000 7faca9200700 1 Processor -- start 2024-03-30T00:27:01.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.023+0000 7faca9200700 1 -- start start 2024-03-30T00:27:01.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.023+0000 7faca9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca40791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:01.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.023+0000 7faca9200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4079780 con 0x7faca407ad50 2024-03-30T00:27:01.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.023+0000 7faca1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca40791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:01.655 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.023+0000 7faca1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca40791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48824/0 (socket says 172.21.15.99:48824) 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.023+0000 7faca1e00700 1 -- 172.21.15.99:0/3638557776 learned_addr learned my addr 172.21.15.99:0/3638557776 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.024+0000 7faca1e00700 1 -- 172.21.15.99:0/3638557776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca40798c0 con 0x7faca407ad50 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.024+0000 7faca1e00700 1 --2- 172.21.15.99:0/3638557776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca40791b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac940098a0 tx=0x7fac94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46598a31279a53ec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.024+0000 7faca0a00700 1 -- 172.21.15.99:0/3638557776 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac94004160 con 0x7faca407ad50 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.024+0000 7faca0a00700 1 -- 172.21.15.99:0/3638557776 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac940042c0 con 0x7faca407ad50 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.024+0000 7faca0a00700 1 -- 172.21.15.99:0/3638557776 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac940118e0 con 0x7faca407ad50 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.025+0000 7faca9200700 1 -- 172.21.15.99:0/3638557776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 msgr2=0x7faca40791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.025+0000 7faca9200700 1 --2- 172.21.15.99:0/3638557776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca40791b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac940098a0 tx=0x7fac94009bb0 comp rx=0 tx=0).stop 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.025+0000 7faca9200700 1 -- 172.21.15.99:0/3638557776 shutdown_connections 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.025+0000 7faca9200700 1 --2- 172.21.15.99:0/3638557776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca40791b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.025+0000 7faca9200700 1 -- 172.21.15.99:0/3638557776 >> 172.21.15.99:0/3638557776 conn(0x7faca4101990 msgr2=0x7faca4103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.025+0000 7faca9200700 1 -- 172.21.15.99:0/3638557776 shutdown_connections 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.025+0000 7faca9200700 1 -- 172.21.15.99:0/3638557776 wait complete. 2024-03-30T00:27:01.656 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.026+0000 7faca9200700 1 Processor -- start 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.026+0000 7faca9200700 1 -- start start 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.026+0000 7faca9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.026+0000 7faca9200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca419c8c0 con 0x7faca407ad50 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.027+0000 7faca1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.027+0000 7faca1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca419c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48834/0 (socket says 172.21.15.99:48834) 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.027+0000 7faca1e00700 1 -- 172.21.15.99:0/174480962 learned_addr learned my addr 172.21.15.99:0/174480962 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.027+0000 7faca1e00700 1 -- 172.21.15.99:0/174480962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac94009550 con 0x7faca407ad50 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.027+0000 7faca1e00700 1 --2- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca419c380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac94003f70 tx=0x7fac940119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.027+0000 7fac9aa00700 1 -- 172.21.15.99:0/174480962 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac940036a0 con 0x7faca407ad50 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.028+0000 7fac9aa00700 1 -- 172.21.15.99:0/174480962 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac9401b440 con 0x7faca407ad50 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.028+0000 7faca9200700 1 -- 172.21.15.99:0/174480962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca419cac0 con 0x7faca407ad50 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.028+0000 7fac9aa00700 1 -- 172.21.15.99:0/174480962 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac94023400 con 0x7faca407ad50 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.028+0000 7faca9200700 1 -- 172.21.15.99:0/174480962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca419cf60 con 0x7faca407ad50 2024-03-30T00:27:01.657 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.029+0000 7fac9aa00700 1 -- 172.21.15.99:0/174480962 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fac94003800 con 0x7faca407ad50 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.029+0000 7faca9200700 1 -- 172.21.15.99:0/174480962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca410c810 con 0x7faca407ad50 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.030+0000 7fac9aa00700 1 --2- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fac880387a0 0x7fac8803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.030+0000 7fac9aa00700 1 -- 172.21.15.99:0/174480962 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fac9404c750 con 0x7faca407ad50 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.030+0000 7faca1400700 1 -- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fac880387a0 msgr2=0x7fac8803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1748928762 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.030+0000 7faca1400700 1 --2- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fac880387a0 0x7fac8803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.035+0000 7fac9aa00700 1 -- 172.21.15.99:0/174480962 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fac9402b740 con 0x7faca407ad50 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.231+0000 7faca1400700 1 -- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fac880387a0 msgr2=0x7fac8803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1748928762 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.231+0000 7faca1400700 1 --2- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fac880387a0 0x7fac8803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.302+0000 7faca9200700 1 -- 172.21.15.99:0/174480962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7faca419f7a0 con 0x7faca407ad50 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.302+0000 7fac9aa00700 1 -- 172.21.15.99:0/174480962 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fac9402b350 con 0x7faca407ad50 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.305+0000 7faca9200700 1 -- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fac880387a0 msgr2=0x7fac8803ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T00:27:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.305+0000 7faca9200700 1 --2- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fac880387a0 0x7fac8803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.305+0000 7faca9200700 1 -- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 msgr2=0x7faca419c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.305+0000 7faca9200700 1 --2- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca419c380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac94003f70 tx=0x7fac940119a0 comp rx=0 tx=0).stop 2024-03-30T00:27:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.305+0000 7faca9200700 1 -- 172.21.15.99:0/174480962 shutdown_connections 2024-03-30T00:27:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.305+0000 7faca9200700 1 --2- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fac880387a0 0x7fac8803ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.305+0000 7faca9200700 1 --2- 172.21.15.99:0/174480962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca407ad50 0x7faca419c380 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.305+0000 7faca9200700 1 -- 172.21.15.99:0/174480962 >> 172.21.15.99:0/174480962 conn(0x7faca4101990 msgr2=0x7faca4102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.305+0000 7faca9200700 1 -- 172.21.15.99:0/174480962 shutdown_connections 2024-03-30T00:27:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:01.305+0000 7faca9200700 1 -- 172.21.15.99:0/174480962 wait complete. 2024-03-30T00:27:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for the mgr to restart... 2024-03-30T00:27:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:Waiting for mgr epoch 9... 2024-03-30T00:27:02.488 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:02.188+0000 7f2eb3134200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:27:02.952 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:02.505+0000 7f2eb3134200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T00:27:02.952 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:02.614+0000 7f2eb3134200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T00:27:02.952 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:02.750+0000 7f2eb3134200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T00:27:02.952 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:02.944+0000 7f2eb3134200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T00:27:03.227 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:03 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:03.108+0000 7f2eb3134200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T00:27:03.488 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:03 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:03.219+0000 7f2eb3134200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T00:27:03.988 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:03 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:03.687+0000 7f2eb3134200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T00:27:04.488 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:04 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:04.172+0000 7f2eb3134200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T00:27:04.488 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:04 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:04.419+0000 7f2eb3134200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T00:27:04.988 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:04 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:04.540+0000 7f2eb3134200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T00:27:05.738 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:05 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:05.477+0000 7f2eb3134200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T00:27:05.738 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:05 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:05.601+0000 7f2eb3134200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T00:27:06.238 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:05 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:05.790+0000 7f2eb3134200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T00:27:06.988 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:06 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:06.581+0000 7f2eb3134200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T00:27:06.988 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:06 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:06.816+0000 7f2eb3134200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T00:27:07.488 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:07 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:07.068+0000 7f2eb3134200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T00:27:08.488 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:08.192+0000 7f2eb3134200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T00:27:08.938 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:08.564+0000 7f2eb3134200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:27:08.938 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:08.680+0000 7f2eb3134200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T00:27:09.238 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:08.930+0000 7f2eb3134200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T00:27:09.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-mon[38079]: Active manager daemon x restarted 2024-03-30T00:27:09.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-mon[38079]: Activating manager daemon x 2024-03-30T00:27:09.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-mon[38079]: osdmap e3: 0 total, 0 up, 0 in 2024-03-30T00:27:09.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-mon[38079]: mgrmap e10: x(active, starting, since 0.00377852s) 2024-03-30T00:27:09.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:27:09.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-30T00:27:09.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-30T00:27:09.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-30T00:27:09.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-30T00:27:09.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-mon[38079]: Manager daemon x is now available 2024-03-30T00:27:09.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:27:10.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-30T00:27:10.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-30T00:27:10.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:09 smithi099 ceph-mon[38079]: mgrmap e11: x(active, since 1.00982s) 2024-03-30T00:27:10.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout { 2024-03-30T00:27:10.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-30T00:27:10.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-30T00:27:10.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout } 2024-03-30T00:27:10.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.090+0000 7fc175400700 1 Processor -- start 2024-03-30T00:27:10.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.090+0000 7fc175400700 1 -- start start 2024-03-30T00:27:10.371 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.090+0000 7fc175400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc170106860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.090+0000 7fc175400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170106e30 con 0x7fc170106440 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.091+0000 7fc16e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc170106860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.091+0000 7fc16e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc170106860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48836/0 (socket says 172.21.15.99:48836) 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.091+0000 7fc16e800700 1 -- 172.21.15.99:0/1661760843 learned_addr learned my addr 172.21.15.99:0/1661760843 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.091+0000 7fc16e800700 1 -- 172.21.15.99:0/1661760843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc170107640 con 0x7fc170106440 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.091+0000 7fc16e800700 1 --2- 172.21.15.99:0/1661760843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc170106860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1600098a0 tx=0x7fc160009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f5576722a85765 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.091+0000 7fc16d400700 1 -- 172.21.15.99:0/1661760843 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc160004160 con 0x7fc170106440 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.092+0000 7fc16d400700 1 -- 172.21.15.99:0/1661760843 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1600042c0 con 0x7fc170106440 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.092+0000 7fc16d400700 1 -- 172.21.15.99:0/1661760843 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1600118e0 con 0x7fc170106440 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.092+0000 7fc175400700 1 -- 172.21.15.99:0/1661760843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 msgr2=0x7fc170106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.092+0000 7fc175400700 1 --2- 172.21.15.99:0/1661760843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc170106860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1600098a0 tx=0x7fc160009bb0 comp rx=0 tx=0).stop 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.092+0000 7fc175400700 1 -- 172.21.15.99:0/1661760843 shutdown_connections 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.092+0000 7fc175400700 1 --2- 172.21.15.99:0/1661760843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc170106860 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:10.372 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.092+0000 7fc175400700 1 -- 172.21.15.99:0/1661760843 >> 172.21.15.99:0/1661760843 conn(0x7fc1701019c0 msgr2=0x7fc170103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.092+0000 7fc175400700 1 -- 172.21.15.99:0/1661760843 shutdown_connections 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.092+0000 7fc175400700 1 -- 172.21.15.99:0/1661760843 wait complete. 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.093+0000 7fc175400700 1 Processor -- start 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.094+0000 7fc175400700 1 -- start start 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.094+0000 7fc175400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc17007a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.094+0000 7fc175400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17007af30 con 0x7fc170106440 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.094+0000 7fc16e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc17007a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.094+0000 7fc16e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc17007a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48844/0 (socket says 172.21.15.99:48844) 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.094+0000 7fc16e800700 1 -- 172.21.15.99:0/2158450207 learned_addr learned my addr 172.21.15.99:0/2158450207 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.094+0000 7fc16e800700 1 -- 172.21.15.99:0/2158450207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc160009550 con 0x7fc170106440 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.095+0000 7fc16e800700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc17007a9f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc160003f70 tx=0x7fc1600119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.095+0000 7fc167400700 1 -- 172.21.15.99:0/2158450207 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1600036a0 con 0x7fc170106440 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.095+0000 7fc167400700 1 -- 172.21.15.99:0/2158450207 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc16001b440 con 0x7fc170106440 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.095+0000 7fc167400700 1 -- 172.21.15.99:0/2158450207 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc160023400 con 0x7fc170106440 2024-03-30T00:27:10.373 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.095+0000 7fc175400700 1 -- 172.21.15.99:0/2158450207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc170079170 con 0x7fc170106440 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.095+0000 7fc175400700 1 -- 172.21.15.99:0/2158450207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc170079610 con 0x7fc170106440 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.096+0000 7fc167400700 1 -- 172.21.15.99:0/2158450207 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fc16001b5b0 con 0x7fc170106440 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.097+0000 7fc167400700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 0x7fc15403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.097+0000 7fc175400700 1 -- 172.21.15.99:0/2158450207 --> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc170108490 con 0x7fc1540387a0 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.097+0000 7fc167400700 1 -- 172.21.15.99:0/2158450207 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc16004c6e0 con 0x7fc170106440 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.097+0000 7fc16de00700 1 -- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 msgr2=0x7fc15403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1748928762 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.097+0000 7fc16de00700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 0x7fc15403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.298+0000 7fc16de00700 1 -- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 msgr2=0x7fc15403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1748928762 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.298+0000 7fc16de00700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 0x7fc15403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.699+0000 7fc16de00700 1 -- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 msgr2=0x7fc15403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1748928762 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:02.699+0000 7fc16de00700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 0x7fc15403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-30T00:27:10.374 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:03.500+0000 7fc16de00700 1 -- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 msgr2=0x7fc15403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1748928762 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:03.500+0000 7fc16de00700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 0x7fc15403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:05.101+0000 7fc16de00700 1 -- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 msgr2=0x7fc15403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1748928762 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:05.101+0000 7fc16de00700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 0x7fc15403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:08.304+0000 7fc16de00700 1 -- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 msgr2=0x7fc15403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.99:6800/1748928762 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:08.304+0000 7fc16de00700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 0x7fc15403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:08.935+0000 7fc167400700 1 -- 172.21.15.99:0/2158450207 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mgrmap(e 10) v1 ==== 45313+0+0 (secure 0 0 0) 0x7fc160003800 con 0x7fc170106440 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:08.935+0000 7fc167400700 1 -- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 msgr2=0x7fc15403ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:08.935+0000 7fc167400700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 0x7fc15403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.939+0000 7fc167400700 1 -- 172.21.15.99:0/2158450207 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 11) v1 ==== 45440+0+0 (secure 0 0 0) 0x7fc16004df50 con 0x7fc170106440 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.939+0000 7fc167400700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc15403bfc0 0x7fc15403e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.940+0000 7fc167400700 1 -- 172.21.15.99:0/2158450207 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc15403e8f0 con 0x7fc15403bfc0 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.940+0000 7fc16de00700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc15403bfc0 0x7fc15403e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.942+0000 7fc16de00700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc15403bfc0 0x7fc15403e3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc15c003a10 tx=0x7fc15c0092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:10.375 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.942+0000 7fc167400700 1 -- 172.21.15.99:0/2158450207 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fc15403e8f0 con 0x7fc15403bfc0 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.949+0000 7fc175400700 1 -- 172.21.15.99:0/2158450207 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc1700623e0 con 0x7fc15403bfc0 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.949+0000 7fc167400700 1 -- 172.21.15.99:0/2158450207 <== mgr.14150 v2:172.21.15.99:6800/1501731896 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fc1700623e0 con 0x7fc15403bfc0 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.950+0000 7fc175400700 1 -- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc15403bfc0 msgr2=0x7fc15403e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.950+0000 7fc175400700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc15403bfc0 0x7fc15403e3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc15c003a10 tx=0x7fc15c0092b0 comp rx=0 tx=0).stop 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.950+0000 7fc175400700 1 -- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 msgr2=0x7fc17007a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.950+0000 7fc175400700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc17007a9f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc160003f70 tx=0x7fc1600119a0 comp rx=0 tx=0).stop 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.950+0000 7fc175400700 1 -- 172.21.15.99:0/2158450207 shutdown_connections 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.950+0000 7fc175400700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc15403bfc0 0x7fc15403e3b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.950+0000 7fc175400700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc170106440 0x7fc17007a9f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.950+0000 7fc175400700 1 --2- 172.21.15.99:0/2158450207 >> [v2:172.21.15.99:6800/1748928762,v1:172.21.15.99:6801/1748928762] conn(0x7fc1540387a0 0x7fc15403ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.950+0000 7fc175400700 1 -- 172.21.15.99:0/2158450207 >> 172.21.15.99:0/2158450207 conn(0x7fc1701019c0 msgr2=0x7fc1701025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.950+0000 7fc175400700 1 -- 172.21.15.99:0/2158450207 shutdown_connections 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:09.950+0000 7fc175400700 1 -- 172.21.15.99:0/2158450207 wait complete. 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:mgr epoch 9 is available 2024-03-30T00:27:10.376 INFO:teuthology.orchestra.run.smithi099.stdout:Generating a dashboard self-signed certificate... 2024-03-30T00:27:11.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:11 smithi099 ceph-mon[38079]: [30/Mar/2024:00:27:10] ENGINE Bus STARTING 2024-03-30T00:27:11.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:11.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:11 smithi099 ceph-mon[38079]: [30/Mar/2024:00:27:10] ENGINE Serving on http://172.21.15.99:8765 2024-03-30T00:27:11.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:11 smithi099 ceph-mon[38079]: [30/Mar/2024:00:27:10] ENGINE Serving on https://172.21.15.99:7150 2024-03-30T00:27:11.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:11 smithi099 ceph-mon[38079]: [30/Mar/2024:00:27:10] ENGINE Bus STARTED 2024-03-30T00:27:11.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:11.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:11.507 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-30T00:27:11.507 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.773+0000 7fb1a2400700 1 Processor -- start 2024-03-30T00:27:11.507 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.773+0000 7fb1a2400700 1 -- start start 2024-03-30T00:27:11.507 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.774+0000 7fb1a2400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c079190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.774+0000 7fb1a2400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c079760 con 0x7fb19c07ad30 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.774+0000 7fb19b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.774+0000 7fb19b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c079190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44812/0 (socket says 172.21.15.99:44812) 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.774+0000 7fb19b400700 1 -- 172.21.15.99:0/2861653174 learned_addr learned my addr 172.21.15.99:0/2861653174 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.774+0000 7fb19b400700 1 -- 172.21.15.99:0/2861653174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c0798a0 con 0x7fb19c07ad30 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.775+0000 7fb19b400700 1 --2- 172.21.15.99:0/2861653174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c079190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0098a0 tx=0x7fb18c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1a66d2e2428021 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.775+0000 7fb19a000700 1 -- 172.21.15.99:0/2861653174 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb18c004160 con 0x7fb19c07ad30 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.775+0000 7fb19a000700 1 -- 172.21.15.99:0/2861653174 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb18c0042c0 con 0x7fb19c07ad30 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.775+0000 7fb19a000700 1 -- 172.21.15.99:0/2861653174 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb18c0118e0 con 0x7fb19c07ad30 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.775+0000 7fb1a2400700 1 -- 172.21.15.99:0/2861653174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 msgr2=0x7fb19c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.775+0000 7fb1a2400700 1 --2- 172.21.15.99:0/2861653174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c079190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0098a0 tx=0x7fb18c009bb0 comp rx=0 tx=0).stop 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.775+0000 7fb1a2400700 1 -- 172.21.15.99:0/2861653174 shutdown_connections 2024-03-30T00:27:11.508 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.775+0000 7fb1a2400700 1 --2- 172.21.15.99:0/2861653174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c079190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.775+0000 7fb1a2400700 1 -- 172.21.15.99:0/2861653174 >> 172.21.15.99:0/2861653174 conn(0x7fb19c101990 msgr2=0x7fb19c103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.776+0000 7fb1a2400700 1 -- 172.21.15.99:0/2861653174 shutdown_connections 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.776+0000 7fb1a2400700 1 -- 172.21.15.99:0/2861653174 wait complete. 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.777+0000 7fb1a2400700 1 Processor -- start 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.777+0000 7fb1a2400700 1 -- start start 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.777+0000 7fb1a2400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.777+0000 7fb1a2400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c19c870 con 0x7fb19c07ad30 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.777+0000 7fb19b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.777+0000 7fb19b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c19c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44826/0 (socket says 172.21.15.99:44826) 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.777+0000 7fb19b400700 1 -- 172.21.15.99:0/513357981 learned_addr learned my addr 172.21.15.99:0/513357981 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.778+0000 7fb19b400700 1 -- 172.21.15.99:0/513357981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c009550 con 0x7fb19c07ad30 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.778+0000 7fb19b400700 1 --2- 172.21.15.99:0/513357981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c19c330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb18c003f70 tx=0x7fb18c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.778+0000 7fb193e00700 1 -- 172.21.15.99:0/513357981 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb18c0036a0 con 0x7fb19c07ad30 2024-03-30T00:27:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.778+0000 7fb193e00700 1 -- 172.21.15.99:0/513357981 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb18c01b440 con 0x7fb19c07ad30 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.778+0000 7fb1a2400700 1 -- 172.21.15.99:0/513357981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c19ca70 con 0x7fb19c07ad30 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.778+0000 7fb193e00700 1 -- 172.21.15.99:0/513357981 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb18c023400 con 0x7fb19c07ad30 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.778+0000 7fb1a2400700 1 -- 172.21.15.99:0/513357981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c19cf10 con 0x7fb19c07ad30 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.779+0000 7fb193e00700 1 -- 172.21.15.99:0/513357981 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 11) v1 ==== 45440+0+0 (secure 0 0 0) 0x7fb18c023630 con 0x7fb19c07ad30 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.780+0000 7fb193e00700 1 --2- 172.21.15.99:0/513357981 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb180038670 0x7fb18003ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.780+0000 7fb193e00700 1 -- 172.21.15.99:0/513357981 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb18c04c750 con 0x7fb19c07ad30 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.780+0000 7fb1a2400700 1 -- 172.21.15.99:0/513357981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c10c7d0 con 0x7fb19c07ad30 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.780+0000 7fb19aa00700 1 --2- 172.21.15.99:0/513357981 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb180038670 0x7fb18003ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.781+0000 7fb19aa00700 1 --2- 172.21.15.99:0/513357981 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb180038670 0x7fb18003ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb188006fd0 tx=0x7fb188006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:10.789+0000 7fb193e00700 1 -- 172.21.15.99:0/513357981 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb18c02c3f0 con 0x7fb19c07ad30 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.043+0000 7fb1a2400700 1 -- 172.21.15.99:0/513357981 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb19c19f7a0 con 0x7fb180038670 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.156+0000 7fb193e00700 1 -- 172.21.15.99:0/513357981 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb19c19f7a0 con 0x7fb180038670 2024-03-30T00:27:11.510 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.158+0000 7fb1a2400700 1 -- 172.21.15.99:0/513357981 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb180038670 msgr2=0x7fb18003ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:11.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.158+0000 7fb1a2400700 1 --2- 172.21.15.99:0/513357981 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb180038670 0x7fb18003ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb188006fd0 tx=0x7fb188006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:11.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.159+0000 7fb1a2400700 1 -- 172.21.15.99:0/513357981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 msgr2=0x7fb19c19c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:11.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.159+0000 7fb1a2400700 1 --2- 172.21.15.99:0/513357981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c19c330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb18c003f70 tx=0x7fb18c0119a0 comp rx=0 tx=0).stop 2024-03-30T00:27:11.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.159+0000 7fb1a2400700 1 -- 172.21.15.99:0/513357981 shutdown_connections 2024-03-30T00:27:11.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.159+0000 7fb1a2400700 1 --2- 172.21.15.99:0/513357981 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb180038670 0x7fb18003ab30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:11.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.159+0000 7fb1a2400700 1 --2- 172.21.15.99:0/513357981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb19c07ad30 0x7fb19c19c330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:11.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.159+0000 7fb1a2400700 1 -- 172.21.15.99:0/513357981 >> 172.21.15.99:0/513357981 conn(0x7fb19c101990 msgr2=0x7fb19c1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:11.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.159+0000 7fb1a2400700 1 -- 172.21.15.99:0/513357981 shutdown_connections 2024-03-30T00:27:11.511 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.159+0000 7fb1a2400700 1 -- 172.21.15.99:0/513357981 wait complete. 2024-03-30T00:27:11.511 INFO:teuthology.orchestra.run.smithi099.stdout:Creating initial admin user... 2024-03-30T00:27:12.338 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:12 smithi099 ceph-mon[38079]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:27:12.339 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:12 smithi099 ceph-mon[38079]: mgrmap e12: x(active, since 2s) 2024-03-30T00:27:12.339 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:12.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$b7qRnR/.LMbSp6pyU0zhhuqUsOct1fP2Z3yvIqvz06KJuz/2BpWsy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711758432, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-30T00:27:12.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.894+0000 7fc2a7800700 1 Processor -- start 2024-03-30T00:27:12.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.894+0000 7fc2a7800700 1 -- start start 2024-03-30T00:27:12.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.894+0000 7fc2a7800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a00791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:12.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.894+0000 7fc2a7800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a0079780 con 0x7fc2a007ad50 2024-03-30T00:27:12.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.895+0000 7fc2a4e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a00791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:12.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.895+0000 7fc2a4e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a00791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44840/0 (socket says 172.21.15.99:44840) 2024-03-30T00:27:12.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.895+0000 7fc2a4e00700 1 -- 172.21.15.99:0/4003460522 learned_addr learned my addr 172.21.15.99:0/4003460522 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:12.810 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.895+0000 7fc2a4e00700 1 -- 172.21.15.99:0/4003460522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a00798c0 con 0x7fc2a007ad50 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.896+0000 7fc2a4e00700 1 --2- 172.21.15.99:0/4003460522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a00791b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2940098a0 tx=0x7fc294009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df595283450e588 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.896+0000 7fc29f400700 1 -- 172.21.15.99:0/4003460522 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc294004160 con 0x7fc2a007ad50 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.896+0000 7fc29f400700 1 -- 172.21.15.99:0/4003460522 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2940042c0 con 0x7fc2a007ad50 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.896+0000 7fc29f400700 1 -- 172.21.15.99:0/4003460522 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2940118e0 con 0x7fc2a007ad50 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.897+0000 7fc2a7800700 1 -- 172.21.15.99:0/4003460522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 msgr2=0x7fc2a00791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.897+0000 7fc2a7800700 1 --2- 172.21.15.99:0/4003460522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a00791b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2940098a0 tx=0x7fc294009bb0 comp rx=0 tx=0).stop 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.897+0000 7fc2a7800700 1 -- 172.21.15.99:0/4003460522 shutdown_connections 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.897+0000 7fc2a7800700 1 --2- 172.21.15.99:0/4003460522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a00791b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.897+0000 7fc2a7800700 1 -- 172.21.15.99:0/4003460522 >> 172.21.15.99:0/4003460522 conn(0x7fc2a01019c0 msgr2=0x7fc2a0103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.897+0000 7fc2a7800700 1 -- 172.21.15.99:0/4003460522 shutdown_connections 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.897+0000 7fc2a7800700 1 -- 172.21.15.99:0/4003460522 wait complete. 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.898+0000 7fc2a7800700 1 Processor -- start 2024-03-30T00:27:12.811 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.898+0000 7fc2a7800700 1 -- start start 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.898+0000 7fc2a7800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.898+0000 7fc2a7800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a019c860 con 0x7fc2a007ad50 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.899+0000 7fc2a4e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.899+0000 7fc2a4e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a019c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44844/0 (socket says 172.21.15.99:44844) 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.899+0000 7fc2a4e00700 1 -- 172.21.15.99:0/664103674 learned_addr learned my addr 172.21.15.99:0/664103674 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.899+0000 7fc2a4e00700 1 -- 172.21.15.99:0/664103674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc294009550 con 0x7fc2a007ad50 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.899+0000 7fc2a4e00700 1 --2- 172.21.15.99:0/664103674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a019c320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc294003f70 tx=0x7fc2940119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.899+0000 7fc29d600700 1 -- 172.21.15.99:0/664103674 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2940036a0 con 0x7fc2a007ad50 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.899+0000 7fc2a7800700 1 -- 172.21.15.99:0/664103674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a019ca60 con 0x7fc2a007ad50 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.899+0000 7fc29d600700 1 -- 172.21.15.99:0/664103674 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc29401b440 con 0x7fc2a007ad50 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.899+0000 7fc29d600700 1 -- 172.21.15.99:0/664103674 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc294023400 con 0x7fc2a007ad50 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.900+0000 7fc2a7800700 1 -- 172.21.15.99:0/664103674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a019cf00 con 0x7fc2a007ad50 2024-03-30T00:27:12.812 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.901+0000 7fc2a7800700 1 -- 172.21.15.99:0/664103674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a010c820 con 0x7fc2a007ad50 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.901+0000 7fc29d600700 1 -- 172.21.15.99:0/664103674 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fc29401b5b0 con 0x7fc2a007ad50 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.901+0000 7fc29d600700 1 --2- 172.21.15.99:0/664103674 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc2880387a0 0x7fc28803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.901+0000 7fc29d600700 1 -- 172.21.15.99:0/664103674 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc29404c810 con 0x7fc2a007ad50 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.902+0000 7fc29fe00700 1 --2- 172.21.15.99:0/664103674 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc2880387a0 0x7fc28803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.902+0000 7fc29fe00700 1 --2- 172.21.15.99:0/664103674 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc2880387a0 0x7fc28803ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc28c006fd0 tx=0x7fc28c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:11.907+0000 7fc29d600700 1 -- 172.21.15.99:0/664103674 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc2940195b0 con 0x7fc2a007ad50 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.143+0000 7fc2a7800700 1 -- 172.21.15.99:0/664103674 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc2a0074570 con 0x7fc2880387a0 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.375+0000 7fc29d600700 1 -- 172.21.15.99:0/664103674 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fc2a0074570 con 0x7fc2880387a0 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.378+0000 7fc2a7800700 1 -- 172.21.15.99:0/664103674 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc2880387a0 msgr2=0x7fc28803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.378+0000 7fc2a7800700 1 --2- 172.21.15.99:0/664103674 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc2880387a0 0x7fc28803ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc28c006fd0 tx=0x7fc28c006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.378+0000 7fc2a7800700 1 -- 172.21.15.99:0/664103674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 msgr2=0x7fc2a019c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.378+0000 7fc2a7800700 1 --2- 172.21.15.99:0/664103674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a019c320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc294003f70 tx=0x7fc2940119a0 comp rx=0 tx=0).stop 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.378+0000 7fc2a7800700 1 -- 172.21.15.99:0/664103674 shutdown_connections 2024-03-30T00:27:12.813 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.378+0000 7fc2a7800700 1 --2- 172.21.15.99:0/664103674 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc2880387a0 0x7fc28803ac60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:12.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.378+0000 7fc2a7800700 1 --2- 172.21.15.99:0/664103674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a007ad50 0x7fc2a019c320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:12.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.378+0000 7fc2a7800700 1 -- 172.21.15.99:0/664103674 >> 172.21.15.99:0/664103674 conn(0x7fc2a01019c0 msgr2=0x7fc2a0102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:12.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.378+0000 7fc2a7800700 1 -- 172.21.15.99:0/664103674 shutdown_connections 2024-03-30T00:27:12.814 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:12.378+0000 7fc2a7800700 1 -- 172.21.15.99:0/664103674 wait complete. 2024-03-30T00:27:12.814 INFO:teuthology.orchestra.run.smithi099.stdout:Fetching dashboard port number... 2024-03-30T00:27:13.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stdout 8443 2024-03-30T00:27:13.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.184+0000 7ff070200700 1 Processor -- start 2024-03-30T00:27:13.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.184+0000 7ff070200700 1 -- start start 2024-03-30T00:27:13.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.184+0000 7ff070200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff068106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:13.713 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.184+0000 7ff070200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff068106e10 con 0x7ff068106420 2024-03-30T00:27:13.713 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.184+0000 7ff06d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff068106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:13.713 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:13 smithi099 ceph-mon[38079]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:27:13.713 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:13 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.185+0000 7ff06d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff068106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44858/0 (socket says 172.21.15.99:44858) 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.185+0000 7ff06d800700 1 -- 172.21.15.99:0/484064273 learned_addr learned my addr 172.21.15.99:0/484064273 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.185+0000 7ff06d800700 1 -- 172.21.15.99:0/484064273 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff068107620 con 0x7ff068106420 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.185+0000 7ff06d800700 1 --2- 172.21.15.99:0/484064273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff068106840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0098a0 tx=0x7ff05c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be26b97a481e1d26 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.185+0000 7ff067e00700 1 -- 172.21.15.99:0/484064273 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff05c004160 con 0x7ff068106420 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.185+0000 7ff067e00700 1 -- 172.21.15.99:0/484064273 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff05c0042c0 con 0x7ff068106420 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.185+0000 7ff067e00700 1 -- 172.21.15.99:0/484064273 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff05c0118e0 con 0x7ff068106420 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.186+0000 7ff070200700 1 -- 172.21.15.99:0/484064273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 msgr2=0x7ff068106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.186+0000 7ff070200700 1 --2- 172.21.15.99:0/484064273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff068106840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0098a0 tx=0x7ff05c009bb0 comp rx=0 tx=0).stop 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.186+0000 7ff070200700 1 -- 172.21.15.99:0/484064273 shutdown_connections 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.186+0000 7ff070200700 1 --2- 172.21.15.99:0/484064273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff068106840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:13.718 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.186+0000 7ff070200700 1 -- 172.21.15.99:0/484064273 >> 172.21.15.99:0/484064273 conn(0x7ff0681019a0 msgr2=0x7ff068103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.186+0000 7ff070200700 1 -- 172.21.15.99:0/484064273 shutdown_connections 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.186+0000 7ff070200700 1 -- 172.21.15.99:0/484064273 wait complete. 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.187+0000 7ff070200700 1 Processor -- start 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.187+0000 7ff070200700 1 -- start start 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.188+0000 7ff070200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff06807a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.188+0000 7ff070200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06807af30 con 0x7ff068106420 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.188+0000 7ff06d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff06807a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.188+0000 7ff06d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff06807a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44874/0 (socket says 172.21.15.99:44874) 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.188+0000 7ff06d800700 1 -- 172.21.15.99:0/2497140435 learned_addr learned my addr 172.21.15.99:0/2497140435 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.188+0000 7ff06d800700 1 -- 172.21.15.99:0/2497140435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c009550 con 0x7ff068106420 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.188+0000 7ff06d800700 1 --2- 172.21.15.99:0/2497140435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff06807a9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff05c006af0 tx=0x7ff05c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.189+0000 7ff066000700 1 -- 172.21.15.99:0/2497140435 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff05c0037d0 con 0x7ff068106420 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.189+0000 7ff070200700 1 -- 172.21.15.99:0/2497140435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff068079170 con 0x7ff068106420 2024-03-30T00:27:13.719 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.189+0000 7ff066000700 1 -- 172.21.15.99:0/2497140435 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff05c01b440 con 0x7ff068106420 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.189+0000 7ff066000700 1 -- 172.21.15.99:0/2497140435 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff05c023400 con 0x7ff068106420 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.189+0000 7ff070200700 1 -- 172.21.15.99:0/2497140435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff068079610 con 0x7ff068106420 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.190+0000 7ff070200700 1 -- 172.21.15.99:0/2497140435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0680623e0 con 0x7ff068106420 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.190+0000 7ff066000700 1 -- 172.21.15.99:0/2497140435 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7ff05c01b5b0 con 0x7ff068106420 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.190+0000 7ff066000700 1 --2- 172.21.15.99:0/2497140435 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff050038700 0x7ff05003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.190+0000 7ff066000700 1 -- 172.21.15.99:0/2497140435 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff05c04c8d0 con 0x7ff068106420 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.190+0000 7ff06ce00700 1 --2- 172.21.15.99:0/2497140435 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff050038700 0x7ff05003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.191+0000 7ff06ce00700 1 --2- 172.21.15.99:0/2497140435 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff050038700 0x7ff05003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff054006fd0 tx=0x7ff054006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.195+0000 7ff066000700 1 -- 172.21.15.99:0/2497140435 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff05c01b8d0 con 0x7ff068106420 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.398+0000 7ff070200700 1 -- 172.21.15.99:0/2497140435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7ff06810a820 con 0x7ff068106420 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.398+0000 7ff066000700 1 -- 172.21.15.99:0/2497140435 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7ff05c00b3e0 con 0x7ff068106420 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.400+0000 7ff070200700 1 -- 172.21.15.99:0/2497140435 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff050038700 msgr2=0x7ff05003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:13.720 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.400+0000 7ff070200700 1 --2- 172.21.15.99:0/2497140435 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff050038700 0x7ff05003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff054006fd0 tx=0x7ff054006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:13.721 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.401+0000 7ff070200700 1 -- 172.21.15.99:0/2497140435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 msgr2=0x7ff06807a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:13.721 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.401+0000 7ff070200700 1 --2- 172.21.15.99:0/2497140435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff06807a9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff05c006af0 tx=0x7ff05c011ed0 comp rx=0 tx=0).stop 2024-03-30T00:27:13.721 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.401+0000 7ff070200700 1 -- 172.21.15.99:0/2497140435 shutdown_connections 2024-03-30T00:27:13.721 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.401+0000 7ff070200700 1 --2- 172.21.15.99:0/2497140435 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff050038700 0x7ff05003abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:13.721 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.401+0000 7ff070200700 1 --2- 172.21.15.99:0/2497140435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff068106420 0x7ff06807a9f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:13.721 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.401+0000 7ff070200700 1 -- 172.21.15.99:0/2497140435 >> 172.21.15.99:0/2497140435 conn(0x7ff0681019a0 msgr2=0x7ff0681025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:13.721 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.401+0000 7ff070200700 1 -- 172.21.15.99:0/2497140435 shutdown_connections 2024-03-30T00:27:13.721 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:13.401+0000 7ff070200700 1 -- 172.21.15.99:0/2497140435 wait complete. 2024-03-30T00:27:13.734 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-30T00:27:13.734 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout disabled 2024-03-30T00:27:13.749 INFO:teuthology.orchestra.run.smithi099.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-30T00:27:13.749 INFO:teuthology.orchestra.run.smithi099.stdout:systemctl: stdout inactive 2024-03-30T00:27:13.749 INFO:teuthology.orchestra.run.smithi099.stdout:firewalld.service is not enabled 2024-03-30T00:27:13.749 INFO:teuthology.orchestra.run.smithi099.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-30T00:27:13.752 INFO:teuthology.orchestra.run.smithi099.stdout:Ceph Dashboard is now available at: 2024-03-30T00:27:13.752 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:13.752 INFO:teuthology.orchestra.run.smithi099.stdout: URL: https://smithi099.front.sepia.ceph.com:8443/ 2024-03-30T00:27:13.752 INFO:teuthology.orchestra.run.smithi099.stdout: User: admin 2024-03-30T00:27:13.752 INFO:teuthology.orchestra.run.smithi099.stdout: Password: 1l5tckzck0 2024-03-30T00:27:13.752 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:13.752 INFO:teuthology.orchestra.run.smithi099.stdout:Saving cluster configuration to /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config directory 2024-03-30T00:27:13.753 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling autotune for osd_memory_target 2024-03-30T00:27:14.704 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.148+0000 7f192ca00700 1 Processor -- start 2024-03-30T00:27:14.704 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.148+0000 7f192ca00700 1 -- start start 2024-03-30T00:27:14.704 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.148+0000 7f192ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f19281067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:14.704 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:14 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2497140435' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-30T00:27:14.709 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.148+0000 7f192ca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1928106da0 con 0x7f19281063b0 2024-03-30T00:27:14.709 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.148+0000 7f1925e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f19281067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:14.709 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.149+0000 7f1925e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f19281067d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44878/0 (socket says 172.21.15.99:44878) 2024-03-30T00:27:14.709 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.149+0000 7f1925e00700 1 -- 172.21.15.99:0/1371799184 learned_addr learned my addr 172.21.15.99:0/1371799184 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.149+0000 7f1925e00700 1 -- 172.21.15.99:0/1371799184 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19281075b0 con 0x7f19281063b0 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.149+0000 7f1925e00700 1 --2- 172.21.15.99:0/1371799184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f19281067d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19180098a0 tx=0x7f1918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345d101a4224f02c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.149+0000 7f1924a00700 1 -- 172.21.15.99:0/1371799184 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1918004160 con 0x7f19281063b0 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.149+0000 7f1924a00700 1 -- 172.21.15.99:0/1371799184 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19180042c0 con 0x7f19281063b0 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.149+0000 7f1924a00700 1 -- 172.21.15.99:0/1371799184 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19180118e0 con 0x7f19281063b0 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.150+0000 7f192ca00700 1 -- 172.21.15.99:0/1371799184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 msgr2=0x7f19281067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.150+0000 7f192ca00700 1 --2- 172.21.15.99:0/1371799184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f19281067d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19180098a0 tx=0x7f1918009bb0 comp rx=0 tx=0).stop 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.150+0000 7f192ca00700 1 -- 172.21.15.99:0/1371799184 shutdown_connections 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.150+0000 7f192ca00700 1 --2- 172.21.15.99:0/1371799184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f19281067d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.150+0000 7f192ca00700 1 -- 172.21.15.99:0/1371799184 >> 172.21.15.99:0/1371799184 conn(0x7f1928101970 msgr2=0x7f1928103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.150+0000 7f192ca00700 1 -- 172.21.15.99:0/1371799184 shutdown_connections 2024-03-30T00:27:14.710 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.150+0000 7f192ca00700 1 -- 172.21.15.99:0/1371799184 wait complete. 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.151+0000 7f192ca00700 1 Processor -- start 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.151+0000 7f192ca00700 1 -- start start 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.152+0000 7f192ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f192819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.152+0000 7f192ca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192819c860 con 0x7f19281063b0 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.152+0000 7f1925e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f192819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.152+0000 7f1925e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f192819c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44880/0 (socket says 172.21.15.99:44880) 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.152+0000 7f1925e00700 1 -- 172.21.15.99:0/1840716577 learned_addr learned my addr 172.21.15.99:0/1840716577 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.152+0000 7f1925e00700 1 -- 172.21.15.99:0/1840716577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1918009550 con 0x7f19281063b0 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.152+0000 7f1925e00700 1 --2- 172.21.15.99:0/1840716577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f192819c320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1918003f70 tx=0x7f19180119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.153+0000 7f191ea00700 1 -- 172.21.15.99:0/1840716577 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19180036a0 con 0x7f19281063b0 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.153+0000 7f191ea00700 1 -- 172.21.15.99:0/1840716577 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f191801b440 con 0x7f19281063b0 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.153+0000 7f192ca00700 1 -- 172.21.15.99:0/1840716577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192819ca60 con 0x7f19281063b0 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.153+0000 7f191ea00700 1 -- 172.21.15.99:0/1840716577 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1918023400 con 0x7f19281063b0 2024-03-30T00:27:14.711 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.153+0000 7f192ca00700 1 -- 172.21.15.99:0/1840716577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192819cf00 con 0x7f19281063b0 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.154+0000 7f191ea00700 1 -- 172.21.15.99:0/1840716577 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7f1918003800 con 0x7f19281063b0 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.154+0000 7f192ca00700 1 -- 172.21.15.99:0/1840716577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192810a7e0 con 0x7f19281063b0 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.154+0000 7f191ea00700 1 --2- 172.21.15.99:0/1840716577 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f190c0387a0 0x7f190c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.155+0000 7f191ea00700 1 -- 172.21.15.99:0/1840716577 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f191804c7c0 con 0x7f19281063b0 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.155+0000 7f1925400700 1 --2- 172.21.15.99:0/1840716577 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f190c0387a0 0x7f190c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.155+0000 7f1925400700 1 --2- 172.21.15.99:0/1840716577 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f190c0387a0 0x7f190c03ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1914006fd0 tx=0x7f1914006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.159+0000 7f191ea00700 1 -- 172.21.15.99:0/1840716577 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f191802b740 con 0x7f19281063b0 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.365+0000 7f192ca00700 1 -- 172.21.15.99:0/1840716577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f192819f790 con 0x7f19281063b0 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.365+0000 7f191ea00700 1 -- 172.21.15.99:0/1840716577 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f191802b350 con 0x7f19281063b0 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.367+0000 7f192ca00700 1 -- 172.21.15.99:0/1840716577 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f190c0387a0 msgr2=0x7f190c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.367+0000 7f192ca00700 1 --2- 172.21.15.99:0/1840716577 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f190c0387a0 0x7f190c03ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1914006fd0 tx=0x7f1914006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.367+0000 7f192ca00700 1 -- 172.21.15.99:0/1840716577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 msgr2=0x7f192819c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.367+0000 7f192ca00700 1 --2- 172.21.15.99:0/1840716577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f192819c320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1918003f70 tx=0x7f19180119a0 comp rx=0 tx=0).stop 2024-03-30T00:27:14.712 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.368+0000 7f192ca00700 1 -- 172.21.15.99:0/1840716577 shutdown_connections 2024-03-30T00:27:14.713 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.368+0000 7f192ca00700 1 --2- 172.21.15.99:0/1840716577 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f190c0387a0 0x7f190c03ac60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:14.713 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.368+0000 7f192ca00700 1 --2- 172.21.15.99:0/1840716577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19281063b0 0x7f192819c320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:14.713 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.368+0000 7f192ca00700 1 -- 172.21.15.99:0/1840716577 >> 172.21.15.99:0/1840716577 conn(0x7f1928101970 msgr2=0x7f1928102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:14.713 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.368+0000 7f192ca00700 1 -- 172.21.15.99:0/1840716577 shutdown_connections 2024-03-30T00:27:14.713 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:14.368+0000 7f192ca00700 1 -- 172.21.15.99:0/1840716577 wait complete. 2024-03-30T00:27:15.751 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.070+0000 7f3da5800700 1 Processor -- start 2024-03-30T00:27:15.751 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.070+0000 7f3da5800700 1 -- start start 2024-03-30T00:27:15.751 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.070+0000 7f3da5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da00791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:15.751 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.070+0000 7f3da5800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da0079780 con 0x7f3da007ad50 2024-03-30T00:27:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.071+0000 7f3d9ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da00791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.071+0000 7f3d9ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da00791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37908/0 (socket says 172.21.15.99:37908) 2024-03-30T00:27:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.071+0000 7f3d9ea00700 1 -- 172.21.15.99:0/1867024458 learned_addr learned my addr 172.21.15.99:0/1867024458 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.071+0000 7f3d9ea00700 1 -- 172.21.15.99:0/1867024458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da00798c0 con 0x7f3da007ad50 2024-03-30T00:27:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.071+0000 7f3d9ea00700 1 --2- 172.21.15.99:0/1867024458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da00791b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d900098a0 tx=0x7f3d90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ce0beadfac1ba7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.071+0000 7f3d9d600700 1 -- 172.21.15.99:0/1867024458 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d90004160 con 0x7f3da007ad50 2024-03-30T00:27:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.072+0000 7f3d9d600700 1 -- 172.21.15.99:0/1867024458 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d900042c0 con 0x7f3da007ad50 2024-03-30T00:27:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.072+0000 7f3d9d600700 1 -- 172.21.15.99:0/1867024458 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d900118e0 con 0x7f3da007ad50 2024-03-30T00:27:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.072+0000 7f3da5800700 1 -- 172.21.15.99:0/1867024458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 msgr2=0x7f3da00791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:15.752 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.072+0000 7f3da5800700 1 --2- 172.21.15.99:0/1867024458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da00791b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d900098a0 tx=0x7f3d90009bb0 comp rx=0 tx=0).stop 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.072+0000 7f3da5800700 1 -- 172.21.15.99:0/1867024458 shutdown_connections 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.072+0000 7f3da5800700 1 --2- 172.21.15.99:0/1867024458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da00791b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.072+0000 7f3da5800700 1 -- 172.21.15.99:0/1867024458 >> 172.21.15.99:0/1867024458 conn(0x7f3da01019a0 msgr2=0x7f3da0103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.072+0000 7f3da5800700 1 -- 172.21.15.99:0/1867024458 shutdown_connections 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.072+0000 7f3da5800700 1 -- 172.21.15.99:0/1867024458 wait complete. 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.073+0000 7f3da5800700 1 Processor -- start 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.073+0000 7f3da5800700 1 -- start start 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.074+0000 7f3da5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.074+0000 7f3da5800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da019c890 con 0x7f3da007ad50 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.074+0000 7f3d9ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.074+0000 7f3d9ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da019c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37924/0 (socket says 172.21.15.99:37924) 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.074+0000 7f3d9ea00700 1 -- 172.21.15.99:0/1834908586 learned_addr learned my addr 172.21.15.99:0/1834908586 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.074+0000 7f3d9ea00700 1 -- 172.21.15.99:0/1834908586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d90009550 con 0x7f3da007ad50 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.074+0000 7f3d9ea00700 1 --2- 172.21.15.99:0/1834908586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da019c350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d90003f70 tx=0x7f3d900119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.075+0000 7f3d97400700 1 -- 172.21.15.99:0/1834908586 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d900036a0 con 0x7f3da007ad50 2024-03-30T00:27:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.075+0000 7f3d97400700 1 -- 172.21.15.99:0/1834908586 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d9001b440 con 0x7f3da007ad50 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.075+0000 7f3da5800700 1 -- 172.21.15.99:0/1834908586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da019ca90 con 0x7f3da007ad50 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.075+0000 7f3d97400700 1 -- 172.21.15.99:0/1834908586 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d90023400 con 0x7f3da007ad50 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.075+0000 7f3da5800700 1 -- 172.21.15.99:0/1834908586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da019cf30 con 0x7f3da007ad50 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.076+0000 7f3d97400700 1 -- 172.21.15.99:0/1834908586 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7f3d9001b5b0 con 0x7f3da007ad50 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.076+0000 7f3da5800700 1 -- 172.21.15.99:0/1834908586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da010c800 con 0x7f3da007ad50 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.076+0000 7f3d97400700 1 --2- 172.21.15.99:0/1834908586 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d84038750 0x7f3d8403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.076+0000 7f3d97400700 1 -- 172.21.15.99:0/1834908586 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3d9004c860 con 0x7f3da007ad50 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.077+0000 7f3d9e000700 1 --2- 172.21.15.99:0/1834908586 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d84038750 0x7f3d8403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.077+0000 7f3d9e000700 1 --2- 172.21.15.99:0/1834908586 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d84038750 0x7f3d8403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c006fd0 tx=0x7f3d8c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.081+0000 7f3d97400700 1 -- 172.21.15.99:0/1834908586 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3d90019840 con 0x7f3da007ad50 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.387+0000 7f3da5800700 1 -- 172.21.15.99:0/1834908586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f3da019f7c0 con 0x7f3da007ad50 2024-03-30T00:27:15.754 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.390+0000 7f3d97400700 1 -- 172.21.15.99:0/1834908586 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f3d90019840 con 0x7f3da007ad50 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.392+0000 7f3da5800700 1 -- 172.21.15.99:0/1834908586 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d84038750 msgr2=0x7f3d8403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.392+0000 7f3da5800700 1 --2- 172.21.15.99:0/1834908586 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d84038750 0x7f3d8403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c006fd0 tx=0x7f3d8c006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.392+0000 7f3da5800700 1 -- 172.21.15.99:0/1834908586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 msgr2=0x7f3da019c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.392+0000 7f3da5800700 1 --2- 172.21.15.99:0/1834908586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da019c350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d90003f70 tx=0x7f3d900119a0 comp rx=0 tx=0).stop 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.392+0000 7f3da5800700 1 -- 172.21.15.99:0/1834908586 shutdown_connections 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.392+0000 7f3da5800700 1 --2- 172.21.15.99:0/1834908586 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d84038750 0x7f3d8403ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.392+0000 7f3da5800700 1 --2- 172.21.15.99:0/1834908586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3da007ad50 0x7f3da019c350 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.392+0000 7f3da5800700 1 -- 172.21.15.99:0/1834908586 >> 172.21.15.99:0/1834908586 conn(0x7f3da01019a0 msgr2=0x7f3da01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.393+0000 7f3da5800700 1 -- 172.21.15.99:0/1834908586 shutdown_connections 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr 2024-03-30T00:27:15.393+0000 7f3da5800700 1 -- 172.21.15.99:0/1834908586 wait complete. 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-30T00:27:15.755 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout:Or, if you are only running a single cluster on this host: 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout: ceph telemetry on 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout:For more information see: 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:15.756 INFO:teuthology.orchestra.run.smithi099.stdout:Bootstrap complete. 2024-03-30T00:27:15.807 INFO:tasks.cephadm:Fetching config... 2024-03-30T00:27:15.807 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:27:15.807 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-30T00:27:15.851 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-30T00:27:15.851 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:27:15.851 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-30T00:27:15.933 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-30T00:27:15.934 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:27:15.934 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/keyring of=/dev/stdout 2024-03-30T00:27:16.044 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-30T00:27:16.044 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:27:16.044 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-30T00:27:16.126 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-30T00:27:16.127 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+Cv5uDS5VlZ9DBDzW/S6j9tw8vu3vM0xFZIFF/UBQccJgQnvp7vcttKmuUnu0uWKeTmXIAXqN+oGrCXw004pkj7M5+/0pujdALwbihqEc+Lx9takTObQbmb8iD4UYs5k8y6+eNqsZixCKA3EQpd++/i3ecO14mvsqrQ15p5yEmo2/gYxIDpsSux+CTeb3MTJC0gLaHQWMRvRRGaxnXqIbJAFAX1i9R0lR6z4+/pudmM7vTAP6Tvz6kz5tTplGjYUR5Bi9tW4aml+7/wBdDl5Ri9fz4by9Gw5ZNylZsGRrUEUPad1gIXo8v8RTUgk5XRWJoY55NqjACSlPRZnGnDVyESYGZdh2PniworL9Gg2cnX1j3mNlFaSCpb4N8Fv1AyofpZ7bB3W48ABu2/UGNqCBJP/Hr8LhFMQPW23k4NdtEVjp97BrAAGJoYcz/pI7BZXwqTEImoQM9sUI6daQgvrTUL0Yz15z9lcGFB5uBo+tTOut0OB9saTDM6IGDy09qUc= ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-30T00:27:16.257 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+Cv5uDS5VlZ9DBDzW/S6j9tw8vu3vM0xFZIFF/UBQccJgQnvp7vcttKmuUnu0uWKeTmXIAXqN+oGrCXw004pkj7M5+/0pujdALwbihqEc+Lx9takTObQbmb8iD4UYs5k8y6+eNqsZixCKA3EQpd++/i3ecO14mvsqrQ15p5yEmo2/gYxIDpsSux+CTeb3MTJC0gLaHQWMRvRRGaxnXqIbJAFAX1i9R0lR6z4+/pudmM7vTAP6Tvz6kz5tTplGjYUR5Bi9tW4aml+7/wBdDl5Ri9fz4by9Gw5ZNylZsGRrUEUPad1gIXo8v8RTUgk5XRWJoY55NqjACSlPRZnGnDVyESYGZdh2PniworL9Gg2cnX1j3mNlFaSCpb4N8Fv1AyofpZ7bB3W48ABu2/UGNqCBJP/Hr8LhFMQPW23k4NdtEVjp97BrAAGJoYcz/pI7BZXwqTEImoQM9sUI6daQgvrTUL0Yz15z9lcGFB5uBo+tTOut0OB9saTDM6IGDy09qUc= ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:27:16.285 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+Cv5uDS5VlZ9DBDzW/S6j9tw8vu3vM0xFZIFF/UBQccJgQnvp7vcttKmuUnu0uWKeTmXIAXqN+oGrCXw004pkj7M5+/0pujdALwbihqEc+Lx9takTObQbmb8iD4UYs5k8y6+eNqsZixCKA3EQpd++/i3ecO14mvsqrQ15p5yEmo2/gYxIDpsSux+CTeb3MTJC0gLaHQWMRvRRGaxnXqIbJAFAX1i9R0lR6z4+/pudmM7vTAP6Tvz6kz5tTplGjYUR5Bi9tW4aml+7/wBdDl5Ri9fz4by9Gw5ZNylZsGRrUEUPad1gIXo8v8RTUgk5XRWJoY55NqjACSlPRZnGnDVyESYGZdh2PniworL9Gg2cnX1j3mNlFaSCpb4N8Fv1AyofpZ7bB3W48ABu2/UGNqCBJP/Hr8LhFMQPW23k4NdtEVjp97BrAAGJoYcz/pI7BZXwqTEImoQM9sUI6daQgvrTUL0Yz15z9lcGFB5uBo+tTOut0OB9saTDM6IGDy09qUc= ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-30T00:27:16.386 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+Cv5uDS5VlZ9DBDzW/S6j9tw8vu3vM0xFZIFF/UBQccJgQnvp7vcttKmuUnu0uWKeTmXIAXqN+oGrCXw004pkj7M5+/0pujdALwbihqEc+Lx9takTObQbmb8iD4UYs5k8y6+eNqsZixCKA3EQpd++/i3ecO14mvsqrQ15p5yEmo2/gYxIDpsSux+CTeb3MTJC0gLaHQWMRvRRGaxnXqIbJAFAX1i9R0lR6z4+/pudmM7vTAP6Tvz6kz5tTplGjYUR5Bi9tW4aml+7/wBdDl5Ri9fz4by9Gw5ZNylZsGRrUEUPad1gIXo8v8RTUgk5XRWJoY55NqjACSlPRZnGnDVyESYGZdh2PniworL9Gg2cnX1j3mNlFaSCpb4N8Fv1AyofpZ7bB3W48ABu2/UGNqCBJP/Hr8LhFMQPW23k4NdtEVjp97BrAAGJoYcz/pI7BZXwqTEImoQM9sUI6daQgvrTUL0Yz15z9lcGFB5uBo+tTOut0OB9saTDM6IGDy09qUc= ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:27:16.414 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+Cv5uDS5VlZ9DBDzW/S6j9tw8vu3vM0xFZIFF/UBQccJgQnvp7vcttKmuUnu0uWKeTmXIAXqN+oGrCXw004pkj7M5+/0pujdALwbihqEc+Lx9takTObQbmb8iD4UYs5k8y6+eNqsZixCKA3EQpd++/i3ecO14mvsqrQ15p5yEmo2/gYxIDpsSux+CTeb3MTJC0gLaHQWMRvRRGaxnXqIbJAFAX1i9R0lR6z4+/pudmM7vTAP6Tvz6kz5tTplGjYUR5Bi9tW4aml+7/wBdDl5Ri9fz4by9Gw5ZNylZsGRrUEUPad1gIXo8v8RTUgk5XRWJoY55NqjACSlPRZnGnDVyESYGZdh2PniworL9Gg2cnX1j3mNlFaSCpb4N8Fv1AyofpZ7bB3W48ABu2/UGNqCBJP/Hr8LhFMQPW23k4NdtEVjp97BrAAGJoYcz/pI7BZXwqTEImoQM9sUI6daQgvrTUL0Yz15z9lcGFB5uBo+tTOut0OB9saTDM6IGDy09qUc= ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-30T00:27:16.515 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+Cv5uDS5VlZ9DBDzW/S6j9tw8vu3vM0xFZIFF/UBQccJgQnvp7vcttKmuUnu0uWKeTmXIAXqN+oGrCXw004pkj7M5+/0pujdALwbihqEc+Lx9takTObQbmb8iD4UYs5k8y6+eNqsZixCKA3EQpd++/i3ecO14mvsqrQ15p5yEmo2/gYxIDpsSux+CTeb3MTJC0gLaHQWMRvRRGaxnXqIbJAFAX1i9R0lR6z4+/pudmM7vTAP6Tvz6kz5tTplGjYUR5Bi9tW4aml+7/wBdDl5Ri9fz4by9Gw5ZNylZsGRrUEUPad1gIXo8v8RTUgk5XRWJoY55NqjACSlPRZnGnDVyESYGZdh2PniworL9Gg2cnX1j3mNlFaSCpb4N8Fv1AyofpZ7bB3W48ABu2/UGNqCBJP/Hr8LhFMQPW23k4NdtEVjp97BrAAGJoYcz/pI7BZXwqTEImoQM9sUI6daQgvrTUL0Yz15z9lcGFB5uBo+tTOut0OB9saTDM6IGDy09qUc= ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:27:16.544 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-30T00:27:16.602 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:16 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1834908586' entity='client.admin' 2024-03-30T00:27:16.602 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:16 smithi099 ceph-mon[38079]: mgrmap e13: x(active, since 6s) 2024-03-30T00:27:16.899 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:27:18.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.151+0000 7faebde00700 1 -- 172.21.15.99:0/2861740888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb8100580 msgr2=0x7faeb81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:18.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.151+0000 7faebde00700 1 --2- 172.21.15.99:0/2861740888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb8100580 0x7faeb81009a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faeac009910 tx=0x7faeac009c20 comp rx=0 tx=0).stop 2024-03-30T00:27:18.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.151+0000 7faebde00700 1 -- 172.21.15.99:0/2861740888 shutdown_connections 2024-03-30T00:27:18.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.151+0000 7faebde00700 1 --2- 172.21.15.99:0/2861740888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb8100580 0x7faeb81009a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:18.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.151+0000 7faebde00700 1 -- 172.21.15.99:0/2861740888 >> 172.21.15.99:0/2861740888 conn(0x7faeb80fbb20 msgr2=0x7faeb80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:18.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.151+0000 7faebde00700 1 -- 172.21.15.99:0/2861740888 shutdown_connections 2024-03-30T00:27:18.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.151+0000 7faebde00700 1 -- 172.21.15.99:0/2861740888 wait complete. 2024-03-30T00:27:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.152+0000 7faebde00700 1 Processor -- start 2024-03-30T00:27:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.152+0000 7faebde00700 1 -- start start 2024-03-30T00:27:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.153+0000 7faebde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb8100580 0x7faeb8074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.153+0000 7faebde00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8074f60 con 0x7faeb8100580 2024-03-30T00:27:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.153+0000 7faebca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb8100580 0x7faeb8074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.153+0000 7faebca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb8100580 0x7faeb8074a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37938/0 (socket says 172.21.15.99:37938) 2024-03-30T00:27:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.153+0000 7faebca00700 1 -- 172.21.15.99:0/2442689618 learned_addr learned my addr 172.21.15.99:0/2442689618 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.153+0000 7faebca00700 1 -- 172.21.15.99:0/2442689618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac0095f0 con 0x7faeb8100580 2024-03-30T00:27:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.153+0000 7faebca00700 1 --2- 172.21.15.99:0/2442689618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb8100580 0x7faeb8074a20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faeac0098e0 tx=0x7faeac005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.154+0000 7faeb5600700 1 -- 172.21.15.99:0/2442689618 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeac00ba90 con 0x7faeb8100580 2024-03-30T00:27:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.154+0000 7faeb5600700 1 -- 172.21.15.99:0/2442689618 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faeac00bbf0 con 0x7faeb8100580 2024-03-30T00:27:18.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.154+0000 7faebde00700 1 -- 172.21.15.99:0/2442689618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb80731a0 con 0x7faeb8100580 2024-03-30T00:27:18.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.154+0000 7faeb5600700 1 -- 172.21.15.99:0/2442689618 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faeac00fe70 con 0x7faeb8100580 2024-03-30T00:27:18.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.154+0000 7faebde00700 1 -- 172.21.15.99:0/2442689618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb8073640 con 0x7faeb8100580 2024-03-30T00:27:18.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.155+0000 7faeb5600700 1 -- 172.21.15.99:0/2442689618 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7faeac00f460 con 0x7faeb8100580 2024-03-30T00:27:18.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.155+0000 7faebde00700 1 -- 172.21.15.99:0/2442689618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb8106a80 con 0x7faeb8100580 2024-03-30T00:27:18.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.155+0000 7faeb5600700 1 --2- 172.21.15.99:0/2442689618 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faea00387f0 0x7faea003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:18.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.155+0000 7faeb5600700 1 -- 172.21.15.99:0/2442689618 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faeac04c980 con 0x7faeb8100580 2024-03-30T00:27:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.156+0000 7faeb7e00700 1 --2- 172.21.15.99:0/2442689618 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faea00387f0 0x7faea003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.156+0000 7faeb7e00700 1 --2- 172.21.15.99:0/2442689618 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faea00387f0 0x7faea003acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faea4006fd0 tx=0x7faea4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:18.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.161+0000 7faeb5600700 1 -- 172.21.15.99:0/2442689618 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faeac02ad30 con 0x7faeb8100580 2024-03-30T00:27:18.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.373+0000 7faebde00700 1 -- 172.21.15.99:0/2442689618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7faeb80623e0 con 0x7faeb8100580 2024-03-30T00:27:18.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.377+0000 7faeb5600700 1 -- 172.21.15.99:0/2442689618 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7faeac027030 con 0x7faeb8100580 2024-03-30T00:27:18.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.379+0000 7faebde00700 1 -- 172.21.15.99:0/2442689618 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faea00387f0 msgr2=0x7faea003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:18.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.379+0000 7faebde00700 1 --2- 172.21.15.99:0/2442689618 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faea00387f0 0x7faea003acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faea4006fd0 tx=0x7faea4006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:18.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.379+0000 7faebde00700 1 -- 172.21.15.99:0/2442689618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb8100580 msgr2=0x7faeb8074a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:18.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.379+0000 7faebde00700 1 --2- 172.21.15.99:0/2442689618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb8100580 0x7faeb8074a20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faeac0098e0 tx=0x7faeac005810 comp rx=0 tx=0).stop 2024-03-30T00:27:18.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.380+0000 7faebde00700 1 -- 172.21.15.99:0/2442689618 shutdown_connections 2024-03-30T00:27:18.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.380+0000 7faebde00700 1 --2- 172.21.15.99:0/2442689618 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faea00387f0 0x7faea003acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:18.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.380+0000 7faebde00700 1 --2- 172.21.15.99:0/2442689618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeb8100580 0x7faeb8074a20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:18.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.380+0000 7faebde00700 1 -- 172.21.15.99:0/2442689618 >> 172.21.15.99:0/2442689618 conn(0x7faeb80fbb20 msgr2=0x7faeb80fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:18.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.380+0000 7faebde00700 1 -- 172.21.15.99:0/2442689618 shutdown_connections 2024-03-30T00:27:18.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:18.380+0000 7faebde00700 1 -- 172.21.15.99:0/2442689618 wait complete. 2024-03-30T00:27:18.747 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-30T00:27:18.747 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-30T00:27:19.109 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:27:19.570 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:19 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2442689618' entity='client.admin' 2024-03-30T00:27:20.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.294+0000 7fd321e00700 1 -- 172.21.15.99:0/2029021360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c074d80 msgr2=0x7fd31c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:20.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.294+0000 7fd321e00700 1 --2- 172.21.15.99:0/2029021360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c074d80 0x7fd31c0731e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009910 tx=0x7fd30c009c20 comp rx=0 tx=0).stop 2024-03-30T00:27:20.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.294+0000 7fd321e00700 1 -- 172.21.15.99:0/2029021360 shutdown_connections 2024-03-30T00:27:20.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.294+0000 7fd321e00700 1 --2- 172.21.15.99:0/2029021360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c074d80 0x7fd31c0731e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:20.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.294+0000 7fd321e00700 1 -- 172.21.15.99:0/2029021360 >> 172.21.15.99:0/2029021360 conn(0x7fd31c0fbcd0 msgr2=0x7fd31c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:20.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.294+0000 7fd321e00700 1 -- 172.21.15.99:0/2029021360 shutdown_connections 2024-03-30T00:27:20.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.294+0000 7fd321e00700 1 -- 172.21.15.99:0/2029021360 wait complete. 2024-03-30T00:27:20.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.295+0000 7fd321e00700 1 Processor -- start 2024-03-30T00:27:20.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.295+0000 7fd321e00700 1 -- start start 2024-03-30T00:27:20.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.296+0000 7fd321e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c074d80 0x7fd31c197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:20.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.296+0000 7fd321e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c1981d0 con 0x7fd31c074d80 2024-03-30T00:27:20.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.296+0000 7fd31aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c074d80 0x7fd31c197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:20.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.296+0000 7fd31aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c074d80 0x7fd31c197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37964/0 (socket says 172.21.15.99:37964) 2024-03-30T00:27:20.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.296+0000 7fd31aa00700 1 -- 172.21.15.99:0/2473355617 learned_addr learned my addr 172.21.15.99:0/2473355617 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:20.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.296+0000 7fd31aa00700 1 -- 172.21.15.99:0/2473355617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c0095f0 con 0x7fd31c074d80 2024-03-30T00:27:20.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.296+0000 7fd31aa00700 1 --2- 172.21.15.99:0/2473355617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c074d80 0x7fd31c197c90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd30c005950 tx=0x7fd30c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:20.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.297+0000 7fd320a00700 1 -- 172.21.15.99:0/2473355617 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd30c00b960 con 0x7fd31c074d80 2024-03-30T00:27:20.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.297+0000 7fd320a00700 1 -- 172.21.15.99:0/2473355617 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd30c00f460 con 0x7fd31c074d80 2024-03-30T00:27:20.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.297+0000 7fd320a00700 1 -- 172.21.15.99:0/2473355617 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd30c017440 con 0x7fd31c074d80 2024-03-30T00:27:20.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.297+0000 7fd321e00700 1 -- 172.21.15.99:0/2473355617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31c1983d0 con 0x7fd31c074d80 2024-03-30T00:27:20.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.297+0000 7fd321e00700 1 -- 172.21.15.99:0/2473355617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31c198870 con 0x7fd31c074d80 2024-03-30T00:27:20.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.298+0000 7fd320a00700 1 -- 172.21.15.99:0/2473355617 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fd30c00f840 con 0x7fd31c074d80 2024-03-30T00:27:20.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.298+0000 7fd321e00700 1 -- 172.21.15.99:0/2473355617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd31c191500 con 0x7fd31c074d80 2024-03-30T00:27:20.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.298+0000 7fd320a00700 1 --2- 172.21.15.99:0/2473355617 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd3000387f0 0x7fd30003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:20.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.298+0000 7fd320a00700 1 -- 172.21.15.99:0/2473355617 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd30c04c810 con 0x7fd31c074d80 2024-03-30T00:27:20.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.299+0000 7fd313e00700 1 --2- 172.21.15.99:0/2473355617 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd3000387f0 0x7fd30003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:20.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.299+0000 7fd313e00700 1 --2- 172.21.15.99:0/2473355617 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd3000387f0 0x7fd30003acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd304006fd0 tx=0x7fd304006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:20.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.303+0000 7fd320a00700 1 -- 172.21.15.99:0/2473355617 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd30c00be40 con 0x7fd31c074d80 2024-03-30T00:27:20.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.509+0000 7fd321e00700 1 -- 172.21.15.99:0/2473355617 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fd31c0611f0 con 0x7fd3000387f0 2024-03-30T00:27:20.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.513+0000 7fd320a00700 1 -- 172.21.15.99:0/2473355617 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd31c0611f0 con 0x7fd3000387f0 2024-03-30T00:27:20.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.515+0000 7fd321e00700 1 -- 172.21.15.99:0/2473355617 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd3000387f0 msgr2=0x7fd30003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:20.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.515+0000 7fd321e00700 1 --2- 172.21.15.99:0/2473355617 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd3000387f0 0x7fd30003acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd304006fd0 tx=0x7fd304006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:20.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.515+0000 7fd321e00700 1 -- 172.21.15.99:0/2473355617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c074d80 msgr2=0x7fd31c197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:20.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.515+0000 7fd321e00700 1 --2- 172.21.15.99:0/2473355617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c074d80 0x7fd31c197c90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd30c005950 tx=0x7fd30c005810 comp rx=0 tx=0).stop 2024-03-30T00:27:20.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.515+0000 7fd321e00700 1 -- 172.21.15.99:0/2473355617 shutdown_connections 2024-03-30T00:27:20.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.515+0000 7fd321e00700 1 --2- 172.21.15.99:0/2473355617 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd3000387f0 0x7fd30003acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:20.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.516+0000 7fd321e00700 1 --2- 172.21.15.99:0/2473355617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c074d80 0x7fd31c197c90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:20.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.516+0000 7fd321e00700 1 -- 172.21.15.99:0/2473355617 >> 172.21.15.99:0/2473355617 conn(0x7fd31c0fbcd0 msgr2=0x7fd31c0fc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:20.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.516+0000 7fd321e00700 1 -- 172.21.15.99:0/2473355617 shutdown_connections 2024-03-30T00:27:20.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:20.516+0000 7fd321e00700 1 -- 172.21.15.99:0/2473355617 wait complete. 2024-03-30T00:27:20.986 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-03-30T00:27:20.986 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-30T00:27:20.986 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-03-30T00:27:21.033 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-30T00:27:21.033 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-30T00:27:21.117 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-03-30T00:27:21.117 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi142 2024-03-30T00:27:21.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:21.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:21.175 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:27:21.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:21.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:27:21.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:21.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:21.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:27:21.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:27:21.176 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:27:21.483 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:27:22.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:21 smithi099 ceph-mon[38079]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:27:22.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:21 smithi099 ceph-mon[38079]: Updating smithi099:/etc/ceph/ceph.conf 2024-03-30T00:27:22.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:21 smithi099 ceph-mon[38079]: Updating smithi099:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:27:22.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.560+0000 7fb576a00700 1 -- 172.21.15.99:0/3641403740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5701066c0 msgr2=0x7fb570106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:22.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.560+0000 7fb576a00700 1 --2- 172.21.15.99:0/3641403740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5701066c0 0x7fb570106aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb560009910 tx=0x7fb560009c20 comp rx=0 tx=0).stop 2024-03-30T00:27:22.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.560+0000 7fb576a00700 1 -- 172.21.15.99:0/3641403740 shutdown_connections 2024-03-30T00:27:22.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.560+0000 7fb576a00700 1 --2- 172.21.15.99:0/3641403740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5701066c0 0x7fb570106aa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:22.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.560+0000 7fb576a00700 1 -- 172.21.15.99:0/3641403740 >> 172.21.15.99:0/3641403740 conn(0x7fb5700fc1b0 msgr2=0x7fb5700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:22.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.560+0000 7fb576a00700 1 -- 172.21.15.99:0/3641403740 shutdown_connections 2024-03-30T00:27:22.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.560+0000 7fb576a00700 1 -- 172.21.15.99:0/3641403740 wait complete. 2024-03-30T00:27:22.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.561+0000 7fb576a00700 1 Processor -- start 2024-03-30T00:27:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.561+0000 7fb576a00700 1 -- start start 2024-03-30T00:27:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.562+0000 7fb576a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5701066c0 0x7fb570195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.562+0000 7fb576a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5701963d0 con 0x7fb5701066c0 2024-03-30T00:27:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.562+0000 7fb56fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5701066c0 0x7fb570195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.562+0000 7fb56fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5701066c0 0x7fb570195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37980/0 (socket says 172.21.15.99:37980) 2024-03-30T00:27:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.562+0000 7fb56fe00700 1 -- 172.21.15.99:0/1710699328 learned_addr learned my addr 172.21.15.99:0/1710699328 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:22.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.562+0000 7fb56fe00700 1 -- 172.21.15.99:0/1710699328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5600095f0 con 0x7fb5701066c0 2024-03-30T00:27:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.563+0000 7fb56fe00700 1 --2- 172.21.15.99:0/1710699328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5701066c0 0x7fb570195e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb560005660 tx=0x7fb560005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.563+0000 7fb56cc00700 1 -- 172.21.15.99:0/1710699328 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb56001d070 con 0x7fb5701066c0 2024-03-30T00:27:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.563+0000 7fb56cc00700 1 -- 172.21.15.99:0/1710699328 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb56000b6f0 con 0x7fb5701066c0 2024-03-30T00:27:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.563+0000 7fb56cc00700 1 -- 172.21.15.99:0/1710699328 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5600037f0 con 0x7fb5701066c0 2024-03-30T00:27:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.563+0000 7fb576a00700 1 -- 172.21.15.99:0/1710699328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb570196630 con 0x7fb5701066c0 2024-03-30T00:27:22.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.563+0000 7fb576a00700 1 -- 172.21.15.99:0/1710699328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb570196ab0 con 0x7fb5701066c0 2024-03-30T00:27:22.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.564+0000 7fb56cc00700 1 -- 172.21.15.99:0/1710699328 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fb56000b860 con 0x7fb5701066c0 2024-03-30T00:27:22.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.565+0000 7fb576a00700 1 -- 172.21.15.99:0/1710699328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb570101e00 con 0x7fb5701066c0 2024-03-30T00:27:22.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.565+0000 7fb56cc00700 1 --2- 172.21.15.99:0/1710699328 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb554038840 0x7fb55403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:22.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.565+0000 7fb56cc00700 1 -- 172.21.15.99:0/1710699328 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb56004c470 con 0x7fb5701066c0 2024-03-30T00:27:22.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.565+0000 7fb56f400700 1 --2- 172.21.15.99:0/1710699328 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb554038840 0x7fb55403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.566+0000 7fb56f400700 1 --2- 172.21.15.99:0/1710699328 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb554038840 0x7fb55403ad00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb55c006fd0 tx=0x7fb55c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:22.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.570+0000 7fb56cc00700 1 -- 172.21.15.99:0/1710699328 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb56002ad40 con 0x7fb5701066c0 2024-03-30T00:27:22.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:22.800+0000 7fb576a00700 1 -- 172.21.15.99:0/1710699328 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}) v1 -- 0x7fb57010a870 con 0x7fb554038840 2024-03-30T00:27:23.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:22 smithi099 ceph-mon[38079]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-03-30T00:27:23.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:22 smithi099 ceph-mon[38079]: Updating smithi099:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T00:27:23.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:23.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:23.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:24.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:23 smithi099 ceph-mon[38079]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:27:25.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:24 smithi099 ceph-mon[38079]: Deploying cephadm binary to smithi142 2024-03-30T00:27:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:25.710+0000 7fb56cc00700 1 -- 172.21.15.99:0/1710699328 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb57010a870 con 0x7fb554038840 2024-03-30T00:27:25.744 INFO:teuthology.orchestra.run.smithi099.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-03-30T00:27:25.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:25.712+0000 7fb576a00700 1 -- 172.21.15.99:0/1710699328 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb554038840 msgr2=0x7fb55403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:25.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:25.712+0000 7fb576a00700 1 --2- 172.21.15.99:0/1710699328 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb554038840 0x7fb55403ad00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb55c006fd0 tx=0x7fb55c006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:25.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:25.712+0000 7fb576a00700 1 -- 172.21.15.99:0/1710699328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5701066c0 msgr2=0x7fb570195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:25.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:25.712+0000 7fb576a00700 1 --2- 172.21.15.99:0/1710699328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5701066c0 0x7fb570195e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb560005660 tx=0x7fb560005740 comp rx=0 tx=0).stop 2024-03-30T00:27:25.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:25.712+0000 7fb576a00700 1 -- 172.21.15.99:0/1710699328 shutdown_connections 2024-03-30T00:27:25.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:25.712+0000 7fb576a00700 1 --2- 172.21.15.99:0/1710699328 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb554038840 0x7fb55403ad00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:25.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:25.712+0000 7fb576a00700 1 --2- 172.21.15.99:0/1710699328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5701066c0 0x7fb570195e90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:25.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:25.712+0000 7fb576a00700 1 -- 172.21.15.99:0/1710699328 >> 172.21.15.99:0/1710699328 conn(0x7fb5700fc1b0 msgr2=0x7fb570108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:25.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:25.712+0000 7fb576a00700 1 -- 172.21.15.99:0/1710699328 shutdown_connections 2024-03-30T00:27:25.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:25.712+0000 7fb576a00700 1 -- 172.21.15.99:0/1710699328 wait complete. 2024-03-30T00:27:26.062 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-03-30T00:27:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:27:26.811 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:26.811 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:26 smithi099 ceph-mon[38079]: Added host smithi142 2024-03-30T00:27:26.811 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:27:26.811 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:26.811 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:27.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.400+0000 7fa358000700 1 -- 172.21.15.99:0/3768301316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3501066c0 msgr2=0x7fa350106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:27.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.400+0000 7fa358000700 1 --2- 172.21.15.99:0/3768301316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3501066c0 0x7fa350106aa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa344009910 tx=0x7fa344009c20 comp rx=0 tx=0).stop 2024-03-30T00:27:27.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.400+0000 7fa358000700 1 -- 172.21.15.99:0/3768301316 shutdown_connections 2024-03-30T00:27:27.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.400+0000 7fa358000700 1 --2- 172.21.15.99:0/3768301316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3501066c0 0x7fa350106aa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:27.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.400+0000 7fa358000700 1 -- 172.21.15.99:0/3768301316 >> 172.21.15.99:0/3768301316 conn(0x7fa3500fc1b0 msgr2=0x7fa3500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:27.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.401+0000 7fa358000700 1 -- 172.21.15.99:0/3768301316 shutdown_connections 2024-03-30T00:27:27.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.401+0000 7fa358000700 1 -- 172.21.15.99:0/3768301316 wait complete. 2024-03-30T00:27:27.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.402+0000 7fa358000700 1 Processor -- start 2024-03-30T00:27:27.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.402+0000 7fa358000700 1 -- start start 2024-03-30T00:27:27.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.402+0000 7fa358000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3501066c0 0x7fa3500730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:27.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.402+0000 7fa358000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa350074f70 con 0x7fa3501066c0 2024-03-30T00:27:27.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.402+0000 7fa355600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3501066c0 0x7fa3500730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:27.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.402+0000 7fa355600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3501066c0 0x7fa3500730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47018/0 (socket says 172.21.15.99:47018) 2024-03-30T00:27:27.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.402+0000 7fa355600700 1 -- 172.21.15.99:0/2108241464 learned_addr learned my addr 172.21.15.99:0/2108241464 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:27.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.403+0000 7fa355600700 1 -- 172.21.15.99:0/2108241464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3440095f0 con 0x7fa3501066c0 2024-03-30T00:27:27.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.403+0000 7fa355600700 1 --2- 172.21.15.99:0/2108241464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3501066c0 0x7fa3500730b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa3440098e0 tx=0x7fa344005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:27.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.403+0000 7fa34e000700 1 -- 172.21.15.99:0/2108241464 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa34401d070 con 0x7fa3501066c0 2024-03-30T00:27:27.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.403+0000 7fa358000700 1 -- 172.21.15.99:0/2108241464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3500735f0 con 0x7fa3501066c0 2024-03-30T00:27:27.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.403+0000 7fa34e000700 1 -- 172.21.15.99:0/2108241464 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa344009e60 con 0x7fa3501066c0 2024-03-30T00:27:27.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.403+0000 7fa34e000700 1 -- 172.21.15.99:0/2108241464 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa344022b30 con 0x7fa3501066c0 2024-03-30T00:27:27.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.403+0000 7fa358000700 1 -- 172.21.15.99:0/2108241464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa350073a70 con 0x7fa3501066c0 2024-03-30T00:27:27.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.404+0000 7fa34e000700 1 -- 172.21.15.99:0/2108241464 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fa344022470 con 0x7fa3501066c0 2024-03-30T00:27:27.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.404+0000 7fa358000700 1 -- 172.21.15.99:0/2108241464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa350101e00 con 0x7fa3501066c0 2024-03-30T00:27:27.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.405+0000 7fa34e000700 1 --2- 172.21.15.99:0/2108241464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3380387a0 0x7fa33803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:27.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.405+0000 7fa34e000700 1 -- 172.21.15.99:0/2108241464 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa34404cd40 con 0x7fa3501066c0 2024-03-30T00:27:27.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.405+0000 7fa354c00700 1 --2- 172.21.15.99:0/2108241464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3380387a0 0x7fa33803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:27.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.406+0000 7fa354c00700 1 --2- 172.21.15.99:0/2108241464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3380387a0 0x7fa33803ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa33c006fd0 tx=0x7fa33c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:27.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.410+0000 7fa34e000700 1 -- 172.21.15.99:0/2108241464 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa34402ba00 con 0x7fa3501066c0 2024-03-30T00:27:27.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.618+0000 7fa358000700 1 -- 172.21.15.99:0/2108241464 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa350073cb0 con 0x7fa3380387a0 2024-03-30T00:27:27.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.619+0000 7fa34e000700 1 -- 172.21.15.99:0/2108241464 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fa350073cb0 con 0x7fa3380387a0 2024-03-30T00:27:27.628 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:27.628 INFO:teuthology.orchestra.run.smithi099.stdout:[{"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-03-30T00:27:27.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.622+0000 7fa358000700 1 -- 172.21.15.99:0/2108241464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3380387a0 msgr2=0x7fa33803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.622+0000 7fa358000700 1 --2- 172.21.15.99:0/2108241464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3380387a0 0x7fa33803ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa33c006fd0 tx=0x7fa33c006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.622+0000 7fa358000700 1 -- 172.21.15.99:0/2108241464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3501066c0 msgr2=0x7fa3500730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.622+0000 7fa358000700 1 --2- 172.21.15.99:0/2108241464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3501066c0 0x7fa3500730b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa3440098e0 tx=0x7fa344005710 comp rx=0 tx=0).stop 2024-03-30T00:27:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.622+0000 7fa358000700 1 -- 172.21.15.99:0/2108241464 shutdown_connections 2024-03-30T00:27:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.622+0000 7fa358000700 1 --2- 172.21.15.99:0/2108241464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3380387a0 0x7fa33803ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.622+0000 7fa358000700 1 --2- 172.21.15.99:0/2108241464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3501066c0 0x7fa3500730b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.622+0000 7fa358000700 1 -- 172.21.15.99:0/2108241464 >> 172.21.15.99:0/2108241464 conn(0x7fa3500fc1b0 msgr2=0x7fa350108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.622+0000 7fa358000700 1 -- 172.21.15.99:0/2108241464 shutdown_connections 2024-03-30T00:27:27.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:27.622+0000 7fa358000700 1 -- 172.21.15.99:0/2108241464 wait complete. 2024-03-30T00:27:27.993 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-03-30T00:27:27.993 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-30T00:27:27.994 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-03-30T00:27:28.040 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-30T00:27:28.040 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-30T00:27:28.123 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-03-30T00:27:28.124 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi161 2024-03-30T00:27:28.468 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:27:28.926 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:28 smithi099 ceph-mon[38079]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:27:28.926 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:28 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:29.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.545+0000 7f387f000700 1 -- 172.21.15.99:0/4130470623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3878073150 msgr2=0x7f3878073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:29.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.545+0000 7f387f000700 1 --2- 172.21.15.99:0/4130470623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3878073150 0x7f3878073530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3868009910 tx=0x7f3868009c20 comp rx=0 tx=0).stop 2024-03-30T00:27:29.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.545+0000 7f387f000700 1 -- 172.21.15.99:0/4130470623 shutdown_connections 2024-03-30T00:27:29.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.546+0000 7f387f000700 1 --2- 172.21.15.99:0/4130470623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3878073150 0x7f3878073530 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:29.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.546+0000 7f387f000700 1 -- 172.21.15.99:0/4130470623 >> 172.21.15.99:0/4130470623 conn(0x7f38780fc1b0 msgr2=0x7f38780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:29.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.546+0000 7f387f000700 1 -- 172.21.15.99:0/4130470623 shutdown_connections 2024-03-30T00:27:29.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.546+0000 7f387f000700 1 -- 172.21.15.99:0/4130470623 wait complete. 2024-03-30T00:27:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.547+0000 7f387f000700 1 Processor -- start 2024-03-30T00:27:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.547+0000 7f387f000700 1 -- start start 2024-03-30T00:27:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.547+0000 7f387f000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3878073150 0x7f387819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.547+0000 7f387f000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387819c990 con 0x7f3878073150 2024-03-30T00:27:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.547+0000 7f3877e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3878073150 0x7f387819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.547+0000 7f3877e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3878073150 0x7f387819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47040/0 (socket says 172.21.15.99:47040) 2024-03-30T00:27:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.547+0000 7f3877e00700 1 -- 172.21.15.99:0/758021793 learned_addr learned my addr 172.21.15.99:0/758021793 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.548+0000 7f3877e00700 1 -- 172.21.15.99:0/758021793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38680095f0 con 0x7f3878073150 2024-03-30T00:27:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.548+0000 7f3877e00700 1 --2- 172.21.15.99:0/758021793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3878073150 0x7f387819c450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f38680098e0 tx=0x7f3868005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.548+0000 7f3874c00700 1 -- 172.21.15.99:0/758021793 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f386801d070 con 0x7f3878073150 2024-03-30T00:27:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.548+0000 7f3874c00700 1 -- 172.21.15.99:0/758021793 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3868009e60 con 0x7f3878073150 2024-03-30T00:27:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.548+0000 7f387f000700 1 -- 172.21.15.99:0/758021793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387819cbf0 con 0x7f3878073150 2024-03-30T00:27:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.548+0000 7f3874c00700 1 -- 172.21.15.99:0/758021793 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3868022c10 con 0x7f3878073150 2024-03-30T00:27:29.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.548+0000 7f387f000700 1 -- 172.21.15.99:0/758021793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387819d070 con 0x7f3878073150 2024-03-30T00:27:29.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.549+0000 7f3874c00700 1 -- 172.21.15.99:0/758021793 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f386800b960 con 0x7f3878073150 2024-03-30T00:27:29.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.549+0000 7f387f000700 1 -- 172.21.15.99:0/758021793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3878074890 con 0x7f3878073150 2024-03-30T00:27:29.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.550+0000 7f3874c00700 1 --2- 172.21.15.99:0/758021793 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f385c0387f0 0x7f385c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:29.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.550+0000 7f3874c00700 1 -- 172.21.15.99:0/758021793 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f386804cd30 con 0x7f3878073150 2024-03-30T00:27:29.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.550+0000 7f3877400700 1 --2- 172.21.15.99:0/758021793 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f385c0387f0 0x7f385c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:29.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.551+0000 7f3877400700 1 --2- 172.21.15.99:0/758021793 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f385c0387f0 0x7f385c03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3864006fd0 tx=0x7f3864006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:29.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.555+0000 7f3874c00700 1 -- 172.21.15.99:0/758021793 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f386802ba00 con 0x7f3878073150 2024-03-30T00:27:29.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:29.758+0000 7f387f000700 1 -- 172.21.15.99:0/758021793 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) v1 -- 0x7f3878110e60 con 0x7f385c0387f0 2024-03-30T00:27:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:29 smithi099 ceph-mon[38079]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:31.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:30 smithi099 ceph-mon[38079]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:27:31.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:30 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:31.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:30 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:31.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:30 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:30 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:30 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:27:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:30 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:27:31.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:30 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:27:32.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:31 smithi099 ceph-mon[38079]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-30T00:27:32.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:31 smithi099 ceph-mon[38079]: Deploying cephadm binary to smithi161 2024-03-30T00:27:32.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:31 smithi099 ceph-mon[38079]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:32.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:31 smithi099 ceph-mon[38079]: Updating smithi142:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:27:32.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:32.785+0000 7f3874c00700 1 -- 172.21.15.99:0/758021793 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3878110e60 con 0x7f385c0387f0 2024-03-30T00:27:32.795 INFO:teuthology.orchestra.run.smithi099.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-03-30T00:27:32.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:32.788+0000 7f387f000700 1 -- 172.21.15.99:0/758021793 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f385c0387f0 msgr2=0x7f385c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:32.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:32.788+0000 7f387f000700 1 --2- 172.21.15.99:0/758021793 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f385c0387f0 0x7f385c03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3864006fd0 tx=0x7f3864006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:32.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:32.788+0000 7f387f000700 1 -- 172.21.15.99:0/758021793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3878073150 msgr2=0x7f387819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:32.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:32.788+0000 7f387f000700 1 --2- 172.21.15.99:0/758021793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3878073150 0x7f387819c450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f38680098e0 tx=0x7f3868005710 comp rx=0 tx=0).stop 2024-03-30T00:27:32.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:32.788+0000 7f387f000700 1 -- 172.21.15.99:0/758021793 shutdown_connections 2024-03-30T00:27:32.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:32.788+0000 7f387f000700 1 --2- 172.21.15.99:0/758021793 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f385c0387f0 0x7f385c03acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:32.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:32.788+0000 7f387f000700 1 --2- 172.21.15.99:0/758021793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3878073150 0x7f387819c450 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:32.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:32.788+0000 7f387f000700 1 -- 172.21.15.99:0/758021793 >> 172.21.15.99:0/758021793 conn(0x7f38780fc1b0 msgr2=0x7f38780fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:32.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:32.788+0000 7f387f000700 1 -- 172.21.15.99:0/758021793 shutdown_connections 2024-03-30T00:27:32.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:32.788+0000 7f387f000700 1 -- 172.21.15.99:0/758021793 wait complete. 2024-03-30T00:27:33.141 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:32 smithi099 ceph-mon[38079]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-03-30T00:27:33.141 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:33.171 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-03-30T00:27:33.521 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:27:34.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:34 smithi099 ceph-mon[38079]: Updating smithi142:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T00:27:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:34 smithi099 ceph-mon[38079]: Added host smithi161 2024-03-30T00:27:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:34 smithi099 ceph-mon[38079]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:34 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:34 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:34 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:34 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:27:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:34 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:34.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.565+0000 7f60f8e00700 1 -- 172.21.15.99:0/1541473869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f4106660 msgr2=0x7f60f4106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:34.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.565+0000 7f60f8e00700 1 --2- 172.21.15.99:0/1541473869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f4106660 0x7f60f4106a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009910 tx=0x7f60e4009c20 comp rx=0 tx=0).stop 2024-03-30T00:27:34.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.565+0000 7f60f8e00700 1 -- 172.21.15.99:0/1541473869 shutdown_connections 2024-03-30T00:27:34.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.565+0000 7f60f8e00700 1 --2- 172.21.15.99:0/1541473869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f4106660 0x7f60f4106a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:34.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.565+0000 7f60f8e00700 1 -- 172.21.15.99:0/1541473869 >> 172.21.15.99:0/1541473869 conn(0x7f60f40fc1b0 msgr2=0x7f60f40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:34.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.565+0000 7f60f8e00700 1 -- 172.21.15.99:0/1541473869 shutdown_connections 2024-03-30T00:27:34.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.565+0000 7f60f8e00700 1 -- 172.21.15.99:0/1541473869 wait complete. 2024-03-30T00:27:34.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.566+0000 7f60f8e00700 1 Processor -- start 2024-03-30T00:27:34.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.566+0000 7f60f8e00700 1 -- start start 2024-03-30T00:27:34.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.567+0000 7f60f8e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f4106660 0x7f60f40730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:34.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.567+0000 7f60f8e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f4074f70 con 0x7f60f4106660 2024-03-30T00:27:34.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.567+0000 7f60f1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f4106660 0x7f60f40730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:34.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.567+0000 7f60f1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f4106660 0x7f60f40730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47068/0 (socket says 172.21.15.99:47068) 2024-03-30T00:27:34.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.567+0000 7f60f1e00700 1 -- 172.21.15.99:0/2521475393 learned_addr learned my addr 172.21.15.99:0/2521475393 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:34.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.567+0000 7f60f1e00700 1 -- 172.21.15.99:0/2521475393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e40095f0 con 0x7f60f4106660 2024-03-30T00:27:34.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.567+0000 7f60f1e00700 1 --2- 172.21.15.99:0/2521475393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f4106660 0x7f60f40730b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60e40098e0 tx=0x7f60e4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:34.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.568+0000 7f60eaa00700 1 -- 172.21.15.99:0/2521475393 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60e401d070 con 0x7f60f4106660 2024-03-30T00:27:34.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.568+0000 7f60eaa00700 1 -- 172.21.15.99:0/2521475393 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f60e4009e60 con 0x7f60f4106660 2024-03-30T00:27:34.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.568+0000 7f60f8e00700 1 -- 172.21.15.99:0/2521475393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f40735f0 con 0x7f60f4106660 2024-03-30T00:27:34.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.568+0000 7f60eaa00700 1 -- 172.21.15.99:0/2521475393 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60e4022b30 con 0x7f60f4106660 2024-03-30T00:27:34.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.568+0000 7f60f8e00700 1 -- 172.21.15.99:0/2521475393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f4073a70 con 0x7f60f4106660 2024-03-30T00:27:34.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.569+0000 7f60eaa00700 1 -- 172.21.15.99:0/2521475393 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f60e4022470 con 0x7f60f4106660 2024-03-30T00:27:34.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.569+0000 7f60f8e00700 1 -- 172.21.15.99:0/2521475393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f4101da0 con 0x7f60f4106660 2024-03-30T00:27:34.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.569+0000 7f60eaa00700 1 --2- 172.21.15.99:0/2521475393 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f60d80387f0 0x7f60d803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:34.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.569+0000 7f60eaa00700 1 -- 172.21.15.99:0/2521475393 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f60e404cda0 con 0x7f60f4106660 2024-03-30T00:27:34.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.570+0000 7f60f1400700 1 --2- 172.21.15.99:0/2521475393 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f60d80387f0 0x7f60d803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:34.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.570+0000 7f60f1400700 1 --2- 172.21.15.99:0/2521475393 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f60d80387f0 0x7f60d803acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60e0006fd0 tx=0x7f60e0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:34.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.574+0000 7f60eaa00700 1 -- 172.21.15.99:0/2521475393 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f60e402ba00 con 0x7f60f4106660 2024-03-30T00:27:34.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.781+0000 7f60f8e00700 1 -- 172.21.15.99:0/2521475393 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60f40746a0 con 0x7f60d80387f0 2024-03-30T00:27:34.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.782+0000 7f60eaa00700 1 -- 172.21.15.99:0/2521475393 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f60f40746a0 con 0x7f60d80387f0 2024-03-30T00:27:34.791 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:27:34.791 INFO:teuthology.orchestra.run.smithi099.stdout:[{"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-03-30T00:27:34.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.784+0000 7f60f8e00700 1 -- 172.21.15.99:0/2521475393 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f60d80387f0 msgr2=0x7f60d803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:34.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.784+0000 7f60f8e00700 1 --2- 172.21.15.99:0/2521475393 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f60d80387f0 0x7f60d803acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60e0006fd0 tx=0x7f60e0006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:34.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.784+0000 7f60f8e00700 1 -- 172.21.15.99:0/2521475393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f4106660 msgr2=0x7f60f40730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:34.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.784+0000 7f60f8e00700 1 --2- 172.21.15.99:0/2521475393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f4106660 0x7f60f40730b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f60e40098e0 tx=0x7f60e4005710 comp rx=0 tx=0).stop 2024-03-30T00:27:34.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.785+0000 7f60f8e00700 1 -- 172.21.15.99:0/2521475393 shutdown_connections 2024-03-30T00:27:34.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.785+0000 7f60f8e00700 1 --2- 172.21.15.99:0/2521475393 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f60d80387f0 0x7f60d803acb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:34.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.785+0000 7f60f8e00700 1 --2- 172.21.15.99:0/2521475393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60f4106660 0x7f60f40730b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:34.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.785+0000 7f60f8e00700 1 -- 172.21.15.99:0/2521475393 >> 172.21.15.99:0/2521475393 conn(0x7f60f40fc1b0 msgr2=0x7f60f4108310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:34.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.785+0000 7f60f8e00700 1 -- 172.21.15.99:0/2521475393 shutdown_connections 2024-03-30T00:27:34.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:34.785+0000 7f60f8e00700 1 -- 172.21.15.99:0/2521475393 wait complete. 2024-03-30T00:27:35.158 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-30T00:27:35.158 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd crush tunables default 2024-03-30T00:27:35.505 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:27:35.676 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:35 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:36.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.555+0000 7f3d93e00700 1 -- 172.21.15.99:0/2412476163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d941065c0 msgr2=0x7f3d941069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:36.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.555+0000 7f3d93e00700 1 --2- 172.21.15.99:0/2412476163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d941065c0 0x7f3d941069a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009910 tx=0x7f3d84009c20 comp rx=0 tx=0).stop 2024-03-30T00:27:36.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.555+0000 7f3d93e00700 1 -- 172.21.15.99:0/2412476163 shutdown_connections 2024-03-30T00:27:36.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.555+0000 7f3d93e00700 1 --2- 172.21.15.99:0/2412476163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d941065c0 0x7f3d941069a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:36.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.555+0000 7f3d93e00700 1 -- 172.21.15.99:0/2412476163 >> 172.21.15.99:0/2412476163 conn(0x7f3d940fbfc0 msgr2=0x7f3d940fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:36.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.555+0000 7f3d93e00700 1 -- 172.21.15.99:0/2412476163 shutdown_connections 2024-03-30T00:27:36.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.555+0000 7f3d93e00700 1 -- 172.21.15.99:0/2412476163 wait complete. 2024-03-30T00:27:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.556+0000 7f3d93e00700 1 Processor -- start 2024-03-30T00:27:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.557+0000 7f3d93e00700 1 -- start start 2024-03-30T00:27:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.557+0000 7f3d93e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d941065c0 0x7f3d94197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.557+0000 7f3d93e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d94198320 con 0x7f3d941065c0 2024-03-30T00:27:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.557+0000 7f3d92a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d941065c0 0x7f3d94197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.557+0000 7f3d92a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d941065c0 0x7f3d94197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43056/0 (socket says 172.21.15.99:43056) 2024-03-30T00:27:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.557+0000 7f3d92a00700 1 -- 172.21.15.99:0/2501037907 learned_addr learned my addr 172.21.15.99:0/2501037907 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:27:36.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.558+0000 7f3d92a00700 1 -- 172.21.15.99:0/2501037907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d840095f0 con 0x7f3d941065c0 2024-03-30T00:27:36.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.558+0000 7f3d92a00700 1 --2- 172.21.15.99:0/2501037907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d941065c0 0x7f3d94197de0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d840098e0 tx=0x7f3d84005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:36.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.558+0000 7f3d8b400700 1 -- 172.21.15.99:0/2501037907 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d8401d070 con 0x7f3d941065c0 2024-03-30T00:27:36.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.558+0000 7f3d8b400700 1 -- 172.21.15.99:0/2501037907 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d84009e60 con 0x7f3d941065c0 2024-03-30T00:27:36.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.558+0000 7f3d93e00700 1 -- 172.21.15.99:0/2501037907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d94198580 con 0x7f3d941065c0 2024-03-30T00:27:36.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.558+0000 7f3d8b400700 1 -- 172.21.15.99:0/2501037907 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d84022b30 con 0x7f3d941065c0 2024-03-30T00:27:36.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.558+0000 7f3d93e00700 1 -- 172.21.15.99:0/2501037907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d94198a80 con 0x7f3d941065c0 2024-03-30T00:27:36.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.559+0000 7f3d8b400700 1 -- 172.21.15.99:0/2501037907 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f3d84022470 con 0x7f3d941065c0 2024-03-30T00:27:36.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.559+0000 7f3d93e00700 1 -- 172.21.15.99:0/2501037907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d940fe960 con 0x7f3d941065c0 2024-03-30T00:27:36.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.560+0000 7f3d8b400700 1 --2- 172.21.15.99:0/2501037907 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d780387f0 0x7f3d7803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:36.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.560+0000 7f3d8b400700 1 -- 172.21.15.99:0/2501037907 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3d8404c8b0 con 0x7f3d941065c0 2024-03-30T00:27:36.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.560+0000 7f3d8be00700 1 --2- 172.21.15.99:0/2501037907 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d780387f0 0x7f3d7803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:36.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.561+0000 7f3d8be00700 1 --2- 172.21.15.99:0/2501037907 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d780387f0 0x7f3d7803acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c006fd0 tx=0x7f3d7c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:36.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.565+0000 7f3d8b400700 1 -- 172.21.15.99:0/2501037907 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3d8402b590 con 0x7f3d941065c0 2024-03-30T00:27:36.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:36 smithi099 ceph-mon[38079]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:27:36.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:36 smithi099 ceph-mon[38079]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:36.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:36.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:36.774+0000 7f3d93e00700 1 -- 172.21.15.99:0/2501037907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3d94198a80 con 0x7f3d941065c0 2024-03-30T00:27:37.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:37.406+0000 7f3d8b400700 1 -- 172.21.15.99:0/2501037907 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3d94198a80 con 0x7f3d941065c0 2024-03-30T00:27:37.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:37.409+0000 7f3d93e00700 1 -- 172.21.15.99:0/2501037907 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d780387f0 msgr2=0x7f3d7803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:37.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:37.409+0000 7f3d93e00700 1 --2- 172.21.15.99:0/2501037907 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d780387f0 0x7f3d7803acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c006fd0 tx=0x7f3d7c006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:37.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:37.409+0000 7f3d93e00700 1 -- 172.21.15.99:0/2501037907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d941065c0 msgr2=0x7f3d94197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:37.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:37.409+0000 7f3d93e00700 1 --2- 172.21.15.99:0/2501037907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d941065c0 0x7f3d94197de0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d840098e0 tx=0x7f3d84005710 comp rx=0 tx=0).stop 2024-03-30T00:27:37.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:37.409+0000 7f3d93e00700 1 -- 172.21.15.99:0/2501037907 shutdown_connections 2024-03-30T00:27:37.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:37.409+0000 7f3d93e00700 1 --2- 172.21.15.99:0/2501037907 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3d780387f0 0x7f3d7803acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:37.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:37.409+0000 7f3d93e00700 1 --2- 172.21.15.99:0/2501037907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d941065c0 0x7f3d94197de0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:37.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:37.409+0000 7f3d93e00700 1 -- 172.21.15.99:0/2501037907 >> 172.21.15.99:0/2501037907 conn(0x7f3d940fbfc0 msgr2=0x7f3d94072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:37.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:37.409+0000 7f3d93e00700 1 -- 172.21.15.99:0/2501037907 shutdown_connections 2024-03-30T00:27:37.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:27:37.409+0000 7f3d93e00700 1 -- 172.21.15.99:0/2501037907 wait complete. 2024-03-30T00:27:37.420 INFO:teuthology.orchestra.run.smithi099.stderr:adjusted tunables profile to default 2024-03-30T00:27:37.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:37 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2501037907' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-30T00:27:37.770 INFO:tasks.cephadm:Adding mon.a on smithi099 2024-03-30T00:27:37.770 INFO:tasks.cephadm:Adding mon.b on smithi142 2024-03-30T00:27:37.770 INFO:tasks.cephadm:Adding mon.c on smithi161 2024-03-30T00:27:37.770 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch apply mon '3;smithi099:172.21.15.99=a;smithi142:172.21.15.142=b;smithi161:172.21.15.161=c' 2024-03-30T00:27:38.140 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-30T00:27:38.244 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-30T00:27:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:38 smithi099 ceph-mon[38079]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:38 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2501037907' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-30T00:27:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:38 smithi099 ceph-mon[38079]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:27:39.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.596+0000 7fe6eae00700 1 -- 172.21.15.161:0/4174496539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6e4100580 msgr2=0x7fe6e41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:39.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.596+0000 7fe6eae00700 1 --2- 172.21.15.161:0/4174496539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6e4100580 0x7fe6e41009a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009910 tx=0x7fe6d8009c20 comp rx=0 tx=0).stop 2024-03-30T00:27:39.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.596+0000 7fe6eae00700 1 -- 172.21.15.161:0/4174496539 shutdown_connections 2024-03-30T00:27:39.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.596+0000 7fe6eae00700 1 --2- 172.21.15.161:0/4174496539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6e4100580 0x7fe6e41009a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:39.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.596+0000 7fe6eae00700 1 -- 172.21.15.161:0/4174496539 >> 172.21.15.161:0/4174496539 conn(0x7fe6e40fbb20 msgr2=0x7fe6e40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:39.605 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.596+0000 7fe6eae00700 1 -- 172.21.15.161:0/4174496539 shutdown_connections 2024-03-30T00:27:39.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.597+0000 7fe6eae00700 1 -- 172.21.15.161:0/4174496539 wait complete. 2024-03-30T00:27:39.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.597+0000 7fe6eae00700 1 Processor -- start 2024-03-30T00:27:39.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.598+0000 7fe6eae00700 1 -- start start 2024-03-30T00:27:39.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.598+0000 7fe6eae00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6e4100580 0x7fe6e4195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:39.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.598+0000 7fe6eae00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e4195e80 con 0x7fe6e4100580 2024-03-30T00:27:39.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.599+0000 7fe6e9a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6e4100580 0x7fe6e4195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:39.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.599+0000 7fe6e9a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6e4100580 0x7fe6e4195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.161:47242/0 (socket says 172.21.15.161:47242) 2024-03-30T00:27:39.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.599+0000 7fe6e9a00700 1 -- 172.21.15.161:0/89093836 learned_addr learned my addr 172.21.15.161:0/89093836 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-30T00:27:39.607 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.599+0000 7fe6e9a00700 1 -- 172.21.15.161:0/89093836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d80095f0 con 0x7fe6e4100580 2024-03-30T00:27:39.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.599+0000 7fe6e9a00700 1 --2- 172.21.15.161:0/89093836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6e4100580 0x7fe6e4195940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80098e0 tx=0x7fe6d8005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:39.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.599+0000 7fe6e2000700 1 -- 172.21.15.161:0/89093836 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6d800ba90 con 0x7fe6e4100580 2024-03-30T00:27:39.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.600+0000 7fe6e2000700 1 -- 172.21.15.161:0/89093836 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6d800bbf0 con 0x7fe6e4100580 2024-03-30T00:27:39.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.600+0000 7fe6eae00700 1 -- 172.21.15.161:0/89093836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e4196080 con 0x7fe6e4100580 2024-03-30T00:27:39.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.600+0000 7fe6e2000700 1 -- 172.21.15.161:0/89093836 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6d800fe70 con 0x7fe6e4100580 2024-03-30T00:27:39.608 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.600+0000 7fe6eae00700 1 -- 172.21.15.161:0/89093836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e4196520 con 0x7fe6e4100580 2024-03-30T00:27:39.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.601+0000 7fe6e2000700 1 -- 172.21.15.161:0/89093836 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fe6d800f460 con 0x7fe6e4100580 2024-03-30T00:27:39.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.601+0000 7fe6eae00700 1 -- 172.21.15.161:0/89093836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e418f150 con 0x7fe6e4100580 2024-03-30T00:27:39.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.601+0000 7fe6e2000700 1 --2- 172.21.15.161:0/89093836 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6cc038840 0x7fe6cc03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:39.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.601+0000 7fe6e2000700 1 -- 172.21.15.161:0/89093836 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe6d804c980 con 0x7fe6e4100580 2024-03-30T00:27:39.609 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.602+0000 7fe6e9000700 1 --2- 172.21.15.161:0/89093836 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6cc038840 0x7fe6cc03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:39.610 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.602+0000 7fe6e9000700 1 --2- 172.21.15.161:0/89093836 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6cc038840 0x7fe6cc03ad00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0006fd0 tx=0x7fe6d0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:39.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.607+0000 7fe6e2000700 1 -- 172.21.15.161:0/89093836 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe6d802ad30 con 0x7fe6e4100580 2024-03-30T00:27:39.862 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.854+0000 7fe6eae00700 1 -- 172.21.15.161:0/89093836 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi142:172.21.15.142=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6e40612b0 con 0x7fe6cc038840 2024-03-30T00:27:39.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.864+0000 7fe6e2000700 1 -- 172.21.15.161:0/89093836 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe6e40612b0 con 0x7fe6cc038840 2024-03-30T00:27:39.872 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mon update... 2024-03-30T00:27:39.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.866+0000 7fe6eae00700 1 -- 172.21.15.161:0/89093836 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6cc038840 msgr2=0x7fe6cc03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:39.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.866+0000 7fe6eae00700 1 --2- 172.21.15.161:0/89093836 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6cc038840 0x7fe6cc03ad00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0006fd0 tx=0x7fe6d0006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:39.874 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.866+0000 7fe6eae00700 1 -- 172.21.15.161:0/89093836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6e4100580 msgr2=0x7fe6e4195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:39.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.866+0000 7fe6eae00700 1 --2- 172.21.15.161:0/89093836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6e4100580 0x7fe6e4195940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80098e0 tx=0x7fe6d8005810 comp rx=0 tx=0).stop 2024-03-30T00:27:39.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.866+0000 7fe6eae00700 1 -- 172.21.15.161:0/89093836 shutdown_connections 2024-03-30T00:27:39.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.866+0000 7fe6eae00700 1 --2- 172.21.15.161:0/89093836 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6cc038840 0x7fe6cc03ad00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:39.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.866+0000 7fe6eae00700 1 --2- 172.21.15.161:0/89093836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6e4100580 0x7fe6e4195940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:39.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.866+0000 7fe6eae00700 1 -- 172.21.15.161:0/89093836 >> 172.21.15.161:0/89093836 conn(0x7fe6e40fbb20 msgr2=0x7fe6e40fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:39.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.866+0000 7fe6eae00700 1 -- 172.21.15.161:0/89093836 shutdown_connections 2024-03-30T00:27:39.875 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:39.866+0000 7fe6eae00700 1 -- 172.21.15.161:0/89093836 wait complete. 2024-03-30T00:27:40.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:39 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:40.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:39 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:40.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:39 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:40.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:39 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:40.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:39 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:27:40.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:39 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:27:40.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:39 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:27:40.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:39 smithi099 ceph-mon[38079]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-30T00:27:40.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:39 smithi099 ceph-mon[38079]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:40.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:39 smithi099 ceph-mon[38079]: Updating smithi161:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:27:40.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:39 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:40.300 DEBUG:teuthology.orchestra.run.smithi142:mon.b> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.b.service 2024-03-30T00:27:40.305 DEBUG:teuthology.orchestra.run.smithi161:mon.c> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.c.service 2024-03-30T00:27:40.309 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-30T00:27:40.309 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-30T00:27:40.378 INFO:journalctl@ceph.mon.c.smithi161.stdout:-- Logs begin at Sat 2024-03-30 00:10:26 UTC. -- 2024-03-30T00:27:40.378 INFO:journalctl@ceph.mon.b.smithi142.stdout:-- Logs begin at Sat 2024-03-30 00:10:40 UTC. -- 2024-03-30T00:27:40.705 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:27:41.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:40 smithi099 ceph-mon[38079]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi142:172.21.15.142=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:27:41.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:40 smithi099 ceph-mon[38079]: Saving service mon spec with placement smithi099:172.21.15.99=a;smithi142:172.21.15.142=b;smithi161:172.21.15.161=c;count:3 2024-03-30T00:27:41.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:40 smithi099 ceph-mon[38079]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-03-30T00:27:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.869+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3846382243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ec01066c0 msgr2=0x7f9ec0106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:41.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.869+0000 7f9ec5a00700 1 --2- 172.21.15.161:0/3846382243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ec01066c0 0x7f9ec0106aa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009910 tx=0x7f9eb0009c20 comp rx=0 tx=0).stop 2024-03-30T00:27:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.869+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3846382243 shutdown_connections 2024-03-30T00:27:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.869+0000 7f9ec5a00700 1 --2- 172.21.15.161:0/3846382243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ec01066c0 0x7f9ec0106aa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.869+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3846382243 >> 172.21.15.161:0/3846382243 conn(0x7f9ec00fc1b0 msgr2=0x7f9ec00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.869+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3846382243 shutdown_connections 2024-03-30T00:27:41.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.869+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3846382243 wait complete. 2024-03-30T00:27:41.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.870+0000 7f9ec5a00700 1 Processor -- start 2024-03-30T00:27:41.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.870+0000 7f9ec5a00700 1 -- start start 2024-03-30T00:27:41.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.871+0000 7f9ec5a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ec01066c0 0x7f9ec00730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:41.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.871+0000 7f9ec5a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec0074f70 con 0x7f9ec01066c0 2024-03-30T00:27:41.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.871+0000 7f9ebea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ec01066c0 0x7f9ec00730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:41.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.871+0000 7f9ebea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ec01066c0 0x7f9ec00730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.161:47254/0 (socket says 172.21.15.161:47254) 2024-03-30T00:27:41.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.871+0000 7f9ebea00700 1 -- 172.21.15.161:0/3593147641 learned_addr learned my addr 172.21.15.161:0/3593147641 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-30T00:27:41.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.872+0000 7f9ebea00700 1 -- 172.21.15.161:0/3593147641 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb00095f0 con 0x7f9ec01066c0 2024-03-30T00:27:41.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.872+0000 7f9ebea00700 1 --2- 172.21.15.161:0/3593147641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ec01066c0 0x7f9ec00730b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00098e0 tx=0x7f9eb0005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:41.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.872+0000 7f9eb7400700 1 -- 172.21.15.161:0/3593147641 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9eb001d070 con 0x7f9ec01066c0 2024-03-30T00:27:41.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.872+0000 7f9eb7400700 1 -- 172.21.15.161:0/3593147641 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9eb0009e60 con 0x7f9ec01066c0 2024-03-30T00:27:41.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.872+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3593147641 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec00735f0 con 0x7f9ec01066c0 2024-03-30T00:27:41.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.872+0000 7f9eb7400700 1 -- 172.21.15.161:0/3593147641 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9eb0022b30 con 0x7f9ec01066c0 2024-03-30T00:27:41.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.872+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3593147641 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec0073a70 con 0x7f9ec01066c0 2024-03-30T00:27:41.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.874+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3593147641 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec0101e00 con 0x7f9ec01066c0 2024-03-30T00:27:41.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.874+0000 7f9eb7400700 1 -- 172.21.15.161:0/3593147641 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f9eb0022470 con 0x7f9ec01066c0 2024-03-30T00:27:41.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.874+0000 7f9eb7400700 1 --2- 172.21.15.161:0/3593147641 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea40387a0 0x7f9ea403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:41.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.874+0000 7f9eb7400700 1 -- 172.21.15.161:0/3593147641 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9eb004cd40 con 0x7f9ec01066c0 2024-03-30T00:27:41.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.874+0000 7f9eb7e00700 1 --2- 172.21.15.161:0/3593147641 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea40387a0 0x7f9ea403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:41.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.875+0000 7f9eb7e00700 1 --2- 172.21.15.161:0/3593147641 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea40387a0 0x7f9ea403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8006fd0 tx=0x7f9ea8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:41.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:41.880+0000 7f9eb7400700 1 -- 172.21.15.161:0/3593147641 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9eb002ba00 con 0x7f9ec01066c0 2024-03-30T00:27:42.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.168+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3593147641 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9ec0073cb0 con 0x7f9ec01066c0 2024-03-30T00:27:42.177 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.169+0000 7f9eb7400700 1 -- 172.21.15.161:0/3593147641 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f9eb0027020 con 0x7f9ec01066c0 2024-03-30T00:27:42.178 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:27:42.178 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","modified":"2024-03-30T00:26:11.889664Z","created":"2024-03-30T00:26:11.889664Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-30T00:27:42.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.172+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3593147641 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea40387a0 msgr2=0x7f9ea403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:42.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.172+0000 7f9ec5a00700 1 --2- 172.21.15.161:0/3593147641 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea40387a0 0x7f9ea403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8006fd0 tx=0x7f9ea8006e40 comp rx=0 tx=0).stop 2024-03-30T00:27:42.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.172+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3593147641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ec01066c0 msgr2=0x7f9ec00730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:42.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.172+0000 7f9ec5a00700 1 --2- 172.21.15.161:0/3593147641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ec01066c0 0x7f9ec00730b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00098e0 tx=0x7f9eb0005710 comp rx=0 tx=0).stop 2024-03-30T00:27:42.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.172+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3593147641 shutdown_connections 2024-03-30T00:27:42.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.172+0000 7f9ec5a00700 1 --2- 172.21.15.161:0/3593147641 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea40387a0 0x7f9ea403ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:42.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.172+0000 7f9ec5a00700 1 --2- 172.21.15.161:0/3593147641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ec01066c0 0x7f9ec00730b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:42.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.172+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3593147641 >> 172.21.15.161:0/3593147641 conn(0x7f9ec00fc1b0 msgr2=0x7f9ec0108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:42.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.172+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3593147641 shutdown_connections 2024-03-30T00:27:42.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:42.172+0000 7f9ec5a00700 1 -- 172.21.15.161:0/3593147641 wait complete. 2024-03-30T00:27:42.181 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 1 2024-03-30T00:27:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:41 smithi099 ceph-mon[38079]: Updating smithi161:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T00:27:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:41 smithi099 ceph-mon[38079]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:41 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:41 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:42.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:41 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:42.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:41 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:27:42.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:41 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:27:42.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:41 smithi099 ceph-mon[38079]: Deploying daemon mon.c on smithi161 2024-03-30T00:27:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:42 smithi099 ceph-mon[38079]: from='client.? 172.21.15.161:0/3593147641' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T00:27:43.763 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-30T00:27:43.763 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-30T00:27:44.235 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:27:44.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:43 smithi099 ceph-mon[38079]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:45.450 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 systemd[1]: Started Ceph mon.c for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:27:45.450 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: set uid:gid to 167:167 (ceph:ceph) 2024-03-30T00:27:45.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable), process ceph-mon, pid 7 2024-03-30T00:27:45.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: pidfile_write: ignore empty --pid-file 2024-03-30T00:27:45.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: load: jerasure load: lrc 2024-03-30T00:27:45.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: RocksDB version: 7.9.2 2024-03-30T00:27:45.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Git sha 0 2024-03-30T00:27:45.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Compile date 2024-03-29 11:49:16 2024-03-30T00:27:45.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: DB SUMMARY 2024-03-30T00:27:45.722 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: DB Session ID: RKJ4QC4YLVVFGKNRDV9N 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: CURRENT file: CURRENT 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: IDENTITY file: IDENTITY 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.error_if_exists: 0 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.create_if_missing: 0 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.paranoid_checks: 1 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.env: 0x56487bd05720 2024-03-30T00:27:45.723 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.fs: PosixFileSystem 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.info_log: 0x56487deed520 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_file_opening_threads: 16 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.statistics: (nil) 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.use_fsync: 0 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_log_file_size: 0 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.keep_log_file_num: 1000 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.recycle_log_file_num: 0 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.allow_fallocate: 1 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.allow_mmap_reads: 0 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.allow_mmap_writes: 0 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.use_direct_reads: 0 2024-03-30T00:27:45.724 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-30T00:27:45.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.create_missing_column_families: 0 2024-03-30T00:27:45.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.db_log_dir: 2024-03-30T00:27:45.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.wal_dir: 2024-03-30T00:27:45.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-30T00:27:45.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-30T00:27:45.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-30T00:27:45.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-30T00:27:45.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-30T00:27:45.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-30T00:27:45.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.advise_random_on_open: 1 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.db_write_buffer_size: 0 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.write_buffer_manager: 0x56487d184320 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.rate_limiter: (nil) 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.wal_recovery_mode: 2 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.enable_thread_tracking: 0 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.enable_pipelined_write: 0 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.unordered_write: 0 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-30T00:27:45.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.row_cache: None 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.wal_filter: None 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.allow_ingest_behind: 0 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.two_write_queues: 0 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.manual_wal_flush: 0 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.wal_compression: 0 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.atomic_flush: 0 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-30T00:27:45.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.log_readahead_size: 0 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.best_efforts_recovery: 0 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.allow_data_in_errors: 0 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.db_host_id: __hostname__ 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_background_jobs: 2 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_background_compactions: -1 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_subcompactions: 1 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-30T00:27:45.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_total_wal_size: 0 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_open_files: -1 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bytes_per_sync: 0 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_readahead_size: 0 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_background_flushes: -1 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Compression algorithms supported: 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: kZSTD supported: 0 2024-03-30T00:27:45.729 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: kXpressCompression supported: 0 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: kLZ4HCCompression supported: 1 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: kZlibCompression supported: 1 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: kSnappyCompression supported: 1 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: kLZ4Compression supported: 1 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: kBZip2Compression supported: 0 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.merge_operator: 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_filter: None 2024-03-30T00:27:45.730 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_filter_factory: None 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.sst_partitioner_factory: None 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56487deed620) 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout: cache_index_and_filter_blocks: 1 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout: pin_top_level_index_and_filter: 1 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_type: 0 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout: data_block_index_type: 0 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_shortening: 1 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout: checksum: 4 2024-03-30T00:27:45.731 INFO:journalctl@ceph.mon.c.smithi161.stdout: no_block_cache: 0 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache: 0x56487d2051f0 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_name: BinnedLRUCache 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_options: 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: capacity : 536870912 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: num_shard_bits : 4 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: strict_capacity_limit : 0 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: high_pri_pool_ratio: 0.000 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_compressed: (nil) 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: persistent_cache: (nil) 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_size: 4096 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_size_deviation: 10 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_restart_interval: 16 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_block_restart_interval: 1 2024-03-30T00:27:45.732 INFO:journalctl@ceph.mon.c.smithi161.stdout: metadata_block_size: 4096 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: partition_filters: 0 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: use_delta_encoding: 1 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: filter_policy: bloomfilter 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: whole_key_filtering: 1 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: verify_compression: 0 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: read_amp_bytes_per_bit: 0 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: format_version: 5 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: enable_index_compression: 1 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_align: 0 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: max_auto_readahead_size: 262144 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: prepopulate_block_cache: 0 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: initial_auto_readahead_size: 8192 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout: num_file_reads_for_auto_readahead: 2 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.write_buffer_size: 33554432 2024-03-30T00:27:45.733 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_write_buffer_number: 2 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compression: NoCompression 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bottommost_compression: Disabled 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.prefix_extractor: nullptr 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.num_levels: 7 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-30T00:27:45.734 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compression_opts.level: 32767 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compression_opts.strategy: 0 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compression_opts.enabled: false 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-30T00:27:45.735 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.target_file_size_base: 67108864 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-30T00:27:45.736 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.arena_block_size: 1048576 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.disable_auto_compactions: 0 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-30T00:27:45.737 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.table_properties_collectors: 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.inplace_update_support: 0 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.bloom_locality: 0 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.max_successive_merges: 0 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.paranoid_file_checks: 0 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.force_consistency_checks: 1 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.report_bg_io_stats: 0 2024-03-30T00:27:45.738 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.ttl: 2592000 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.enable_blob_files: false 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.min_blob_size: 0 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.blob_file_size: 268435456 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.blob_file_starting_level: 0 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-30T00:27:45.739 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 99461e75-7dac-465e-99f5-1986d85c39e5 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711758465453957, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711758465455403, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711758465, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "99461e75-7dac-465e-99f5-1986d85c39e5", "db_session_id": "RKJ4QC4YLVVFGKNRDV9N", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711758465455778, "job": 1, "event": "recovery_finished"} 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56487d2a2000 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: DB pointer 0x56487d290000 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-30T00:27:45.740 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** DB Stats ** 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** Compaction Stats [default] ** 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:27:45.741 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** Compaction Stats [default] ** 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-30T00:27:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative compaction: 0.00 GB write, 0.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval compaction: 0.00 GB write, 0.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout: Block cache BinnedLRUCache@0x56487d2051f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: using public_addr v2:172.21.15.161:0/0 -> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: starting mon.c rank -1 at public addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] at bind addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(???) e0 preinit fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).mds e1 new map 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-30T00:27:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout: e1 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout: legacy client fscid: -1 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout: No filesystems configured 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T00:27:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2501037907' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2501037907' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:27:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: Updating smithi161:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi142:172.21.15.142=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: Saving service mon spec with placement smithi099:172.21.15.99=a;smithi142:172.21.15.142=b;smithi161:172.21.15.161=c;count:3 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: Updating smithi161:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:45.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:45.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:27:45.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:27:45.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: Deploying daemon mon.c on smithi161 2024-03-30T00:27:45.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: from='client.? 172.21.15.161:0/3593147641' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T00:27:45.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:45.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:45 smithi161 ceph-mon[42221]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-30T00:27:50.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:27:50.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: mon.a calling monitor election 2024-03-30T00:27:50.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:50.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: mon.c calling monitor election 2024-03-30T00:27:50.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:50.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T00:27:50.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: monmap e2: 2 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T00:27:50.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: fsmap 2024-03-30T00:27:50.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:27:50.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: mgrmap e13: x(active, since 41s) 2024-03-30T00:27:50.973 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: overall HEALTH_OK 2024-03-30T00:27:50.973 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:50.973 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:50.973 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:50.973 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:27:50.973 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:27:50.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:27:50.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: mon.a calling monitor election 2024-03-30T00:27:50.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:50.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: mon.c calling monitor election 2024-03-30T00:27:50.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:50.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:50.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T00:27:50.990 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: monmap e2: 2 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T00:27:50.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: fsmap 2024-03-30T00:27:50.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:27:50.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: mgrmap e13: x(active, since 41s) 2024-03-30T00:27:50.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: overall HEALTH_OK 2024-03-30T00:27:50.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:50.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:50.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:50.991 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:27:50.992 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:27:51.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.624+0000 7f58d2000700 1 -- 172.21.15.161:0/606740269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58b0005c90 msgr2=0x7f58b00080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:51.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.624+0000 7f58d2000700 1 --2- 172.21.15.161:0/606740269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58b0005c90 0x7f58b00080c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f58cc101c80 tx=0x7f58bc00ff20 comp rx=0 tx=0).stop 2024-03-30T00:27:51.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.624+0000 7f58d2000700 1 -- 172.21.15.161:0/606740269 shutdown_connections 2024-03-30T00:27:51.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.624+0000 7f58d2000700 1 --2- 172.21.15.161:0/606740269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58b0005c90 0x7f58b00080c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:51.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.624+0000 7f58d2000700 1 -- 172.21.15.161:0/606740269 >> 172.21.15.161:0/606740269 conn(0x7f58cc0fc300 msgr2=0x7f58cc0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:51.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.624+0000 7f58d2000700 1 -- 172.21.15.161:0/606740269 shutdown_connections 2024-03-30T00:27:51.633 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.624+0000 7f58d2000700 1 -- 172.21.15.161:0/606740269 wait complete. 2024-03-30T00:27:51.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.625+0000 7f58d2000700 1 Processor -- start 2024-03-30T00:27:51.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.625+0000 7f58d2000700 1 -- start start 2024-03-30T00:27:51.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.626+0000 7f58d2000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b0005c90 0x7f58cc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:51.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.626+0000 7f58d2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58cc198820 0x7f58cc19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:51.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.626+0000 7f58d2000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc198db0 con 0x7f58cc198820 2024-03-30T00:27:51.634 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.626+0000 7f58d2000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc198f20 con 0x7f58b0005c90 2024-03-30T00:27:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.626+0000 7f58caa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58cc198820 0x7f58cc19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.626+0000 7f58caa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58cc198820 0x7f58cc19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.161:33448/0 (socket says 172.21.15.161:33448) 2024-03-30T00:27:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.626+0000 7f58caa00700 1 -- 172.21.15.161:0/4155886575 learned_addr learned my addr 172.21.15.161:0/4155886575 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-30T00:27:51.636 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.627+0000 7f58cb400700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b0005c90 0x7f58cc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.627+0000 7f58cb400700 1 -- 172.21.15.161:0/4155886575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b0005c90 msgr2=0x7f58cc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.627+0000 7f58cb400700 1 -- 172.21.15.161:0/4155886575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b0005c90 msgr2=0x7f58cc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.627+0000 7f58cb400700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b0005c90 0x7f58cc1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.627+0000 7f58caa00700 1 -- 172.21.15.161:0/4155886575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b0005c90 msgr2=0x7f58cc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.627+0000 7f58cb400700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b0005c90 0x7f58cc1982e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.627+0000 7f58caa00700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b0005c90 0x7f58cc1982e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.627+0000 7f58caa00700 1 -- 172.21.15.161:0/4155886575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc00f970 con 0x7f58cc198820 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.627+0000 7f58caa00700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58cc198820 0x7f58cc19cbd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58b800b700 tx=0x7f58b800bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.628+0000 7f58c3e00700 1 -- 172.21.15.161:0/4155886575 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f58b8011630 con 0x7f58cc198820 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.628+0000 7f58d2000700 1 -- 172.21.15.161:0/4155886575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58cc19d170 con 0x7f58cc198820 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.628+0000 7f58c3e00700 1 -- 172.21.15.161:0/4155886575 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58b8011c20 con 0x7f58cc198820 2024-03-30T00:27:51.637 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.628+0000 7f58c3e00700 1 -- 172.21.15.161:0/4155886575 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f58b8010c60 con 0x7f58cc198820 2024-03-30T00:27:51.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.628+0000 7f58d2000700 1 -- 172.21.15.161:0/4155886575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58cc19d6c0 con 0x7f58cc198820 2024-03-30T00:27:51.638 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.629+0000 7f58c3e00700 1 -- 172.21.15.161:0/4155886575 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f58b801a400 con 0x7f58cc198820 2024-03-30T00:27:51.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.629+0000 7f58d2000700 1 -- 172.21.15.161:0/4155886575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58cc104aa0 con 0x7f58cc198820 2024-03-30T00:27:51.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.630+0000 7f58c3e00700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58ac0385e0 0x7f58ac03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:51.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.630+0000 7f58c3e00700 1 -- 172.21.15.161:0/4155886575 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f58b804afd0 con 0x7f58cc198820 2024-03-30T00:27:51.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.630+0000 7f58cb400700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58ac0385e0 0x7f58ac03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:51.639 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.631+0000 7f58cb400700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58ac0385e0 0x7f58ac03aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58bc005950 tx=0x7f58bc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:51.644 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.636+0000 7f58c3e00700 1 -- 172.21.15.161:0/4155886575 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f58b8017680 con 0x7f58cc198820 2024-03-30T00:27:51.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.919+0000 7f58d2000700 1 -- 172.21.15.161:0/4155886575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f58cc19da40 con 0x7f58cc198820 2024-03-30T00:27:51.927 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.919+0000 7f58c3e00700 1 -- 172.21.15.161:0/4155886575 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f58cc19da40 con 0x7f58cc198820 2024-03-30T00:27:51.928 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:27:51.928 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","modified":"2024-03-30T00:27:45.498652Z","created":"2024-03-30T00:26:11.889664Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-30T00:27:51.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.922+0000 7f58d2000700 1 -- 172.21.15.161:0/4155886575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58ac0385e0 msgr2=0x7f58ac03aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:51.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.922+0000 7f58d2000700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58ac0385e0 0x7f58ac03aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58bc005950 tx=0x7f58bc0058e0 comp rx=0 tx=0).stop 2024-03-30T00:27:51.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.922+0000 7f58d2000700 1 -- 172.21.15.161:0/4155886575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58cc198820 msgr2=0x7f58cc19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:51.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.922+0000 7f58d2000700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58cc198820 0x7f58cc19cbd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58b800b700 tx=0x7f58b800bac0 comp rx=0 tx=0).stop 2024-03-30T00:27:51.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.922+0000 7f58d2000700 1 -- 172.21.15.161:0/4155886575 shutdown_connections 2024-03-30T00:27:51.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.922+0000 7f58d2000700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58ac0385e0 0x7f58ac03aaa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:51.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.922+0000 7f58d2000700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58b0005c90 0x7f58cc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:51.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.922+0000 7f58d2000700 1 --2- 172.21.15.161:0/4155886575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58cc198820 0x7f58cc19cbd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:51.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.922+0000 7f58d2000700 1 -- 172.21.15.161:0/4155886575 >> 172.21.15.161:0/4155886575 conn(0x7f58cc0fc300 msgr2=0x7f58cc105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:51.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.922+0000 7f58d2000700 1 -- 172.21.15.161:0/4155886575 shutdown_connections 2024-03-30T00:27:51.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:51.922+0000 7f58d2000700 1 -- 172.21.15.161:0/4155886575 wait complete. 2024-03-30T00:27:51.931 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-03-30T00:27:51.952 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:51 smithi161 ceph-mon[42221]: Deploying daemon mon.b on smithi142 2024-03-30T00:27:51.952 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:51 smithi161 ceph-mon[42221]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:51.952 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:51 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:51.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:51 smithi099 ceph-mon[38079]: Deploying daemon mon.b on smithi142 2024-03-30T00:27:51.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:51 smithi099 ceph-mon[38079]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:51.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:51 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:52.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:52 smithi161 ceph-mon[42221]: from='client.? 172.21.15.161:0/4155886575' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T00:27:52.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:52 smithi099 ceph-mon[38079]: from='client.? 172.21.15.161:0/4155886575' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T00:27:52.988 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:27:52 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:27:52.498+0000 7f2e6f200700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-30T00:27:53.340 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-30T00:27:53.340 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-30T00:27:53.550 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 systemd[1]: Starting Ceph mon.b for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:27:53.699 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:27:53.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:53 smithi161 ceph-mon[42221]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:53.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 bash[41740]: b775c6a430406a541c9421f46c601e13c1756e5bd9b61def85f4f5820c68c602 2024-03-30T00:27:53.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 systemd[1]: Started Ceph mon.b for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:27:53.939 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: set uid:gid to 167:167 (ceph:ceph) 2024-03-30T00:27:53.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:53 smithi099 ceph-mon[38079]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:54.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable), process ceph-mon, pid 7 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: pidfile_write: ignore empty --pid-file 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: load: jerasure load: lrc 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: RocksDB version: 7.9.2 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Git sha 0 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Compile date 2024-03-29 11:49:16 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: DB SUMMARY 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: DB Session ID: T0CL9XAGT18GJXFG9FBE 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: CURRENT file: CURRENT 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: IDENTITY file: IDENTITY 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-30T00:27:54.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.error_if_exists: 0 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.create_if_missing: 0 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.paranoid_checks: 1 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.env: 0x56120586d720 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.fs: PosixFileSystem 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.info_log: 0x561208cc7520 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_file_opening_threads: 16 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.statistics: (nil) 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.use_fsync: 0 2024-03-30T00:27:54.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_log_file_size: 0 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.keep_log_file_num: 1000 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.recycle_log_file_num: 0 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.allow_fallocate: 1 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.allow_mmap_reads: 0 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.allow_mmap_writes: 0 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.use_direct_reads: 0 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.create_missing_column_families: 0 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.db_log_dir: 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.wal_dir: 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-30T00:27:54.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.advise_random_on_open: 1 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.db_write_buffer_size: 0 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.write_buffer_manager: 0x561207f5e320 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.rate_limiter: (nil) 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.wal_recovery_mode: 2 2024-03-30T00:27:54.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.enable_thread_tracking: 0 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.enable_pipelined_write: 0 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.unordered_write: 0 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.row_cache: None 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.wal_filter: None 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.allow_ingest_behind: 0 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.two_write_queues: 0 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.manual_wal_flush: 0 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.wal_compression: 0 2024-03-30T00:27:54.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.atomic_flush: 0 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.log_readahead_size: 0 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.best_efforts_recovery: 0 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.allow_data_in_errors: 0 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.db_host_id: __hostname__ 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_background_jobs: 2 2024-03-30T00:27:54.324 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_background_compactions: -1 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_subcompactions: 1 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_total_wal_size: 0 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_open_files: -1 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bytes_per_sync: 0 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_readahead_size: 0 2024-03-30T00:27:54.325 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_background_flushes: -1 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Compression algorithms supported: 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: kZSTD supported: 0 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: kXpressCompression supported: 0 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: kLZ4HCCompression supported: 1 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: kZlibCompression supported: 1 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: kSnappyCompression supported: 1 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: kLZ4Compression supported: 1 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: kBZip2Compression supported: 0 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-30T00:27:54.326 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.merge_operator: 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_filter: None 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_filter_factory: None 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.sst_partitioner_factory: None 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561208cc7620) 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks: 1 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_top_level_index_and_filter: 1 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_type: 0 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_index_type: 0 2024-03-30T00:27:54.327 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_shortening: 1 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: checksum: 4 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: no_block_cache: 0 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache: 0x561207fdf350 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_name: BinnedLRUCache 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_options: 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: capacity : 536870912 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_shard_bits : 4 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: strict_capacity_limit : 0 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: high_pri_pool_ratio: 0.000 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_compressed: (nil) 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: persistent_cache: (nil) 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size: 4096 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size_deviation: 10 2024-03-30T00:27:54.328 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_restart_interval: 16 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_block_restart_interval: 1 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: metadata_block_size: 4096 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: partition_filters: 0 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: use_delta_encoding: 1 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: filter_policy: bloomfilter 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: whole_key_filtering: 1 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: verify_compression: 0 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: read_amp_bytes_per_bit: 0 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: format_version: 5 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_index_compression: 1 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_align: 0 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: max_auto_readahead_size: 262144 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: prepopulate_block_cache: 0 2024-03-30T00:27:54.329 INFO:journalctl@ceph.mon.b.smithi142.stdout: initial_auto_readahead_size: 8192 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.write_buffer_size: 33554432 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_write_buffer_number: 2 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compression: NoCompression 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bottommost_compression: Disabled 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.prefix_extractor: nullptr 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.num_levels: 7 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-30T00:27:54.330 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compression_opts.level: 32767 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compression_opts.strategy: 0 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-30T00:27:54.331 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compression_opts.enabled: false 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.target_file_size_base: 67108864 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-30T00:27:54.332 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.arena_block_size: 1048576 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.disable_auto_compactions: 0 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-30T00:27:54.333 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-30T00:27:54.334 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.table_properties_collectors: 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.inplace_update_support: 0 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-30T00:27:54.335 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.bloom_locality: 0 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.max_successive_merges: 0 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.paranoid_file_checks: 0 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.force_consistency_checks: 1 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.report_bg_io_stats: 0 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.ttl: 2592000 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.enable_blob_files: false 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.min_blob_size: 0 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.blob_file_size: 268435456 2024-03-30T00:27:54.336 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.blob_file_starting_level: 0 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d472005d-70bf-4f49-a243-531228c5fa4a 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711758473940743, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711758473941554, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711758473, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d472005d-70bf-4f49-a243-531228c5fa4a", "db_session_id": "T0CL9XAGT18GJXFG9FBE", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711758473941768, "job": 1, "event": "recovery_finished"} 2024-03-30T00:27:54.337 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56120807c000 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: DB pointer 0x56120806a000 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** DB Stats ** 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-30T00:27:54.338 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-30T00:27:54.339 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache BinnedLRUCache@0x561207fdf350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.2e-05 secs_since: 0 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-30T00:27:54.340 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: using public_addr v2:172.21.15.142:0/0 -> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: starting mon.b rank -1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(???) e0 preinit fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).mds e1 new map 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout: e1 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout: legacy client fscid: -1 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout: No filesystems configured 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-30T00:27:54.341 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2501037907' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:54.342 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2501037907' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: Updating smithi161:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi099:172.21.15.99=a;smithi142:172.21.15.142=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:27:54.343 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: Saving service mon spec with placement smithi099:172.21.15.99=a;smithi142:172.21.15.142=b;smithi161:172.21.15.161=c;count:3 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: Updating smithi161:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: Deploying daemon mon.c on smithi161 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='client.? 172.21.15.161:0/3593147641' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:54.344 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.a calling monitor election 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.c calling monitor election 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: monmap e2: 2 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: fsmap 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mgrmap e13: x(active, since 41s) 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: overall HEALTH_OK 2024-03-30T00:27:54.345 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.346 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.346 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:54.346 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:27:54.346 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:27:54.346 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: Deploying daemon mon.b on smithi142 2024-03-30T00:27:54.346 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:54.346 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:54.346 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: from='client.? 172.21.15.161:0/4155886575' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T00:27:54.346 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:54.346 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:27:53 smithi142 ceph-mon[41779]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-30T00:27:59.257 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.246+0000 7fa4a7400700 1 -- 172.21.15.161:0/2374000133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4880035c0 msgr2=0x7fa488005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:59.258 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.246+0000 7fa4a7400700 1 --2- 172.21.15.161:0/2374000133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4880035c0 0x7fa488005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0100720 tx=0x7fa49400fe60 comp rx=0 tx=0).stop 2024-03-30T00:27:59.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.246+0000 7fa4a7400700 1 -- 172.21.15.161:0/2374000133 shutdown_connections 2024-03-30T00:27:59.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.246+0000 7fa4a7400700 1 --2- 172.21.15.161:0/2374000133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4880035c0 0x7fa488005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:59.259 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.246+0000 7fa4a7400700 1 -- 172.21.15.161:0/2374000133 >> 172.21.15.161:0/2374000133 conn(0x7fa4a00fc2e0 msgr2=0x7fa4a00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.247+0000 7fa4a7400700 1 -- 172.21.15.161:0/2374000133 shutdown_connections 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.247+0000 7fa4a7400700 1 -- 172.21.15.161:0/2374000133 wait complete. 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.248+0000 7fa4a7400700 1 Processor -- start 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.248+0000 7fa4a7400700 1 -- start start 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.249+0000 7fa4a7400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4880035c0 0x7fa4a01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.249+0000 7fa4a7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4a0198820 0x7fa4a019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.249+0000 7fa4a7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a019d110 0x7fa4a019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.249+0000 7fa4a7400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a0198f10 con 0x7fa4a0198820 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.249+0000 7fa4a7400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01a4e80 con 0x7fa4a019d110 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.249+0000 7fa4a7400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01a4fc0 con 0x7fa4880035c0 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a4a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4880035c0 0x7fa4a01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a019d110 0x7fa4a019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:59.260 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a4a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4880035c0 0x7fa4a01982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57078/0 (socket says 172.21.15.161:57078) 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a4a00700 1 -- 172.21.15.161:0/1831556175 learned_addr learned my addr 172.21.15.161:0/1831556175 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa49fe00700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4a0198820 0x7fa4a019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a5400700 1 -- 172.21.15.161:0/1831556175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a019d110 msgr2=0x7fa4a019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a5400700 1 -- 172.21.15.161:0/1831556175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a019d110 msgr2=0x7fa4a019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a5400700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a019d110 0x7fa4a019d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a5400700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a019d110 0x7fa4a019d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a4a00700 1 -- 172.21.15.161:0/1831556175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4880035c0 msgr2=0x7fa4a01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a4a00700 1 -- 172.21.15.161:0/1831556175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4880035c0 msgr2=0x7fa4a01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a4a00700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4880035c0 0x7fa4a01982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa4a4a00700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4880035c0 0x7fa4a01982e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa49fe00700 1 -- 172.21.15.161:0/1831556175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4880035c0 msgr2=0x7fa4a01982e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa49fe00700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4880035c0 0x7fa4a01982e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:59.261 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa49fe00700 1 -- 172.21.15.161:0/1831556175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a019d110 msgr2=0x7fa4a019d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T00:27:59.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa49fe00700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a019d110 0x7fa4a019d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:59.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa49fe00700 1 -- 172.21.15.161:0/1831556175 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49400f8b0 con 0x7fa4a0198820 2024-03-30T00:27:59.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.250+0000 7fa49fe00700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4a0198820 0x7fa4a019cbd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00b550 tx=0x7fa48c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:59.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.251+0000 7fa49d600700 1 -- 172.21.15.161:0/1831556175 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c004230 con 0x7fa4a0198820 2024-03-30T00:27:59.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.251+0000 7fa49d600700 1 -- 172.21.15.161:0/1831556175 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa48c004390 con 0x7fa4a0198820 2024-03-30T00:27:59.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.251+0000 7fa49d600700 1 -- 172.21.15.161:0/1831556175 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c011a20 con 0x7fa4a0198820 2024-03-30T00:27:59.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.251+0000 7fa4a7400700 1 -- 172.21.15.161:0/1831556175 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a01a52a0 con 0x7fa4a0198820 2024-03-30T00:27:59.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.251+0000 7fa4a7400700 1 -- 172.21.15.161:0/1831556175 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a01a5750 con 0x7fa4a0198820 2024-03-30T00:27:59.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.252+0000 7fa4a7400700 1 -- 172.21.15.161:0/1831556175 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a0104aa0 con 0x7fa4a0198820 2024-03-30T00:27:59.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.252+0000 7fa49d600700 1 -- 172.21.15.161:0/1831556175 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fa48c011b80 con 0x7fa4a0198820 2024-03-30T00:27:59.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.253+0000 7fa49d600700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa480038c20 0x7fa48003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:27:59.263 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.253+0000 7fa49d600700 1 -- 172.21.15.161:0/1831556175 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa48c04ad20 con 0x7fa4a0198820 2024-03-30T00:27:59.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.253+0000 7fa4a4a00700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa480038c20 0x7fa48003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:27:59.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.254+0000 7fa4a4a00700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa480038c20 0x7fa48003b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa494001eb0 tx=0x7fa494005de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:27:59.266 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.258+0000 7fa49d600700 1 -- 172.21.15.161:0/1831556175 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa48c01a400 con 0x7fa4a0198820 2024-03-30T00:27:59.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:27:59.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: mon.a calling monitor election 2024-03-30T00:27:59.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: mon.c calling monitor election 2024-03-30T00:27:59.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:59.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:59.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: fsmap 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: mgrmap e13: x(active, since 50s) 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:59.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:59.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:27:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:59.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:27:59.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: mon.a calling monitor election 2024-03-30T00:27:59.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: mon.c calling monitor election 2024-03-30T00:27:59.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: fsmap 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: mgrmap e13: x(active, since 50s) 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-30T00:27:59.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2024-03-30T00:27:59.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:59.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:59.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:27:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:27:59.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.564+0000 7fa4a7400700 1 -- 172.21.15.161:0/1831556175 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa4a01995f0 con 0x7fa4a0198820 2024-03-30T00:27:59.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.564+0000 7fa49d600700 1 -- 172.21.15.161:0/1831556175 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7fa48c00f2e0 con 0x7fa4a0198820 2024-03-30T00:27:59.574 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-30T00:27:59.574 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":3,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","modified":"2024-03-30T00:27:53.983239Z","created":"2024-03-30T00:26:11.889664Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-30T00:27:59.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 -- 172.21.15.161:0/1831556175 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa480038c20 msgr2=0x7fa48003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:59.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa480038c20 0x7fa48003b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa494001eb0 tx=0x7fa494005de0 comp rx=0 tx=0).stop 2024-03-30T00:27:59.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 -- 172.21.15.161:0/1831556175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4a0198820 msgr2=0x7fa4a019cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:27:59.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4a0198820 0x7fa4a019cbd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00b550 tx=0x7fa48c00b910 comp rx=0 tx=0).stop 2024-03-30T00:27:59.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 -- 172.21.15.161:0/1831556175 shutdown_connections 2024-03-30T00:27:59.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa480038c20 0x7fa48003b0e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:59.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4a019d110 0x7fa4a019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:59.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4880035c0 0x7fa4a01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:59.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 --2- 172.21.15.161:0/1831556175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4a0198820 0x7fa4a019cbd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:27:59.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 -- 172.21.15.161:0/1831556175 >> 172.21.15.161:0/1831556175 conn(0x7fa4a00fc2e0 msgr2=0x7fa4a00fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:27:59.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 -- 172.21.15.161:0/1831556175 shutdown_connections 2024-03-30T00:27:59.577 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:27:59.567+0000 7fa4a7400700 1 -- 172.21.15.161:0/1831556175 wait complete. 2024-03-30T00:27:59.577 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 3 2024-03-30T00:27:59.957 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-30T00:27:59.958 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-03-30T00:28:00.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:00 smithi161 ceph-mon[42221]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:00.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:00 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:00.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:00 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:00.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:00 smithi161 ceph-mon[42221]: from='client.? 172.21.15.161:0/1831556175' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T00:28:00.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:00 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:00.308 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:28:00.459 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:00 smithi099 ceph-mon[38079]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:00.459 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:00 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:00.459 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:00 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:00.459 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:00 smithi099 ceph-mon[38079]: from='client.? 172.21.15.161:0/1831556175' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T00:28:00.459 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:00 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: mon.a calling monitor election 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: mon.c calling monitor election 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:01.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: fsmap 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: mgrmap e13: x(active, since 50s) 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: mon.b (rank 2) addr [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] is down (out of quorum) 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:01.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:01.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:01.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='client.? 172.21.15.161:0/1831556175' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T00:28:01.320 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.388+0000 7f0251e00700 1 -- 172.21.15.99:0/1767085328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f024c1066c0 msgr2=0x7f024c106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.388+0000 7f0251e00700 1 --2- 172.21.15.99:0/1767085328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f024c1066c0 0x7f024c106aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f023c009910 tx=0x7f023c009c20 comp rx=0 tx=0).stop 2024-03-30T00:28:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.388+0000 7f0251e00700 1 -- 172.21.15.99:0/1767085328 shutdown_connections 2024-03-30T00:28:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.388+0000 7f0251e00700 1 --2- 172.21.15.99:0/1767085328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f024c1066c0 0x7f024c106aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.388+0000 7f0251e00700 1 -- 172.21.15.99:0/1767085328 >> 172.21.15.99:0/1767085328 conn(0x7f024c0fc1b0 msgr2=0x7f024c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:28:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.388+0000 7f0251e00700 1 -- 172.21.15.99:0/1767085328 shutdown_connections 2024-03-30T00:28:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.388+0000 7f0251e00700 1 -- 172.21.15.99:0/1767085328 wait complete. 2024-03-30T00:28:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.389+0000 7f0251e00700 1 Processor -- start 2024-03-30T00:28:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.390+0000 7f0251e00700 1 -- start start 2024-03-30T00:28:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.390+0000 7f0251e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f024c1066c0 0x7f024c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.390+0000 7f0251e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f024c0735f0 0x7f024c073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.390+0000 7f0251e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f024c1a2250 0x7f024c1a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.390+0000 7f0251e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c1a4c10 con 0x7f024c1a2250 2024-03-30T00:28:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.390+0000 7f0251e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c1a4d50 con 0x7f024c0735f0 2024-03-30T00:28:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.390+0000 7f0251e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c1a4f10 con 0x7f024c1066c0 2024-03-30T00:28:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.391+0000 7f024b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f024c1a2250 0x7f024c1a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.391+0000 7f024b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f024c1a2250 0x7f024c1a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36832/0 (socket says 172.21.15.99:36832) 2024-03-30T00:28:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.391+0000 7f024b400700 1 -- 172.21.15.99:0/3933179363 learned_addr learned my addr 172.21.15.99:0/3933179363 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:28:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.391+0000 7f024a000700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f024c0735f0 0x7f024c073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.391+0000 7f024aa00700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f024c1066c0 0x7f024c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.391+0000 7f024b400700 1 -- 172.21.15.99:0/3933179363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f024c1066c0 msgr2=0x7f024c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.391+0000 7f024b400700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f024c1066c0 0x7f024c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.391+0000 7f024b400700 1 -- 172.21.15.99:0/3933179363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f024c0735f0 msgr2=0x7f024c073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.391+0000 7f024b400700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f024c0735f0 0x7f024c073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:01.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.391+0000 7f024b400700 1 -- 172.21.15.99:0/3933179363 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023c0095f0 con 0x7f024c1a2250 2024-03-30T00:28:01.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.392+0000 7f024b400700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f024c1a2250 0x7f024c1a4640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f024400ba50 tx=0x7f024400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:28:01.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.392+0000 7f0250a00700 1 -- 172.21.15.99:0/3933179363 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024400c930 con 0x7f024c1a2250 2024-03-30T00:28:01.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.392+0000 7f0250a00700 1 -- 172.21.15.99:0/3933179363 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0244014440 con 0x7f024c1a2250 2024-03-30T00:28:01.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.392+0000 7f0251e00700 1 -- 172.21.15.99:0/3933179363 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024c1a51f0 con 0x7f024c1a2250 2024-03-30T00:28:01.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.392+0000 7f0250a00700 1 -- 172.21.15.99:0/3933179363 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02440075b0 con 0x7f024c1a2250 2024-03-30T00:28:01.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.392+0000 7f0251e00700 1 -- 172.21.15.99:0/3933179363 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024c1a56f0 con 0x7f024c1a2250 2024-03-30T00:28:01.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.393+0000 7f0251e00700 1 -- 172.21.15.99:0/3933179363 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024c101e00 con 0x7f024c1a2250 2024-03-30T00:28:01.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.393+0000 7f0250a00700 1 -- 172.21.15.99:0/3933179363 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f024400ca90 con 0x7f024c1a2250 2024-03-30T00:28:01.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.394+0000 7f0250a00700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0230038c70 0x7f023003b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:01.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.394+0000 7f0250a00700 1 -- 172.21.15.99:0/3933179363 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f024404aac0 con 0x7f024c1a2250 2024-03-30T00:28:01.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.394+0000 7f024aa00700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0230038c70 0x7f023003b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:01.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.395+0000 7f024aa00700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0230038c70 0x7f023003b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f023c004690 tx=0x7f023c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:28:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.400+0000 7f0250a00700 1 -- 172.21.15.99:0/3933179363 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f024404ce60 con 0x7f024c1a2250 2024-03-30T00:28:01.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.616+0000 7f0251e00700 1 -- 172.21.15.99:0/3933179363 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f024c0623e0 con 0x7f024c1a2250 2024-03-30T00:28:01.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.617+0000 7f0250a00700 1 -- 172.21.15.99:0/3933179363 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f02440148b0 con 0x7f024c1a2250 2024-03-30T00:28:01.625 INFO:teuthology.orchestra.run.smithi099.stdout:# minimal ceph.conf for 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:28:01.625 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2024-03-30T00:28:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:28:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: mon_host = [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 -- 172.21.15.99:0/3933179363 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0230038c70 msgr2=0x7f023003b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0230038c70 0x7f023003b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f023c004690 tx=0x7f023c005fb0 comp rx=0 tx=0).stop 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 -- 172.21.15.99:0/3933179363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f024c1a2250 msgr2=0x7f024c1a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f024c1a2250 0x7f024c1a4640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f024400ba50 tx=0x7f024400bd60 comp rx=0 tx=0).stop 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 -- 172.21.15.99:0/3933179363 shutdown_connections 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f024c1a2250 0x7f024c1a4640 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f024c1066c0 0x7f024c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0230038c70 0x7f023003b130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 --2- 172.21.15.99:0/3933179363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f024c0735f0 0x7f024c073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 -- 172.21.15.99:0/3933179363 >> 172.21.15.99:0/3933179363 conn(0x7f024c0fc1b0 msgr2=0x7f024c108300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 -- 172.21.15.99:0/3933179363 shutdown_connections 2024-03-30T00:28:01.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:01.620+0000 7f0251e00700 1 -- 172.21.15.99:0/3933179363 wait complete. 2024-03-30T00:28:01.968 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: mon.b calling monitor election 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: Updating smithi099:/etc/ceph/ceph.conf 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: mon.b calling monitor election 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: mon.a calling monitor election 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: mon.c calling monitor election 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: fsmap 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: mgrmap e13: x(active, since 52s) 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: Cluster is now healthy 2024-03-30T00:28:01.969 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: overall HEALTH_OK 2024-03-30T00:28:01.970 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: Updating smithi099:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:28:01.970 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: Updating smithi142:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:28:01.970 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: Updating smithi161:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:28:01.970 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:01 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3933179363' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: mon.b calling monitor election 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: Updating smithi099:/etc/ceph/ceph.conf 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: mon.b calling monitor election 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: mon.a calling monitor election 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: mon.c calling monitor election 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: fsmap 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:28:01.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: mgrmap e13: x(active, since 52s) 2024-03-30T00:28:01.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-30T00:28:01.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: Cluster is now healthy 2024-03-30T00:28:01.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: overall HEALTH_OK 2024-03-30T00:28:01.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: Updating smithi099:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:28:01.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: Updating smithi142:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:28:01.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: Updating smithi161:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:28:01.972 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:01 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3933179363' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:01.996 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-30T00:28:01.997 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:28:01.997 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2024-03-30T00:28:02.065 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:28:02.066 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-30T00:28:02.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: mon.b calling monitor election 2024-03-30T00:28:02.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: Updating smithi099:/etc/ceph/ceph.conf 2024-03-30T00:28:02.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: Updating smithi142:/etc/ceph/ceph.conf 2024-03-30T00:28:02.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: Updating smithi161:/etc/ceph/ceph.conf 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: mon.b calling monitor election 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: mon.a calling monitor election 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: mon.c calling monitor election 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: monmap e3: 3 mons at {a=[v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0],b=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: fsmap 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: mgrmap e13: x(active, since 52s) 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: Cluster is now healthy 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: overall HEALTH_OK 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: Updating smithi099:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: Updating smithi142:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:28:02.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: Updating smithi161:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T00:28:02.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:01 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3933179363' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:02.173 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-30T00:28:02.173 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-03-30T00:28:02.242 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-30T00:28:02.242 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-30T00:28:02.347 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-30T00:28:02.347 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-03-30T00:28:02.415 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-30T00:28:02.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-30T00:28:02.521 INFO:tasks.cephadm:Adding mgr.x on smithi099 2024-03-30T00:28:02.521 INFO:tasks.cephadm:Adding mgr.y on smithi142 2024-03-30T00:28:02.521 INFO:tasks.cephadm:Adding mgr.z on smithi161 2024-03-30T00:28:02.521 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch apply mgr '3;smithi099=x;smithi142=y;smithi161=z' 2024-03-30T00:28:02.902 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:28:02.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:02.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:02.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:02.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:02.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:02.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:02.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:02.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:02.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:02.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:02.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:02.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:02.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: Reconfiguring mon.a (unknown last config time)... 2024-03-30T00:28:02.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:28:02.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T00:28:02.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:02.978 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:02 smithi099 ceph-mon[38079]: Reconfiguring daemon mon.a on smithi099 2024-03-30T00:28:03.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: Reconfiguring mon.a (unknown last config time)... 2024-03-30T00:28:03.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:28:03.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T00:28:03.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:03.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:02 smithi142 ceph-mon[41779]: Reconfiguring daemon mon.a on smithi099 2024-03-30T00:28:03.097 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.097 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.097 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.097 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.097 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.097 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.097 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.097 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:28:03.097 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:03.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: Reconfiguring mon.a (unknown last config time)... 2024-03-30T00:28:03.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:28:03.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T00:28:03.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:03.098 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:02 smithi161 ceph-mon[42221]: Reconfiguring daemon mon.a on smithi099 2024-03-30T00:28:04.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.166+0000 7f91f8800700 1 -- 172.21.15.161:0/2942216654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f0100840 msgr2=0x7f91f0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:04.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.166+0000 7f91f8800700 1 --2- 172.21.15.161:0/2942216654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f0100840 0x7f91f0100c60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009910 tx=0x7f91e4009c20 comp rx=0 tx=0).stop 2024-03-30T00:28:04.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.166+0000 7f91f8800700 1 -- 172.21.15.161:0/2942216654 shutdown_connections 2024-03-30T00:28:04.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.166+0000 7f91f8800700 1 --2- 172.21.15.161:0/2942216654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f0100840 0x7f91f0100c60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:04.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.166+0000 7f91f8800700 1 -- 172.21.15.161:0/2942216654 >> 172.21.15.161:0/2942216654 conn(0x7f91f00fbe00 msgr2=0x7f91f00fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:28:04.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.167+0000 7f91f8800700 1 -- 172.21.15.161:0/2942216654 shutdown_connections 2024-03-30T00:28:04.176 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.167+0000 7f91f8800700 1 -- 172.21.15.161:0/2942216654 wait complete. 2024-03-30T00:28:04.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.168+0000 7f91f8800700 1 Processor -- start 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.168+0000 7f91f8800700 1 -- start start 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f8800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91f0100840 0x7f91f0195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f8800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f01961f0 0x7f91f019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f8800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f0196670 0x7f91f019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f8800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f019df50 con 0x7f91f0196670 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f8800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f019e0c0 con 0x7f91f0100840 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f8800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f019e280 con 0x7f91f01961f0 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f5400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f01961f0 0x7f91f019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91f0100840 0x7f91f0195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f5400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f01961f0 0x7f91f019b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:57082/0 (socket says 172.21.15.161:57082) 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f5400700 1 -- 172.21.15.161:0/3749503725 learned_addr learned my addr 172.21.15.161:0/3749503725 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-30T00:28:04.179 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f6800700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f0196670 0x7f91f019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:04.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f5400700 1 -- 172.21.15.161:0/3749503725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91f0100840 msgr2=0x7f91f0195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:04.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f5400700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91f0100840 0x7f91f0195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:04.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f5400700 1 -- 172.21.15.161:0/3749503725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f0196670 msgr2=0x7f91f019d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:04.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f5400700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f0196670 0x7f91f019d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:04.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f5400700 1 -- 172.21.15.161:0/3749503725 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40095f0 con 0x7f91f01961f0 2024-03-30T00:28:04.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.169+0000 7f91f5e00700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91f0100840 0x7f91f0195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:28:04.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.170+0000 7f91f5400700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f01961f0 0x7f91f019b1d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f91e000b780 tx=0x7f91e000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:28:04.180 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.170+0000 7f91eaa00700 1 -- 172.21.15.161:0/3749503725 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0004230 con 0x7f91f01961f0 2024-03-30T00:28:04.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.170+0000 7f91f8800700 1 -- 172.21.15.161:0/3749503725 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f019e560 con 0x7f91f01961f0 2024-03-30T00:28:04.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.170+0000 7f91eaa00700 1 -- 172.21.15.161:0/3749503725 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91e0004390 con 0x7f91f01961f0 2024-03-30T00:28:04.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.170+0000 7f91eaa00700 1 -- 172.21.15.161:0/3749503725 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0011a30 con 0x7f91f01961f0 2024-03-30T00:28:04.181 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.170+0000 7f91f8800700 1 -- 172.21.15.161:0/3749503725 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f019eae0 con 0x7f91f01961f0 2024-03-30T00:28:04.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.171+0000 7f91f8800700 1 -- 172.21.15.161:0/3749503725 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f00623e0 con 0x7f91f01961f0 2024-03-30T00:28:04.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.171+0000 7f91eaa00700 1 -- 172.21.15.161:0/3749503725 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f91e00036a0 con 0x7f91f01961f0 2024-03-30T00:28:04.182 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.172+0000 7f91eaa00700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f91d8038c70 0x7f91d803b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:04.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.172+0000 7f91eaa00700 1 -- 172.21.15.161:0/3749503725 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f91e004a060 con 0x7f91f01961f0 2024-03-30T00:28:04.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.172+0000 7f91f5e00700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f91d8038c70 0x7f91d803b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:04.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.173+0000 7f91f5e00700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f91d8038c70 0x7f91d803b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91f0197140 tx=0x7f91e4005160 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:28:04.185 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.178+0000 7f91eaa00700 1 -- 172.21.15.161:0/3749503725 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f91e001a3b0 con 0x7f91f01961f0 2024-03-30T00:28:04.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:04 smithi161 ceph-mon[42221]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:04.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:04.399 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.392+0000 7f91f8800700 1 -- 172.21.15.161:0/3749503725 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi142=y;smithi161=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f91f00fde10 con 0x7f91d8038c70 2024-03-30T00:28:04.408 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.400+0000 7f91eaa00700 1 -- 172.21.15.161:0/3749503725 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f91f00fde10 con 0x7f91d8038c70 2024-03-30T00:28:04.408 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mgr update... 2024-03-30T00:28:04.411 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.403+0000 7f91f8800700 1 -- 172.21.15.161:0/3749503725 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f91d8038c70 msgr2=0x7f91d803b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:04.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.403+0000 7f91f8800700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f91d8038c70 0x7f91d803b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91f0197140 tx=0x7f91e4005160 comp rx=0 tx=0).stop 2024-03-30T00:28:04.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.403+0000 7f91f8800700 1 -- 172.21.15.161:0/3749503725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f01961f0 msgr2=0x7f91f019b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:04.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.403+0000 7f91f8800700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f01961f0 0x7f91f019b1d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f91e000b780 tx=0x7f91e000ba90 comp rx=0 tx=0).stop 2024-03-30T00:28:04.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.403+0000 7f91f8800700 1 -- 172.21.15.161:0/3749503725 shutdown_connections 2024-03-30T00:28:04.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.403+0000 7f91f8800700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91f0196670 0x7f91f019d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:04.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.403+0000 7f91f8800700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f91d8038c70 0x7f91d803b130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:04.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.404+0000 7f91f8800700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91f0100840 0x7f91f0195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:04.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.404+0000 7f91f8800700 1 --2- 172.21.15.161:0/3749503725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91f01961f0 0x7f91f019b1d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:04.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.404+0000 7f91f8800700 1 -- 172.21.15.161:0/3749503725 >> 172.21.15.161:0/3749503725 conn(0x7f91f00fbe00 msgr2=0x7f91f00fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:28:04.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.404+0000 7f91f8800700 1 -- 172.21.15.161:0/3749503725 shutdown_connections 2024-03-30T00:28:04.412 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:04.404+0000 7f91f8800700 1 -- 172.21.15.161:0/3749503725 wait complete. 2024-03-30T00:28:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:04 smithi161 ceph-mon[42221]: Reconfiguring mon.b (monmap changed)... 2024-03-30T00:28:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:28:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T00:28:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:04 smithi161 ceph-mon[42221]: Reconfiguring daemon mon.b on smithi142 2024-03-30T00:28:04.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:04 smithi099 ceph-mon[38079]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:04.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:04.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:04 smithi099 ceph-mon[38079]: Reconfiguring mon.b (monmap changed)... 2024-03-30T00:28:04.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:28:04.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T00:28:04.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:04.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:04 smithi099 ceph-mon[38079]: Reconfiguring daemon mon.b on smithi142 2024-03-30T00:28:04.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:04.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:04 smithi142 ceph-mon[41779]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:04.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:04.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:04.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:04 smithi142 ceph-mon[41779]: Reconfiguring mon.b (monmap changed)... 2024-03-30T00:28:04.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:28:04.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T00:28:04.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:04.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:04 smithi142 ceph-mon[41779]: Reconfiguring daemon mon.b on smithi142 2024-03-30T00:28:04.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:04.784 DEBUG:teuthology.orchestra.run.smithi142:mgr.y> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.y.service 2024-03-30T00:28:04.788 DEBUG:teuthology.orchestra.run.smithi161:mgr.z> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.z.service 2024-03-30T00:28:04.792 INFO:tasks.cephadm:Deploying OSDs... 2024-03-30T00:28:04.792 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:28:04.792 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T00:28:04.839 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T00:28:04.839 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-03-30T00:28:04.856 INFO:journalctl@ceph.mgr.y.smithi142.stdout:-- Logs begin at Sat 2024-03-30 00:10:40 UTC. -- 2024-03-30T00:28:04.860 INFO:journalctl@ceph.mgr.z.smithi161.stdout:-- Logs begin at Sat 2024-03-30 00:10:26 UTC. -- 2024-03-30T00:28:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T00:28:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:04.924 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 448 Links: 1 2024-03-30T00:28:04.924 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:04.924 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:04.924 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-30 00:27:19.074099716 +0000 2024-03-30T00:28:04.925 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-30 00:20:11.471266030 +0000 2024-03-30T00:28:04.925 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-30 00:20:11.471266030 +0000 2024-03-30T00:28:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-30 00:20:11.444266678 +0000 2024-03-30T00:28:04.925 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T00:28:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-30T00:28:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-30T00:28:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000312585 s, 1.6 MB/s 2024-03-30T00:28:05.031 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T00:28:05.075 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-03-30T00:28:05.157 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T00:28:05.157 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:05.157 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 454 Links: 1 2024-03-30T00:28:05.157 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:05.158 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:05.158 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-30 00:27:19.143098034 +0000 2024-03-30T00:28:05.158 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-30 00:20:11.947254613 +0000 2024-03-30T00:28:05.158 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-30 00:20:11.947254613 +0000 2024-03-30T00:28:05.158 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-30 00:20:11.917255333 +0000 2024-03-30T00:28:05.158 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T00:28:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-30T00:28:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-30T00:28:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.00028613 s, 1.8 MB/s 2024-03-30T00:28:05.262 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T00:28:05.304 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-03-30T00:28:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T00:28:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:05.388 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 460 Links: 1 2024-03-30T00:28:05.388 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:05.388 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:05.388 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-30 00:27:19.209096426 +0000 2024-03-30T00:28:05.388 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-30 00:20:12.430243029 +0000 2024-03-30T00:28:05.388 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-30 00:20:12.430243029 +0000 2024-03-30T00:28:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-30 00:20:12.404243652 +0000 2024-03-30T00:28:05.389 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T00:28:05.490 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-30T00:28:05.490 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-30T00:28:05.490 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000289488 s, 1.8 MB/s 2024-03-30T00:28:05.492 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T00:28:05.535 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-03-30T00:28:05.619 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T00:28:05.619 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:05.619 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 466 Links: 1 2024-03-30T00:28:05.619 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:05.619 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:05.619 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-30 00:27:19.277094768 +0000 2024-03-30T00:28:05.619 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-30 00:20:12.882232187 +0000 2024-03-30T00:28:05.619 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-30 00:20:12.882232187 +0000 2024-03-30T00:28:05.619 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-30 00:20:12.856232811 +0000 2024-03-30T00:28:05.619 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T00:28:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:05 smithi099 ceph-mon[38079]: Reconfiguring mon.c (monmap changed)... 2024-03-30T00:28:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:28:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T00:28:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:05 smithi099 ceph-mon[38079]: Reconfiguring daemon mon.c on smithi161 2024-03-30T00:28:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:05 smithi099 ceph-mon[38079]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi142=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:05 smithi099 ceph-mon[38079]: Saving service mgr spec with placement smithi099=x;smithi142=y;smithi161=z;count:3 2024-03-30T00:28:05.716 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:05.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:05.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:05.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:05 smithi161 ceph-mon[42221]: Reconfiguring mon.c (monmap changed)... 2024-03-30T00:28:05.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:28:05.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T00:28:05.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:05.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:05 smithi161 ceph-mon[42221]: Reconfiguring daemon mon.c on smithi161 2024-03-30T00:28:05.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:05 smithi161 ceph-mon[42221]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi142=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:05.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:05 smithi161 ceph-mon[42221]: Saving service mgr spec with placement smithi099=x;smithi142=y;smithi161=z;count:3 2024-03-30T00:28:05.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-30T00:28:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-30T00:28:05.721 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000277984 s, 1.8 MB/s 2024-03-30T00:28:05.724 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T00:28:05.769 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-03-30T00:28:05.769 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T00:28:05.816 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T00:28:05.816 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-03-30T00:28:05.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:05.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:05.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:05 smithi142 ceph-mon[41779]: Reconfiguring mon.c (monmap changed)... 2024-03-30T00:28:05.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T00:28:05.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T00:28:05.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:05.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:05 smithi142 ceph-mon[41779]: Reconfiguring daemon mon.c on smithi161 2024-03-30T00:28:05.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:05 smithi142 ceph-mon[41779]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi099=x;smithi142=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:05.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:05 smithi142 ceph-mon[41779]: Saving service mgr spec with placement smithi099=x;smithi142=y;smithi161=z;count:3 2024-03-30T00:28:05.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:05.861 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T00:28:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:05.862 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 448 Links: 1 2024-03-30T00:28:05.862 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:05.862 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:05.862 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-03-30 00:27:53.358452450 +0000 2024-03-30T00:28:05.862 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-03-30 00:20:11.843618245 +0000 2024-03-30T00:28:05.862 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-03-30 00:20:11.843618245 +0000 2024-03-30T00:28:05.862 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-03-30 00:20:11.817618727 +0000 2024-03-30T00:28:05.862 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T00:28:05.965 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-03-30T00:28:05.965 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-03-30T00:28:05.965 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000309542 s, 1.7 MB/s 2024-03-30T00:28:05.968 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T00:28:06.010 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-03-30T00:28:06.092 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T00:28:06.093 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:06.093 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 454 Links: 1 2024-03-30T00:28:06.093 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:06.093 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:06.093 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-03-30 00:27:53.358452450 +0000 2024-03-30T00:28:06.093 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-03-30 00:20:12.317609472 +0000 2024-03-30T00:28:06.093 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-03-30 00:20:12.317609472 +0000 2024-03-30T00:28:06.093 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-03-30 00:20:12.294609898 +0000 2024-03-30T00:28:06.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T00:28:06.194 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-03-30T00:28:06.194 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-03-30T00:28:06.194 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000288701 s, 1.8 MB/s 2024-03-30T00:28:06.197 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T00:28:06.239 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-03-30T00:28:06.321 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T00:28:06.321 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:06.321 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 460 Links: 1 2024-03-30T00:28:06.322 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:06.322 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:06.322 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-03-30 00:27:53.359452432 +0000 2024-03-30T00:28:06.322 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-03-30 00:20:12.775600995 +0000 2024-03-30T00:28:06.322 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-03-30 00:20:12.775600995 +0000 2024-03-30T00:28:06.322 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-03-30 00:20:12.752601421 +0000 2024-03-30T00:28:06.322 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T00:28:06.424 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-03-30T00:28:06.424 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-03-30T00:28:06.424 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000290759 s, 1.8 MB/s 2024-03-30T00:28:06.426 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T00:28:06.469 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-03-30T00:28:06.551 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T00:28:06.551 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:06.552 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 466 Links: 1 2024-03-30T00:28:06.552 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:06.552 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:06.552 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-03-30 00:27:53.359452432 +0000 2024-03-30T00:28:06.552 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-03-30 00:20:13.245592296 +0000 2024-03-30T00:28:06.552 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-03-30 00:20:13.245592296 +0000 2024-03-30T00:28:06.552 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-03-30 00:20:13.220592759 +0000 2024-03-30T00:28:06.552 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T00:28:06.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:06 smithi142 ceph-mon[41779]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:06.647 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:06 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:06.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:06 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:06.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:06 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:06.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:06 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:06.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:06 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:06.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:06 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:06.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:06 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T00:28:06.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:06 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T00:28:06.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:06 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T00:28:06.648 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:06 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:06.653 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-03-30T00:28:06.653 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-03-30T00:28:06.653 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000306258 s, 1.7 MB/s 2024-03-30T00:28:06.656 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T00:28:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:06 smithi161 ceph-mon[42221]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:06 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:06 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:06 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:06 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:06 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:06 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:06 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T00:28:06.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:06 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T00:28:06.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:06 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T00:28:06.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:06 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:06.699 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-30T00:28:06.700 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T00:28:06.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:06 smithi099 ceph-mon[38079]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:06.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:06 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:06.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:06 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:06.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:06 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:06.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:06 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:06.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:06 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:06.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:06 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:06.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:06 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T00:28:06.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:06 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T00:28:06.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:06 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T00:28:06.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:06 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:06.741 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T00:28:06.741 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-03-30T00:28:06.823 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T00:28:06.823 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:06.823 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 448 Links: 1 2024-03-30T00:28:06.823 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:06.823 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:06.823 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-30 00:27:44.480299190 +0000 2024-03-30T00:28:06.823 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-30 00:20:25.931487083 +0000 2024-03-30T00:28:06.823 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-30 00:20:25.931487083 +0000 2024-03-30T00:28:06.823 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-30 00:20:25.904487652 +0000 2024-03-30T00:28:06.823 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T00:28:07.123 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-30T00:28:07.123 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-30T00:28:07.123 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000323483 s, 1.6 MB/s 2024-03-30T00:28:07.125 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T00:28:07.169 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-03-30T00:28:07.266 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T00:28:07.266 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:07.266 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 454 Links: 1 2024-03-30T00:28:07.266 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:07.266 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:07.266 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-30 00:28:06.991826719 +0000 2024-03-30T00:28:07.266 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-30 00:20:26.396477282 +0000 2024-03-30T00:28:07.267 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-30 00:20:26.396477282 +0000 2024-03-30T00:28:07.267 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-30 00:20:26.373477766 +0000 2024-03-30T00:28:07.267 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T00:28:07.408 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:07 smithi161 ceph-mon[42221]: Deploying daemon mgr.z on smithi161 2024-03-30T00:28:07.414 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-30T00:28:07.414 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-30T00:28:07.414 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000286692 s, 1.8 MB/s 2024-03-30T00:28:07.417 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T00:28:07.462 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-03-30T00:28:07.547 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T00:28:07.547 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:07.547 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 460 Links: 1 2024-03-30T00:28:07.547 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:07.547 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:07.547 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-30 00:28:07.275820762 +0000 2024-03-30T00:28:07.548 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-30 00:20:26.873467227 +0000 2024-03-30T00:28:07.548 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-30 00:20:26.873467227 +0000 2024-03-30T00:28:07.548 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-30 00:20:26.850467712 +0000 2024-03-30T00:28:07.548 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T00:28:07.655 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-30T00:28:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-30T00:28:07.656 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.00028822 s, 1.8 MB/s 2024-03-30T00:28:07.658 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T00:28:07.706 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-03-30T00:28:07.720 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:07 smithi161 systemd[1]: Starting Ceph mgr.z for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:28:07.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:07 smithi099 ceph-mon[38079]: Deploying daemon mgr.z on smithi161 2024-03-30T00:28:07.763 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T00:28:07.763 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T00:28:07.763 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 466 Links: 1 2024-03-30T00:28:07.763 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:28:07.763 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T00:28:07.763 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-30 00:28:07.275820762 +0000 2024-03-30T00:28:07.763 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-30 00:20:27.336457468 +0000 2024-03-30T00:28:07.763 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-30 00:20:27.336457468 +0000 2024-03-30T00:28:07.763 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-30 00:20:27.312457973 +0000 2024-03-30T00:28:07.763 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T00:28:07.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:07 smithi142 ceph-mon[41779]: Deploying daemon mgr.z on smithi161 2024-03-30T00:28:07.866 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-30T00:28:07.866 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-30T00:28:07.866 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000287491 s, 1.8 MB/s 2024-03-30T00:28:07.869 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T00:28:07.914 INFO:tasks.cephadm:Deploying osd.0 on smithi099 with /dev/vg_nvme/lv_4... 2024-03-30T00:28:07.914 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-30T00:28:08.220 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:07 smithi161 bash[44058]: 3eea1cdabea7d1a1aac30e546d4a85c00dc7d0789b344b8b9f59b1c51baf371f 2024-03-30T00:28:08.220 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:07 smithi161 systemd[1]: Started Ceph mgr.z for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:28:08.221 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:07 smithi161 ceph-mgr[44179]: -- 172.21.15.161:0/2512316431 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b2db84e1a0 con 0x55b2db826c00 2024-03-30T00:28:08.270 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:28:08.668 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:08.432+0000 7fc3e26ba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T00:28:08.668 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:08.548+0000 7fc3e26ba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T00:28:08.668 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:08.659+0000 7fc3e26ba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T00:28:08.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-mon[42221]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:08.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:08.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:08.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:08.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T00:28:08.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T00:28:08.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T00:28:08.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:08.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:08 smithi099 ceph-mon[38079]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:08.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:08.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:08.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:08.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T00:28:08.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T00:28:08.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T00:28:08.692 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:08.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:08 smithi142 ceph-mon[41779]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:08.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:08 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:08.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:08 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:08.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:08 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:08.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:08 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T00:28:08.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:08 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T00:28:08.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:08 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T00:28:08.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:08 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:08.970 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:08 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:08.768+0000 7fc3e26ba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T00:28:09.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:09 smithi142 ceph-mon[41779]: Deploying daemon mgr.y on smithi142 2024-03-30T00:28:09.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:09 smithi161 ceph-mon[42221]: Deploying daemon mgr.y on smithi142 2024-03-30T00:28:09.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:09 smithi099 ceph-mon[38079]: Deploying daemon mgr.y on smithi142 2024-03-30T00:28:09.871 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-30T00:28:09.871 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-30T00:28:09.871 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-03-30T00:28:09.871 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-03-30T00:28:09.871 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0381186 s, 275 MB/s 2024-03-30T00:28:09.871 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-03-30T00:28:09.928 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:09 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:09.611+0000 7fc3e26ba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:28:10.067 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:09 smithi142 systemd[1]: Starting Ceph mgr.y for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:28:10.220 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:09 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:09.920+0000 7fc3e26ba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T00:28:10.220 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:10 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:10.029+0000 7fc3e26ba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T00:28:10.220 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:10 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:10.166+0000 7fc3e26ba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T00:28:10.270 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2024-03-30T00:28:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:10 smithi142 ceph-mon[41779]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:10.568 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:10 smithi142 bash[43215]: 86e2af8b711feed70653df543de05ca9570a3503abdafe6ae8442a8348a49ea5 2024-03-30T00:28:10.568 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:10 smithi142 systemd[1]: Started Ceph mgr.y for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:28:10.568 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:10 smithi142 ceph-mgr[43253]: -- 172.21.15.142:0/3599875631 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557b779f41a0 con 0x557b779ccc00 2024-03-30T00:28:10.624 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:28:10.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:10 smithi161 ceph-mon[42221]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:10.645 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:10 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:10.361+0000 7fc3e26ba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T00:28:10.645 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:10 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:10.528+0000 7fc3e26ba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T00:28:10.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:10 smithi099 ceph-mon[38079]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:10.970 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:10 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:10.637+0000 7fc3e26ba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T00:28:11.084 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:10 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:10.821+0000 7fcf57939200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T00:28:11.084 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:10 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:10.957+0000 7fcf57939200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T00:28:11.085 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:11 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:11.077+0000 7fcf57939200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T00:28:11.430 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:11 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:11.140+0000 7fc3e26ba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T00:28:11.430 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:11 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:11.204+0000 7fcf57939200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T00:28:11.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.673+0000 7f2be0000700 1 -- 172.21.15.99:0/2978097535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd8074de0 msgr2=0x7f2bd8073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:11.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.673+0000 7f2be0000700 1 --2- 172.21.15.99:0/2978097535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd8074de0 0x7f2bd8073240 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009990 tx=0x7f2bcc009ca0 comp rx=0 tx=0).stop 2024-03-30T00:28:11.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.673+0000 7f2be0000700 1 -- 172.21.15.99:0/2978097535 shutdown_connections 2024-03-30T00:28:11.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.673+0000 7f2be0000700 1 --2- 172.21.15.99:0/2978097535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd8108cd0 0x7f2bd810b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:11.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.673+0000 7f2be0000700 1 --2- 172.21.15.99:0/2978097535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd8074de0 0x7f2bd8073240 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:11.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.673+0000 7f2be0000700 1 --2- 172.21.15.99:0/2978097535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd8073810 0x7f2bd8073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:11.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.673+0000 7f2be0000700 1 -- 172.21.15.99:0/2978097535 >> 172.21.15.99:0/2978097535 conn(0x7f2bd80fc3c0 msgr2=0x7f2bd80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:28:11.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.673+0000 7f2be0000700 1 -- 172.21.15.99:0/2978097535 shutdown_connections 2024-03-30T00:28:11.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.673+0000 7f2be0000700 1 -- 172.21.15.99:0/2978097535 wait complete. 2024-03-30T00:28:11.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.674+0000 7f2be0000700 1 Processor -- start 2024-03-30T00:28:11.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.674+0000 7f2be0000700 1 -- start start 2024-03-30T00:28:11.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.675+0000 7f2be0000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd8073810 0x7f2bd8198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:11.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.675+0000 7f2be0000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd8074de0 0x7f2bd81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:11.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.675+0000 7f2be0000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd8108cd0 0x7f2bd81a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:11.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.675+0000 7f2be0000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd81a09d0 con 0x7f2bd8073810 2024-03-30T00:28:11.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.675+0000 7f2be0000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd81a0b10 con 0x7f2bd8108cd0 2024-03-30T00:28:11.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.675+0000 7f2be0000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd81a0ca0 con 0x7f2bd8074de0 2024-03-30T00:28:11.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.675+0000 7f2bdd600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd8073810 0x7f2bd8198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:11.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.675+0000 7f2bdd600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd8073810 0x7f2bd8198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45658/0 (socket says 172.21.15.99:45658) 2024-03-30T00:28:11.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.675+0000 7f2bdd600700 1 -- 172.21.15.99:0/274511570 learned_addr learned my addr 172.21.15.99:0/274511570 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:28:11.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.675+0000 7f2bdcc00700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd8074de0 0x7f2bd81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:11.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.675+0000 7f2bde000700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd8108cd0 0x7f2bd81a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:11.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.676+0000 7f2bdd600700 1 -- 172.21.15.99:0/274511570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd8074de0 msgr2=0x7f2bd81993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:11.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.676+0000 7f2bdd600700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd8074de0 0x7f2bd81993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:11.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.676+0000 7f2bdd600700 1 -- 172.21.15.99:0/274511570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd8108cd0 msgr2=0x7f2bd81a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:11.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.676+0000 7f2bdd600700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd8108cd0 0x7f2bd81a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:11.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.676+0000 7f2bdd600700 1 -- 172.21.15.99:0/274511570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc0095f0 con 0x7f2bd8073810 2024-03-30T00:28:11.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.676+0000 7f2bdd600700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd8073810 0x7f2bd8198e90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0098e0 tx=0x7f2bcc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:28:11.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.676+0000 7f2bd6000700 1 -- 172.21.15.99:0/274511570 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc01d070 con 0x7f2bd8073810 2024-03-30T00:28:11.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.676+0000 7f2bd6000700 1 -- 172.21.15.99:0/274511570 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bcc004000 con 0x7f2bd8073810 2024-03-30T00:28:11.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.676+0000 7f2bd6000700 1 -- 172.21.15.99:0/274511570 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc022cc0 con 0x7f2bd8073810 2024-03-30T00:28:11.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.676+0000 7f2be0000700 1 -- 172.21.15.99:0/274511570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd81a0f20 con 0x7f2bd8073810 2024-03-30T00:28:11.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.676+0000 7f2be0000700 1 -- 172.21.15.99:0/274511570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd81a1280 con 0x7f2bd8073810 2024-03-30T00:28:11.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.677+0000 7f2bd6000700 1 -- 172.21.15.99:0/274511570 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f2bcc022470 con 0x7f2bd8073810 2024-03-30T00:28:11.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.678+0000 7f2be0000700 1 -- 172.21.15.99:0/274511570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd8107a90 con 0x7f2bd8073810 2024-03-30T00:28:11.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.678+0000 7f2bd6000700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2bc00389d0 0x7f2bc003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:11.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.678+0000 7f2bd6000700 1 -- 172.21.15.99:0/274511570 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2bcc04c740 con 0x7f2bd8073810 2024-03-30T00:28:11.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.678+0000 7f2bdcc00700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2bc00389d0 0x7f2bc003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:11.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.679+0000 7f2bdcc00700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2bc00389d0 0x7f2bc003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8074b10 tx=0x7f2bc400a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:28:11.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.683+0000 7f2bd6000700 1 -- 172.21.15.99:0/274511570 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2bcc00e5c0 con 0x7f2bd8073810 2024-03-30T00:28:11.720 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:11 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:11.667+0000 7fc3e26ba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T00:28:11.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:11.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:11.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:11.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:11.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:11.892+0000 7f2be0000700 1 -- 172.21.15.99:0/274511570 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bd802cdf0 con 0x7f2bc00389d0 2024-03-30T00:28:12.220 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:11 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:11.946+0000 7fc3e26ba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T00:28:12.220 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:12.069+0000 7fc3e26ba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T00:28:12.432 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:12.154+0000 7fcf57939200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:28:12.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:12.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:28:12.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:28:12.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:12.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:12.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:12.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T00:28:12.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T00:28:12.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:12.721 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:12.470+0000 7fcf57939200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T00:28:12.721 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:12.577+0000 7fcf57939200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T00:28:12.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:12.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:28:12.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:28:12.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:12.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.722 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.723 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:12.723 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:12.723 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.723 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T00:28:12.723 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T00:28:12.723 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:12.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:12.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:28:12.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:28:12.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:12.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:12.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:12.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:12.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T00:28:12.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T00:28:12.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:13.067 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:12.713+0000 7fcf57939200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T00:28:13.067 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:12 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:12.907+0000 7fcf57939200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T00:28:13.355 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:13 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:13.029+0000 7fc3e26ba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T00:28:13.355 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:13 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:13.154+0000 7fc3e26ba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T00:28:13.355 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:13 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:13.347+0000 7fc3e26ba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T00:28:13.436 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:13 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:13.072+0000 7fcf57939200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T00:28:13.437 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:13 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:13.181+0000 7fcf57939200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T00:28:13.718 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:13 smithi099 ceph-mon[38079]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:13.718 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:13 smithi099 ceph-mon[38079]: Reconfiguring mgr.x (unknown last config time)... 2024-03-30T00:28:13.718 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:13 smithi099 ceph-mon[38079]: Reconfiguring daemon mgr.x on smithi099 2024-03-30T00:28:13.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:13 smithi161 ceph-mon[42221]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:13.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:13 smithi161 ceph-mon[42221]: Reconfiguring mgr.x (unknown last config time)... 2024-03-30T00:28:13.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:13 smithi161 ceph-mon[42221]: Reconfiguring daemon mgr.x on smithi099 2024-03-30T00:28:13.817 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:13 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:13.650+0000 7fcf57939200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T00:28:13.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:13 smithi142 ceph-mon[41779]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:13.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:13 smithi142 ceph-mon[41779]: Reconfiguring mgr.x (unknown last config time)... 2024-03-30T00:28:13.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:13 smithi142 ceph-mon[41779]: Reconfiguring daemon mgr.x on smithi099 2024-03-30T00:28:14.440 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:14 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:14.129+0000 7fcf57939200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T00:28:14.440 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:14 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:14.374+0000 7fcf57939200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T00:28:14.440 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:14 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:14.147+0000 7fc3e26ba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T00:28:14.441 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:14 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:14.383+0000 7fc3e26ba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T00:28:14.720 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:14 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:14.642+0000 7fc3e26ba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T00:28:14.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:14 smithi161 ceph-mon[42221]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:14.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:14.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:14.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:14.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:14.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:14.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:14.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:14.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:14 smithi099 ceph-mon[38079]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:14.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:14.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:14.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:14.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:14.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:14.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:14.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:14.817 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:14 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:14.506+0000 7fcf57939200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T00:28:14.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:14 smithi142 ceph-mon[41779]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:14.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:14.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:14.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:14.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:14.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:14.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:14.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:15.768 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:15.448+0000 7fcf57939200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T00:28:15.768 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:15.571+0000 7fcf57939200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T00:28:16.067 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:15.760+0000 7fcf57939200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T00:28:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:15 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2058451205' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c244d549-0aff-48af-9523-f7096beda3e4"}]: dispatch 2024-03-30T00:28:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:15 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2058451205' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c244d549-0aff-48af-9523-f7096beda3e4"}]': finished 2024-03-30T00:28:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:15 smithi142 ceph-mon[41779]: osdmap e5: 1 total, 0 up, 1 in 2024-03-30T00:28:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:15 smithi142 ceph-mon[41779]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:15 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2115825461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:28:16.164 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:15 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:15.783+0000 7fc3e26ba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T00:28:16.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:15 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2058451205' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c244d549-0aff-48af-9523-f7096beda3e4"}]: dispatch 2024-03-30T00:28:16.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:15 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2058451205' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c244d549-0aff-48af-9523-f7096beda3e4"}]': finished 2024-03-30T00:28:16.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:15 smithi161 ceph-mon[42221]: osdmap e5: 1 total, 0 up, 1 in 2024-03-30T00:28:16.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:16.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:15 smithi161 ceph-mon[42221]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:16.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:15 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2115825461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:28:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:15 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2058451205' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c244d549-0aff-48af-9523-f7096beda3e4"}]: dispatch 2024-03-30T00:28:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:15 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2058451205' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c244d549-0aff-48af-9523-f7096beda3e4"}]': finished 2024-03-30T00:28:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:15 smithi099 ceph-mon[38079]: osdmap e5: 1 total, 0 up, 1 in 2024-03-30T00:28:16.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:16.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:15 smithi099 ceph-mon[38079]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:16.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:15 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2115825461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:28:16.470 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:16 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:16.156+0000 7fc3e26ba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:28:16.470 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:16 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:16.272+0000 7fc3e26ba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T00:28:16.817 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:16 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:16.557+0000 7fcf57939200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T00:28:16.817 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:16 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:16.789+0000 7fcf57939200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T00:28:16.879 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:28:16 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z[44119]: 2024-03-30T00:28:16.523+0000 7fc3e26ba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T00:28:16.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:16.881+0000 7f2bd6000700 1 -- 172.21.15.99:0/274511570 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 14) v1 ==== 90699+0+0 (secure 0 0 0) 0x7f2bcc00f460 con 0x7f2bd8073810 2024-03-30T00:28:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:16 smithi161 ceph-mon[42221]: Standby manager daemon z started 2024-03-30T00:28:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:16 smithi161 ceph-mon[42221]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-30T00:28:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:16 smithi161 ceph-mon[42221]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T00:28:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:16 smithi161 ceph-mon[42221]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-30T00:28:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:16 smithi161 ceph-mon[42221]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T00:28:17.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:16 smithi099 ceph-mon[38079]: Standby manager daemon z started 2024-03-30T00:28:17.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:16 smithi099 ceph-mon[38079]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-30T00:28:17.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:16 smithi099 ceph-mon[38079]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T00:28:17.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:16 smithi099 ceph-mon[38079]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-30T00:28:17.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:16 smithi099 ceph-mon[38079]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T00:28:17.317 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:17 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:17.039+0000 7fcf57939200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T00:28:17.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:16 smithi142 ceph-mon[41779]: Standby manager daemon z started 2024-03-30T00:28:17.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:16 smithi142 ceph-mon[41779]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-30T00:28:17.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:16 smithi142 ceph-mon[41779]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T00:28:17.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:16 smithi142 ceph-mon[41779]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-30T00:28:17.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:16 smithi142 ceph-mon[41779]: from='mgr.? 172.21.15.161:0/823637675' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T00:28:18.173 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:17 smithi142 ceph-mon[41779]: mgrmap e14: x(active, since 67s), standbys: z 2024-03-30T00:28:18.173 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:17 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-30T00:28:18.173 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:17 smithi142 ceph-mon[41779]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:18.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:17 smithi161 ceph-mon[42221]: mgrmap e14: x(active, since 67s), standbys: z 2024-03-30T00:28:18.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:17 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-30T00:28:18.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:17 smithi161 ceph-mon[42221]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:18.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:17 smithi099 ceph-mon[38079]: mgrmap e14: x(active, since 67s), standbys: z 2024-03-30T00:28:18.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:17 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-30T00:28:18.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:17 smithi099 ceph-mon[38079]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:18.537 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:18.164+0000 7fcf57939200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T00:28:18.817 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:18.529+0000 7fcf57939200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T00:28:18.817 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:18.643+0000 7fcf57939200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T00:28:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:18.959+0000 7f2bd6000700 1 -- 172.21.15.99:0/274511570 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7f2bcc003b70 con 0x7f2bd8073810 2024-03-30T00:28:19.229 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:18 smithi099 ceph-mon[38079]: Standby manager daemon y started 2024-03-30T00:28:19.229 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:18 smithi099 ceph-mon[38079]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-30T00:28:19.229 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:18 smithi099 ceph-mon[38079]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T00:28:19.229 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:18 smithi099 ceph-mon[38079]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-30T00:28:19.229 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:18 smithi099 ceph-mon[38079]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T00:28:19.317 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:28:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y[43230]: 2024-03-30T00:28:18.895+0000 7fcf57939200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T00:28:19.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:18 smithi142 ceph-mon[41779]: Standby manager daemon y started 2024-03-30T00:28:19.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:18 smithi142 ceph-mon[41779]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-30T00:28:19.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:18 smithi142 ceph-mon[41779]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T00:28:19.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:18 smithi142 ceph-mon[41779]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-30T00:28:19.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:18 smithi142 ceph-mon[41779]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T00:28:19.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:18 smithi161 ceph-mon[42221]: Standby manager daemon y started 2024-03-30T00:28:19.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:18 smithi161 ceph-mon[42221]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-30T00:28:19.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:18 smithi161 ceph-mon[42221]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T00:28:19.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:18 smithi161 ceph-mon[42221]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-30T00:28:19.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:18 smithi161 ceph-mon[42221]: from='mgr.? 172.21.15.142:0/2102302668' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T00:28:20.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:19 smithi099 ceph-mon[38079]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:20.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:19 smithi099 ceph-mon[38079]: mgrmap e15: x(active, since 70s), standbys: y, z 2024-03-30T00:28:20.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:19 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-30T00:28:20.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:19 smithi142 ceph-mon[41779]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:20.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:19 smithi142 ceph-mon[41779]: mgrmap e15: x(active, since 70s), standbys: y, z 2024-03-30T00:28:20.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:19 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-30T00:28:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:19 smithi161 ceph-mon[42221]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:19 smithi161 ceph-mon[42221]: mgrmap e15: x(active, since 70s), standbys: y, z 2024-03-30T00:28:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:19 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-30T00:28:22.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:21 smithi099 ceph-mon[38079]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:22.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-30T00:28:22.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:22.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:21 smithi099 ceph-mon[38079]: Deploying daemon osd.0 on smithi099 2024-03-30T00:28:22.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:21 smithi142 ceph-mon[41779]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:22.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-30T00:28:22.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:22.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:21 smithi142 ceph-mon[41779]: Deploying daemon osd.0 on smithi099 2024-03-30T00:28:22.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:21 smithi161 ceph-mon[42221]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:22.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-30T00:28:22.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:22.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:21 smithi161 ceph-mon[42221]: Deploying daemon osd.0 on smithi099 2024-03-30T00:28:24.162 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:23 smithi099 ceph-mon[38079]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:24.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:23 smithi142 ceph-mon[41779]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:24.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:23 smithi161 ceph-mon[42221]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:26.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:25 smithi142 ceph-mon[41779]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:26.151 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:25 smithi099 ceph-mon[38079]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:25 smithi161 ceph-mon[42221]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:27.429 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:27.429 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:27.429 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:27.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:27.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:27.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:27.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:27.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:27.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:28.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:28 smithi142 ceph-mon[41779]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:28.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:28 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:28.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:28 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:28.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:28 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:28.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:28 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:28.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:28 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:28.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:28 smithi099 ceph-mon[38079]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:28.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:28 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:28.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:28 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:28.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:28 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:28.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:28 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:28.696 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:28 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:28.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:28 smithi161 ceph-mon[42221]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:28.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:28 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:28.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:28 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:28.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:28 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:28.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:28 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:28.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:28 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:28.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.892+0000 7f2bd6000700 1 -- 172.21.15.99:0/274511570 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2bd802cdf0 con 0x7f2bc00389d0 2024-03-30T00:28:28.900 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 0 on host 'smithi099' 2024-03-30T00:28:28.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 -- 172.21.15.99:0/274511570 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2bc00389d0 msgr2=0x7f2bc003ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:28.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2bc00389d0 0x7f2bc003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8074b10 tx=0x7f2bc400a3b0 comp rx=0 tx=0).stop 2024-03-30T00:28:28.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 -- 172.21.15.99:0/274511570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd8073810 msgr2=0x7f2bd8198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:28.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd8073810 0x7f2bd8198e90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0098e0 tx=0x7f2bcc005740 comp rx=0 tx=0).stop 2024-03-30T00:28:28.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 -- 172.21.15.99:0/274511570 shutdown_connections 2024-03-30T00:28:28.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2bc00389d0 0x7f2bc003ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:28.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2bd8108cd0 0x7f2bd81a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:28.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd8073810 0x7f2bd8198e90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:28.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 --2- 172.21.15.99:0/274511570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bd8074de0 0x7f2bd81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:28.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 -- 172.21.15.99:0/274511570 >> 172.21.15.99:0/274511570 conn(0x7f2bd80fc3c0 msgr2=0x7f2bd8104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:28:28.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 -- 172.21.15.99:0/274511570 shutdown_connections 2024-03-30T00:28:28.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:28:28.895+0000 7f2be0000700 1 -- 172.21.15.99:0/274511570 wait complete. 2024-03-30T00:28:29.287 DEBUG:teuthology.orchestra.run.smithi099:osd.0> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.0.service 2024-03-30T00:28:29.291 INFO:tasks.cephadm:Deploying osd.1 on smithi142 with /dev/vg_nvme/lv_4... 2024-03-30T00:28:29.291 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-30T00:28:29.361 INFO:journalctl@ceph.osd.0.smithi099.stdout:-- Logs begin at Sat 2024-03-30 00:10:25 UTC. -- 2024-03-30T00:28:29.649 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:28:29.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:29 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:29.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:29 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:29.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:29 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:29.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:29 smithi099 ceph-mon[38079]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:30.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:29 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:30.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:29 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:30.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:29 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:30.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:29 smithi142 ceph-mon[41779]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:30.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:29 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:30.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:29 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:30.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:29 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:30.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:29 smithi161 ceph-mon[42221]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:30.739 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:28:30 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-0[55748]: 2024-03-30T00:28:30.327+0000 7fee68a94640 -1 osd.0 0 log_to_monitors true 2024-03-30T00:28:31.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:30 smithi161 ceph-mon[42221]: from='osd.0 [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T00:28:31.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:30 smithi161 ceph-mon[42221]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T00:28:31.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:30 smithi099 ceph-mon[38079]: from='osd.0 [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T00:28:31.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:30 smithi099 ceph-mon[38079]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T00:28:31.278 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-30T00:28:31.278 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-30T00:28:31.278 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-03-30T00:28:31.278 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-03-30T00:28:31.278 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0307002 s, 342 MB/s 2024-03-30T00:28:31.278 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-03-30T00:28:31.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:30 smithi142 ceph-mon[41779]: from='osd.0 [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T00:28:31.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:30 smithi142 ceph-mon[41779]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T00:28:31.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-03-30T00:28:31.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:31 smithi099 ceph-mon[38079]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-30T00:28:31.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:31 smithi099 ceph-mon[38079]: osdmap e6: 1 total, 0 up, 1 in 2024-03-30T00:28:31.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:31.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:31 smithi099 ceph-mon[38079]: from='osd.0 [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:28:31.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:31 smithi099 ceph-mon[38079]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:28:31.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:31 smithi099 ceph-mon[38079]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:32.001 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:28:32.181 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:31 smithi142 ceph-mon[41779]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-30T00:28:32.181 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:31 smithi142 ceph-mon[41779]: osdmap e6: 1 total, 0 up, 1 in 2024-03-30T00:28:32.181 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:32.181 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:31 smithi142 ceph-mon[41779]: from='osd.0 [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:28:32.181 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:31 smithi142 ceph-mon[41779]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:28:32.181 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:31 smithi142 ceph-mon[41779]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:32.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:31 smithi161 ceph-mon[42221]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-30T00:28:32.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:31 smithi161 ceph-mon[42221]: osdmap e6: 1 total, 0 up, 1 in 2024-03-30T00:28:32.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:32.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:31 smithi161 ceph-mon[42221]: from='osd.0 [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:28:32.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:31 smithi161 ceph-mon[42221]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:28:32.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:31 smithi161 ceph-mon[42221]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:33.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.104+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2724805351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd78101f60 msgr2=0x7fbd781023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:33.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.104+0000 7fbd7ce00700 1 --2- 172.21.15.142:0/2724805351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd78101f60 0x7fbd781023e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009990 tx=0x7fbd64009ca0 comp rx=0 tx=0).stop 2024-03-30T00:28:33.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.104+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2724805351 shutdown_connections 2024-03-30T00:28:33.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.104+0000 7fbd7ce00700 1 --2- 172.21.15.142:0/2724805351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd78102920 0x7fbd78109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:33.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.104+0000 7fbd7ce00700 1 --2- 172.21.15.142:0/2724805351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd78100e00 0x7fbd78101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:33.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.104+0000 7fbd7ce00700 1 --2- 172.21.15.142:0/2724805351 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd78101f60 0x7fbd781023e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:33.113 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.104+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2724805351 >> 172.21.15.142:0/2724805351 conn(0x7fbd780fc3c0 msgr2=0x7fbd780fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:28:33.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.104+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2724805351 shutdown_connections 2024-03-30T00:28:33.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.105+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2724805351 wait complete. 2024-03-30T00:28:33.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd7ce00700 1 Processor -- start 2024-03-30T00:28:33.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd7ce00700 1 -- start start 2024-03-30T00:28:33.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd7ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd78100e00 0x7fbd78196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:33.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd7ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd78101f60 0x7fbd781971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:33.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd7ce00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd78102920 0x7fbd7819e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:33.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd7ce00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7819e7a0 con 0x7fbd78100e00 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd7ce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7819e8e0 con 0x7fbd78101f60 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd7ce00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7819eaa0 con 0x7fbd78102920 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd75400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd78101f60 0x7fbd781971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd75400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd78101f60 0x7fbd781971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:41362/0 (socket says 172.21.15.142:41362) 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd75400700 1 -- 172.21.15.142:0/2349007257 learned_addr learned my addr 172.21.15.142:0/2349007257 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.106+0000 7fbd76800700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd78102920 0x7fbd7819e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.107+0000 7fbd75e00700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd78100e00 0x7fbd78196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.107+0000 7fbd75400700 1 -- 172.21.15.142:0/2349007257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd78102920 msgr2=0x7fbd7819e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.107+0000 7fbd75400700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd78102920 0x7fbd7819e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.107+0000 7fbd75400700 1 -- 172.21.15.142:0/2349007257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd78100e00 msgr2=0x7fbd78196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.107+0000 7fbd75400700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd78100e00 0x7fbd78196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.107+0000 7fbd75400700 1 -- 172.21.15.142:0/2349007257 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd640095f0 con 0x7fbd78101f60 2024-03-30T00:28:33.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.107+0000 7fbd75400700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd78101f60 0x7fbd781971e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400b5c0 tx=0x7fbd64005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:28:33.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.107+0000 7fbd6ea00700 1 -- 172.21.15.142:0/2349007257 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6401d070 con 0x7fbd78101f60 2024-03-30T00:28:33.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.107+0000 7fbd6ea00700 1 -- 172.21.15.142:0/2349007257 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd64003ec0 con 0x7fbd78101f60 2024-03-30T00:28:33.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.107+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2349007257 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7819ed20 con 0x7fbd78101f60 2024-03-30T00:28:33.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.107+0000 7fbd6ea00700 1 -- 172.21.15.142:0/2349007257 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64022b10 con 0x7fbd78101f60 2024-03-30T00:28:33.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.108+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2349007257 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7819f190 con 0x7fbd78101f60 2024-03-30T00:28:33.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.109+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2349007257 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd781082a0 con 0x7fbd78101f60 2024-03-30T00:28:33.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.111+0000 7fbd6ea00700 1 -- 172.21.15.142:0/2349007257 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7fbd64004030 con 0x7fbd78101f60 2024-03-30T00:28:33.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.112+0000 7fbd6ea00700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbd5c0a1110 0x7fbd5c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:33.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.112+0000 7fbd6ea00700 1 -- 172.21.15.142:0/2349007257 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7fbd640cc320 con 0x7fbd78101f60 2024-03-30T00:28:33.120 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.112+0000 7fbd75e00700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbd5c0a1110 0x7fbd5c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:33.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.113+0000 7fbd75e00700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbd5c0a1110 0x7fbd5c0a35d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd78198130 tx=0x7fbd6800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:28:33.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.117+0000 7fbd6ea00700 1 -- 172.21.15.142:0/2349007257 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbd64097750 con 0x7fbd78101f60 2024-03-30T00:28:33.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:28:33.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: osdmap e7: 1 total, 0 up, 1 in 2024-03-30T00:28:33.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:33.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:33.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi099 2024-03-30T00:28:33.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:28:33.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi099 to 17270M 2024-03-30T00:28:33.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:33.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:33.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:32 smithi161 ceph-mon[42221]: from='osd.0 ' entity='osd.0' 2024-03-30T00:28:33.238 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-0[55748]: 2024-03-30T00:28:32.849+0000 7fee5c600700 -1 osd.0 0 waiting for initial osdmap 2024-03-30T00:28:33.238 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-0[55748]: 2024-03-30T00:28:32.863+0000 7fee52e00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:28:33.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:28:33.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: osdmap e7: 1 total, 0 up, 1 in 2024-03-30T00:28:33.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:33.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:33.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi099 2024-03-30T00:28:33.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:28:33.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi099 to 17270M 2024-03-30T00:28:33.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:33.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:33.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:32 smithi099 ceph-mon[38079]: from='osd.0 ' entity='osd.0' 2024-03-30T00:28:33.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:28:33.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: osdmap e7: 1 total, 0 up, 1 in 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi099 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi099 to 17270M 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:33.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:32 smithi142 ceph-mon[41779]: from='osd.0 ' entity='osd.0' 2024-03-30T00:28:33.333 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:33.325+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2349007257 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd780611f0 con 0x7fbd5c0a1110 2024-03-30T00:28:34.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:33 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:28:34.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:33 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:28:34.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:34.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:33 smithi161 ceph-mon[42221]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:34.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:33 smithi161 ceph-mon[42221]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:34.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:28:34.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:28:34.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:34.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:33 smithi161 ceph-mon[42221]: osd.0 [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857] boot 2024-03-30T00:28:34.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:33 smithi161 ceph-mon[42221]: osdmap e8: 1 total, 1 up, 1 in 2024-03-30T00:28:34.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:34.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:33 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:28:34.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:33 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:28:34.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:34.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:33 smithi099 ceph-mon[38079]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:34.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:33 smithi099 ceph-mon[38079]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:28:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:28:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:33 smithi099 ceph-mon[38079]: osd.0 [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857] boot 2024-03-30T00:28:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:33 smithi099 ceph-mon[38079]: osdmap e8: 1 total, 1 up, 1 in 2024-03-30T00:28:34.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:34.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:33 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:28:34.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:33 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:28:34.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:34.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:33 smithi142 ceph-mon[41779]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T00:28:34.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:33 smithi142 ceph-mon[41779]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:34.270 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:28:34.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:28:34.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:34.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:33 smithi142 ceph-mon[41779]: osd.0 [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857] boot 2024-03-30T00:28:34.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:33 smithi142 ceph-mon[41779]: osdmap e8: 1 total, 1 up, 1 in 2024-03-30T00:28:34.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T00:28:36.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:35 smithi161 ceph-mon[42221]: osdmap e9: 1 total, 1 up, 1 in 2024-03-30T00:28:36.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:35 smithi161 ceph-mon[42221]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:36.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:35 smithi161 ceph-mon[42221]: from='client.? 172.21.15.142:0/1744597171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b541e0e-4b00-4af5-92c5-adb75a359bbc"}]: dispatch 2024-03-30T00:28:36.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:35 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b541e0e-4b00-4af5-92c5-adb75a359bbc"}]: dispatch 2024-03-30T00:28:36.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:35 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b541e0e-4b00-4af5-92c5-adb75a359bbc"}]': finished 2024-03-30T00:28:36.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:35 smithi161 ceph-mon[42221]: osdmap e10: 2 total, 1 up, 2 in 2024-03-30T00:28:36.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:35 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:36.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:35 smithi099 ceph-mon[38079]: osdmap e9: 1 total, 1 up, 1 in 2024-03-30T00:28:36.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:35 smithi099 ceph-mon[38079]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:36.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:35 smithi099 ceph-mon[38079]: from='client.? 172.21.15.142:0/1744597171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b541e0e-4b00-4af5-92c5-adb75a359bbc"}]: dispatch 2024-03-30T00:28:36.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:35 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b541e0e-4b00-4af5-92c5-adb75a359bbc"}]: dispatch 2024-03-30T00:28:36.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:35 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b541e0e-4b00-4af5-92c5-adb75a359bbc"}]': finished 2024-03-30T00:28:36.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:35 smithi099 ceph-mon[38079]: osdmap e10: 2 total, 1 up, 2 in 2024-03-30T00:28:36.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:35 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:36.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:35 smithi142 ceph-mon[41779]: osdmap e9: 1 total, 1 up, 1 in 2024-03-30T00:28:36.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:35 smithi142 ceph-mon[41779]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:36.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:35 smithi142 ceph-mon[41779]: from='client.? 172.21.15.142:0/1744597171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b541e0e-4b00-4af5-92c5-adb75a359bbc"}]: dispatch 2024-03-30T00:28:36.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:35 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b541e0e-4b00-4af5-92c5-adb75a359bbc"}]: dispatch 2024-03-30T00:28:36.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:35 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b541e0e-4b00-4af5-92c5-adb75a359bbc"}]': finished 2024-03-30T00:28:36.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:35 smithi142 ceph-mon[41779]: osdmap e10: 2 total, 1 up, 2 in 2024-03-30T00:28:36.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:35 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:37.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:36 smithi161 ceph-mon[42221]: from='client.? 172.21.15.142:0/3001950520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:28:37.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:36 smithi099 ceph-mon[38079]: from='client.? 172.21.15.142:0/3001950520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:28:37.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:36 smithi142 ceph-mon[41779]: from='client.? 172.21.15.142:0/3001950520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:28:38.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:37 smithi161 ceph-mon[42221]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:38.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:37 smithi099 ceph-mon[38079]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:38.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:37 smithi142 ceph-mon[41779]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:40.177 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:40 smithi142 ceph-mon[41779]: pgmap v45: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:40 smithi161 ceph-mon[42221]: pgmap v45: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:40.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:40 smithi099 ceph-mon[38079]: pgmap v45: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:42.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:42 smithi142 ceph-mon[41779]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:42 smithi161 ceph-mon[42221]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:42 smithi099 ceph-mon[38079]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:43.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-30T00:28:43.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:43.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:43 smithi142 ceph-mon[41779]: Deploying daemon osd.1 on smithi142 2024-03-30T00:28:43.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-30T00:28:43.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:43.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:43 smithi161 ceph-mon[42221]: Deploying daemon osd.1 on smithi142 2024-03-30T00:28:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-30T00:28:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:43 smithi099 ceph-mon[38079]: Deploying daemon osd.1 on smithi142 2024-03-30T00:28:44.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:44 smithi142 ceph-mon[41779]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:44.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:44 smithi161 ceph-mon[42221]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:44.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:44 smithi099 ceph-mon[38079]: pgmap v47: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:46.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:45 smithi142 ceph-mon[41779]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:46.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:45 smithi161 ceph-mon[42221]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:46.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:45 smithi099 ceph-mon[38079]: pgmap v48: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:48.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:48 smithi142 ceph-mon[41779]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:48.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:48 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:48.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:48 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:48.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:48 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:48.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:48 smithi161 ceph-mon[42221]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:48.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:48 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:48.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:48 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:48.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:48 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:48.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:48 smithi099 ceph-mon[38079]: pgmap v49: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:48.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:48 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:48.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:48 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:48.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:48 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:50.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:49 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:50.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:49 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:50.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:49 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:50.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:49 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:50.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:49 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:50.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:49 smithi142 ceph-mon[41779]: pgmap v50: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:50.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.189+0000 7fbd6ea00700 1 -- 172.21.15.142:0/2349007257 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbd780611f0 con 0x7fbd5c0a1110 2024-03-30T00:28:50.198 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 1 on host 'smithi142' 2024-03-30T00:28:50.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2349007257 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbd5c0a1110 msgr2=0x7fbd5c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:50.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbd5c0a1110 0x7fbd5c0a35d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd78198130 tx=0x7fbd6800b410 comp rx=0 tx=0).stop 2024-03-30T00:28:50.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2349007257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd78101f60 msgr2=0x7fbd781971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:50.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd78101f60 0x7fbd781971e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400b5c0 tx=0x7fbd64005740 comp rx=0 tx=0).stop 2024-03-30T00:28:50.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2349007257 shutdown_connections 2024-03-30T00:28:50.200 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd78102920 0x7fbd7819e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:50.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd78100e00 0x7fbd78196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:50.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbd5c0a1110 0x7fbd5c0a35d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:50.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 --2- 172.21.15.142:0/2349007257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd78101f60 0x7fbd781971e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:50.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2349007257 >> 172.21.15.142:0/2349007257 conn(0x7fbd780fc3c0 msgr2=0x7fbd781053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:28:50.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2349007257 shutdown_connections 2024-03-30T00:28:50.201 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:28:50.192+0000 7fbd7ce00700 1 -- 172.21.15.142:0/2349007257 wait complete. 2024-03-30T00:28:50.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:49 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:50.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:49 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:50.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:49 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:50.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:49 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:50.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:49 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:50.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:49 smithi161 ceph-mon[42221]: pgmap v50: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:50.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:49 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:50.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:49 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:50.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:49 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:50.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:49 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:50.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:49 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:50.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:49 smithi099 ceph-mon[38079]: pgmap v50: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:50.567 DEBUG:teuthology.orchestra.run.smithi142:osd.1> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.1.service 2024-03-30T00:28:50.572 INFO:tasks.cephadm:Deploying osd.2 on smithi161 with /dev/vg_nvme/lv_4... 2024-03-30T00:28:50.572 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-30T00:28:50.652 INFO:journalctl@ceph.osd.1.smithi142.stdout:-- Logs begin at Sat 2024-03-30 00:10:40 UTC. -- 2024-03-30T00:28:50.929 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:28:51.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:51 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:51.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:51 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:51.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:51 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:51.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:51 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:51.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:51 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:51.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:51 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:51.550 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:51 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:51.550 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:51 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:51.550 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:51 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:28:51.551 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:28:51 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-1[46863]: 2024-03-30T00:28:51.392+0000 7f3e7df8e640 -1 osd.1 0 log_to_monitors true 2024-03-30T00:28:52.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:52 smithi161 ceph-mon[42221]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:52.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:52 smithi161 ceph-mon[42221]: from='osd.1 [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T00:28:52.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:52 smithi161 ceph-mon[42221]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T00:28:52.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:52 smithi099 ceph-mon[38079]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:52.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:52 smithi099 ceph-mon[38079]: from='osd.1 [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T00:28:52.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:52 smithi099 ceph-mon[38079]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T00:28:52.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:52 smithi142 ceph-mon[41779]: pgmap v51: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:52.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:52 smithi142 ceph-mon[41779]: from='osd.1 [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T00:28:52.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:52 smithi142 ceph-mon[41779]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T00:28:52.713 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-30T00:28:52.720 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-30T00:28:52.720 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-03-30T00:28:52.720 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-03-30T00:28:52.720 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0374257 s, 280 MB/s 2024-03-30T00:28:52.721 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-03-30T00:28:53.105 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-03-30T00:28:53.456 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:28:53.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:53 smithi161 ceph-mon[42221]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-30T00:28:53.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:53 smithi161 ceph-mon[42221]: osdmap e11: 2 total, 1 up, 2 in 2024-03-30T00:28:53.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:53 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:53.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:53 smithi161 ceph-mon[42221]: from='osd.1 [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:28:53.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:53 smithi161 ceph-mon[42221]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:28:53.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:53 smithi099 ceph-mon[38079]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-30T00:28:53.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:53 smithi099 ceph-mon[38079]: osdmap e11: 2 total, 1 up, 2 in 2024-03-30T00:28:53.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:53 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:53.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:53 smithi099 ceph-mon[38079]: from='osd.1 [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:28:53.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:53 smithi099 ceph-mon[38079]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:28:53.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:53 smithi142 ceph-mon[41779]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-30T00:28:53.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:53 smithi142 ceph-mon[41779]: osdmap e11: 2 total, 1 up, 2 in 2024-03-30T00:28:53.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:53 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:53.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:53 smithi142 ceph-mon[41779]: from='osd.1 [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:28:53.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:53 smithi142 ceph-mon[41779]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:28:54.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:54.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:28:54.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: osdmap e12: 2 total, 1 up, 2 in 2024-03-30T00:28:54.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:54.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:54.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi142 2024-03-30T00:28:54.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:28:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi142 to 17270M 2024-03-30T00:28:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:54 smithi161 ceph-mon[42221]: from='osd.1 ' entity='osd.1' 2024-03-30T00:28:54.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:54.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:28:54.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: osdmap e12: 2 total, 1 up, 2 in 2024-03-30T00:28:54.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:54.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:54.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi142 2024-03-30T00:28:54.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:28:54.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi142 to 17270M 2024-03-30T00:28:54.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:54.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:54.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:54 smithi099 ceph-mon[38079]: from='osd.1 ' entity='osd.1' 2024-03-30T00:28:54.567 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-1[46863]: 2024-03-30T00:28:54.137+0000 7f3e71c00700 -1 osd.1 0 waiting for initial osdmap 2024-03-30T00:28:54.568 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-1[46863]: 2024-03-30T00:28:54.150+0000 7f3e68400700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:28:54.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: pgmap v53: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:54.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:28:54.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: osdmap e12: 2 total, 1 up, 2 in 2024-03-30T00:28:54.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:54.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:54.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi142 2024-03-30T00:28:54.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:28:54.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi142 to 17270M 2024-03-30T00:28:54.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:54.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:28:54.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:28:54.569 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:54 smithi142 ceph-mon[41779]: from='osd.1 ' entity='osd.1' 2024-03-30T00:28:54.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.603+0000 7f454ee00700 1 -- 172.21.15.161:0/4049514222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4548101f60 msgr2=0x7f45481023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:54.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.603+0000 7f454ee00700 1 --2- 172.21.15.161:0/4049514222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4548101f60 0x7f45481023e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4534009990 tx=0x7f4534009ca0 comp rx=0 tx=0).stop 2024-03-30T00:28:54.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.603+0000 7f454ee00700 1 -- 172.21.15.161:0/4049514222 shutdown_connections 2024-03-30T00:28:54.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.603+0000 7f454ee00700 1 --2- 172.21.15.161:0/4049514222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4548102920 0x7f4548109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:54.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.603+0000 7f454ee00700 1 --2- 172.21.15.161:0/4049514222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4548100e00 0x7f4548101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:54.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.603+0000 7f454ee00700 1 --2- 172.21.15.161:0/4049514222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4548101f60 0x7f45481023e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:54.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.603+0000 7f454ee00700 1 -- 172.21.15.161:0/4049514222 >> 172.21.15.161:0/4049514222 conn(0x7f45480fc3c0 msgr2=0x7f45480fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:28:54.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.604+0000 7f454ee00700 1 -- 172.21.15.161:0/4049514222 shutdown_connections 2024-03-30T00:28:54.612 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.604+0000 7f454ee00700 1 -- 172.21.15.161:0/4049514222 wait complete. 2024-03-30T00:28:54.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.605+0000 7f454ee00700 1 Processor -- start 2024-03-30T00:28:54.614 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.605+0000 7f454ee00700 1 -- start start 2024-03-30T00:28:54.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.605+0000 7f454ee00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4548100e00 0x7f4548196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:54.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.605+0000 7f454ee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4548101f60 0x7f45481971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:54.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.605+0000 7f454ee00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4548102920 0x7f454819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:54.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.605+0000 7f454ee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454819e770 con 0x7f4548101f60 2024-03-30T00:28:54.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.605+0000 7f454ee00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454819e8b0 con 0x7f4548100e00 2024-03-30T00:28:54.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.605+0000 7f454ee00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454819ea40 con 0x7f4548102920 2024-03-30T00:28:54.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.605+0000 7f454ce00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4548102920 0x7f454819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:54.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.605+0000 7f454ce00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4548102920 0x7f454819e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:54576/0 (socket says 172.21.15.161:54576) 2024-03-30T00:28:54.615 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.605+0000 7f454ce00700 1 -- 172.21.15.161:0/2830228887 learned_addr learned my addr 172.21.15.161:0/2830228887 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f4547400700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4548101f60 0x7f45481971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f4547e00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4548100e00 0x7f4548196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f454ce00700 1 -- 172.21.15.161:0/2830228887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4548100e00 msgr2=0x7f4548196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f454ce00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4548100e00 0x7f4548196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f454ce00700 1 -- 172.21.15.161:0/2830228887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4548101f60 msgr2=0x7f45481971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f454ce00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4548101f60 0x7f45481971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f454ce00700 1 -- 172.21.15.161:0/2830228887 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340095f0 con 0x7f4548102920 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f454ce00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4548102920 0x7f454819e1a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4540007f10 tx=0x7f454000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f4544c00700 1 -- 172.21.15.161:0/2830228887 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000dd20 con 0x7f4548102920 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f4544c00700 1 -- 172.21.15.161:0/2830228887 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f454000f040 con 0x7f4548102920 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f454ee00700 1 -- 172.21.15.161:0/2830228887 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454819ed20 con 0x7f4548102920 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.606+0000 7f4544c00700 1 -- 172.21.15.161:0/2830228887 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4540004870 con 0x7f4548102920 2024-03-30T00:28:54.616 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.607+0000 7f454ee00700 1 -- 172.21.15.161:0/2830228887 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454819f210 con 0x7f4548102920 2024-03-30T00:28:54.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.608+0000 7f454ee00700 1 -- 172.21.15.161:0/2830228887 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4548108370 con 0x7f4548102920 2024-03-30T00:28:54.617 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.609+0000 7f4544c00700 1 -- 172.21.15.161:0/2830228887 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7f4540003a00 con 0x7f4548102920 2024-03-30T00:28:54.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.611+0000 7f4544c00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f452c0a1110 0x7f452c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:28:54.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.612+0000 7f4544c00700 1 -- 172.21.15.161:0/2830228887 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7f45400ca750 con 0x7f4548102920 2024-03-30T00:28:54.619 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.612+0000 7f4547e00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f452c0a1110 0x7f452c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:28:54.620 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.612+0000 7f4547e00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f452c0a1110 0x7f452c0a35d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4538009c60 tx=0x7f4538009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:28:54.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.616+0000 7f4544c00700 1 -- 172.21.15.161:0/2830228887 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f45400957b0 con 0x7f4548102920 2024-03-30T00:28:54.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:28:54.826+0000 7f454ee00700 1 -- 172.21.15.161:0/2830228887 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f454802cd70 con 0x7f452c0a1110 2024-03-30T00:28:55.444 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:55 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:28:55.444 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:55 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:28:55.444 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:55 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:55.444 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:55 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:28:55.445 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:55 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:28:55.445 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:55 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:55.445 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:55 smithi161 ceph-mon[42221]: osd.1 [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361] boot 2024-03-30T00:28:55.445 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:55 smithi161 ceph-mon[42221]: osdmap e13: 2 total, 2 up, 2 in 2024-03-30T00:28:55.445 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:55 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:55.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:55 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:28:55.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:55 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:28:55.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:55 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:55.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:55 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:28:55.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:55 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:28:55.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:55 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:55.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:55 smithi099 ceph-mon[38079]: osd.1 [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361] boot 2024-03-30T00:28:55.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:55 smithi099 ceph-mon[38079]: osdmap e13: 2 total, 2 up, 2 in 2024-03-30T00:28:55.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:55 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:55.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:55 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:28:55.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:55 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:28:55.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:55 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:55.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:55 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:28:55.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:55 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:28:55.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:55 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:28:55.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:55 smithi142 ceph-mon[41779]: osd.1 [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361] boot 2024-03-30T00:28:55.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:55 smithi142 ceph-mon[41779]: osdmap e13: 2 total, 2 up, 2 in 2024-03-30T00:28:55.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:55 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T00:28:56.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:56 smithi161 ceph-mon[42221]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:56.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:56 smithi161 ceph-mon[42221]: pgmap v55: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:56.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:56 smithi099 ceph-mon[38079]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:56.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:56 smithi099 ceph-mon[38079]: pgmap v55: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:56.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:56 smithi142 ceph-mon[41779]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:28:56.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:56 smithi142 ceph-mon[41779]: pgmap v55: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-03-30T00:28:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:57 smithi161 ceph-mon[42221]: osdmap e14: 2 total, 2 up, 2 in 2024-03-30T00:28:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:57 smithi161 ceph-mon[42221]: from='client.? 172.21.15.161:0/3201056570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0faf760-1f43-4799-bc8d-bdc2f8ba5386"}]: dispatch 2024-03-30T00:28:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:57 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0faf760-1f43-4799-bc8d-bdc2f8ba5386"}]: dispatch 2024-03-30T00:28:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:57 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0faf760-1f43-4799-bc8d-bdc2f8ba5386"}]': finished 2024-03-30T00:28:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:57 smithi161 ceph-mon[42221]: osdmap e15: 3 total, 2 up, 3 in 2024-03-30T00:28:57.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:57 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:28:57.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:57 smithi099 ceph-mon[38079]: osdmap e14: 2 total, 2 up, 2 in 2024-03-30T00:28:57.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:57 smithi099 ceph-mon[38079]: from='client.? 172.21.15.161:0/3201056570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0faf760-1f43-4799-bc8d-bdc2f8ba5386"}]: dispatch 2024-03-30T00:28:57.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:57 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0faf760-1f43-4799-bc8d-bdc2f8ba5386"}]: dispatch 2024-03-30T00:28:57.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:57 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0faf760-1f43-4799-bc8d-bdc2f8ba5386"}]': finished 2024-03-30T00:28:57.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:57 smithi099 ceph-mon[38079]: osdmap e15: 3 total, 2 up, 3 in 2024-03-30T00:28:57.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:57 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:28:57.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:57 smithi142 ceph-mon[41779]: osdmap e14: 2 total, 2 up, 2 in 2024-03-30T00:28:57.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:57 smithi142 ceph-mon[41779]: from='client.? 172.21.15.161:0/3201056570' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0faf760-1f43-4799-bc8d-bdc2f8ba5386"}]: dispatch 2024-03-30T00:28:57.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:57 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0faf760-1f43-4799-bc8d-bdc2f8ba5386"}]: dispatch 2024-03-30T00:28:57.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:57 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0faf760-1f43-4799-bc8d-bdc2f8ba5386"}]': finished 2024-03-30T00:28:57.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:57 smithi142 ceph-mon[41779]: osdmap e15: 3 total, 2 up, 3 in 2024-03-30T00:28:57.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:57 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:28:58.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:58 smithi099 ceph-mon[38079]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:28:58.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:28:58 smithi099 ceph-mon[38079]: from='client.? 172.21.15.161:0/906128744' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:28:58.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:58 smithi142 ceph-mon[41779]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:28:58.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:28:58 smithi142 ceph-mon[41779]: from='client.? 172.21.15.161:0/906128744' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:28:58.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:58 smithi161 ceph-mon[42221]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:28:58.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:28:58 smithi161 ceph-mon[42221]: from='client.? 172.21.15.161:0/906128744' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:29:00.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:00 smithi099 ceph-mon[38079]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:00.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:00 smithi142 ceph-mon[41779]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:00.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:00 smithi161 ceph-mon[42221]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:02.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:02 smithi099 ceph-mon[38079]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:02.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:02 smithi142 ceph-mon[41779]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:02.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:02 smithi161 ceph-mon[42221]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:04.558 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:04 smithi161 ceph-mon[42221]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:04.559 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-30T00:29:04.559 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:04.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:04 smithi142 ceph-mon[41779]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:04.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-30T00:29:04.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:04.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:04 smithi099 ceph-mon[38079]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:04.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-30T00:29:04.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:05.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:05 smithi142 ceph-mon[41779]: Deploying daemon osd.2 on smithi161 2024-03-30T00:29:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:05 smithi161 ceph-mon[42221]: Deploying daemon osd.2 on smithi161 2024-03-30T00:29:05.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:05 smithi099 ceph-mon[38079]: Deploying daemon osd.2 on smithi161 2024-03-30T00:29:06.373 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:06 smithi161 ceph-mon[42221]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:06.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:06 smithi142 ceph-mon[41779]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:06.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:06 smithi099 ceph-mon[38079]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:08.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:08 smithi161 ceph-mon[42221]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:08.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:08 smithi142 ceph-mon[41779]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:08.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:08 smithi099 ceph-mon[38079]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:10.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:10.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:10.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:10.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:09 smithi161 ceph-mon[42221]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:10.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:10.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:10.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:10.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:09 smithi099 ceph-mon[38079]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:10.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:10.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:10.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:10.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:09 smithi142 ceph-mon[41779]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:11.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:11.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:11.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:11.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:11.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:11.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:11 smithi161 ceph-mon[42221]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:11.799 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 2 on host 'smithi161' 2024-03-30T00:29:11.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.791+0000 7f4544c00700 1 -- 172.21.15.161:0/2830228887 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f454802cd70 con 0x7f452c0a1110 2024-03-30T00:29:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 -- 172.21.15.161:0/2830228887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f452c0a1110 msgr2=0x7f452c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f452c0a1110 0x7f452c0a35d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4538009c60 tx=0x7f4538009260 comp rx=0 tx=0).stop 2024-03-30T00:29:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 -- 172.21.15.161:0/2830228887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4548102920 msgr2=0x7f454819e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4548102920 0x7f454819e1a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4540007f10 tx=0x7f454000d3b0 comp rx=0 tx=0).stop 2024-03-30T00:29:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 -- 172.21.15.161:0/2830228887 shutdown_connections 2024-03-30T00:29:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4548102920 0x7f454819e1a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f452c0a1110 0x7f452c0a35d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4548100e00 0x7f4548196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:11.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 --2- 172.21.15.161:0/2830228887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4548101f60 0x7f45481971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:11.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 -- 172.21.15.161:0/2830228887 >> 172.21.15.161:0/2830228887 conn(0x7f45480fc3c0 msgr2=0x7f45481054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:29:11.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 -- 172.21.15.161:0/2830228887 shutdown_connections 2024-03-30T00:29:11.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:11.794+0000 7f454ee00700 1 -- 172.21.15.161:0/2830228887 wait complete. 2024-03-30T00:29:11.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:11.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:11.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:11.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:11.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:11.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:11 smithi142 ceph-mon[41779]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:11.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:11.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:11.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:11.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:11.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:11.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:11 smithi099 ceph-mon[38079]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:12.190 DEBUG:teuthology.orchestra.run.smithi161:osd.2> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.2.service 2024-03-30T00:29:12.195 INFO:tasks.cephadm:Deploying osd.3 on smithi099 with /dev/vg_nvme/lv_3... 2024-03-30T00:29:12.195 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-30T00:29:12.277 INFO:journalctl@ceph.osd.2.smithi161.stdout:-- Logs begin at Sat 2024-03-30 00:10:26 UTC. -- 2024-03-30T00:29:12.556 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:29:12.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:12.865 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:12.866 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:12.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:12.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:12.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:13.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:13.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:13.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:13.219 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:29:12 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-2[47835]: 2024-03-30T00:29:12.996+0000 7fd1d7bce640 -1 osd.2 0 log_to_monitors true 2024-03-30T00:29:14.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:13 smithi142 ceph-mon[41779]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:14.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:13 smithi142 ceph-mon[41779]: from='osd.2 [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T00:29:14.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:13 smithi142 ceph-mon[41779]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T00:29:14.191 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-30T00:29:14.192 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-30T00:29:14.192 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-03-30T00:29:14.192 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-03-30T00:29:14.192 INFO:teuthology.orchestra.run.smithi099.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0292849 s, 358 MB/s 2024-03-30T00:29:14.192 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-03-30T00:29:14.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:13 smithi161 ceph-mon[42221]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:14.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:13 smithi161 ceph-mon[42221]: from='osd.2 [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T00:29:14.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:13 smithi161 ceph-mon[42221]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T00:29:14.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:13 smithi099 ceph-mon[38079]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:14.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:13 smithi099 ceph-mon[38079]: from='osd.2 [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T00:29:14.230 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:13 smithi099 ceph-mon[38079]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T00:29:14.581 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2024-03-30T00:29:14.932 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:29:14.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:14 smithi099 ceph-mon[38079]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-30T00:29:14.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:14 smithi099 ceph-mon[38079]: osdmap e16: 3 total, 2 up, 3 in 2024-03-30T00:29:14.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:14.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:14 smithi099 ceph-mon[38079]: from='osd.2 [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:29:14.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:14 smithi099 ceph-mon[38079]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:29:15.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:14 smithi142 ceph-mon[41779]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-30T00:29:15.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:14 smithi142 ceph-mon[41779]: osdmap e16: 3 total, 2 up, 3 in 2024-03-30T00:29:15.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:15.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:14 smithi142 ceph-mon[41779]: from='osd.2 [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:29:15.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:14 smithi142 ceph-mon[41779]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:29:15.111 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:14 smithi161 ceph-mon[42221]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-30T00:29:15.111 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:14 smithi161 ceph-mon[42221]: osdmap e16: 3 total, 2 up, 3 in 2024-03-30T00:29:15.111 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:15.111 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:14 smithi161 ceph-mon[42221]: from='osd.2 [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:29:15.112 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:14 smithi161 ceph-mon[42221]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:29:16.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.993+0000 7fa2fca00700 1 -- 172.21.15.99:0/3586633045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8073810 msgr2=0x7fa2f8073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:16.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.993+0000 7fa2fca00700 1 --2- 172.21.15.99:0/3586633045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8073810 0x7fa2f8073c90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009990 tx=0x7fa2e0009ca0 comp rx=0 tx=0).stop 2024-03-30T00:29:16.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.993+0000 7fa2fca00700 1 -- 172.21.15.99:0/3586633045 shutdown_connections 2024-03-30T00:29:16.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.993+0000 7fa2fca00700 1 --2- 172.21.15.99:0/3586633045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f810d200 0x7fa2f810f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:16.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.993+0000 7fa2fca00700 1 --2- 172.21.15.99:0/3586633045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f8074de0 0x7fa2f8073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:16.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.993+0000 7fa2fca00700 1 --2- 172.21.15.99:0/3586633045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8073810 0x7fa2f8073c90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:16.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.993+0000 7fa2fca00700 1 -- 172.21.15.99:0/3586633045 >> 172.21.15.99:0/3586633045 conn(0x7fa2f80fc1c0 msgr2=0x7fa2f80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:29:16.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.993+0000 7fa2fca00700 1 -- 172.21.15.99:0/3586633045 shutdown_connections 2024-03-30T00:29:16.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.994+0000 7fa2fca00700 1 -- 172.21.15.99:0/3586633045 wait complete. 2024-03-30T00:29:16.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.994+0000 7fa2fca00700 1 Processor -- start 2024-03-30T00:29:16.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2fca00700 1 -- start start 2024-03-30T00:29:16.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2fca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8073810 0x7fa2f819d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:16.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2fca00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f8074de0 0x7fa2f819d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:16.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2fca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f810d200 0x7fa2f81a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:16.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2fca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f81a4c80 con 0x7fa2f8073810 2024-03-30T00:29:16.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2fca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f81a4dc0 con 0x7fa2f810d200 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2fca00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f81a4f50 con 0x7fa2f8074de0 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2ef400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8073810 0x7fa2f819d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2ef400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8073810 0x7fa2f819d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42930/0 (socket says 172.21.15.99:42930) 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2ef400700 1 -- 172.21.15.99:0/2734478671 learned_addr learned my addr 172.21.15.99:0/2734478671 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2f7400700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f8074de0 0x7fa2f819d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.995+0000 7fa2f7e00700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f810d200 0x7fa2f81a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2ef400700 1 -- 172.21.15.99:0/2734478671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f8074de0 msgr2=0x7fa2f819d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2ef400700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f8074de0 0x7fa2f819d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2ef400700 1 -- 172.21.15.99:0/2734478671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f810d200 msgr2=0x7fa2f81a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2ef400700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f810d200 0x7fa2f81a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2ef400700 1 -- 172.21.15.99:0/2734478671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00095f0 con 0x7fa2f8073810 2024-03-30T00:29:16.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2f7400700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f8074de0 0x7fa2f819d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:29:16.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2ef400700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8073810 0x7fa2f819d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800d9b0 tx=0x7fa2e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:29:16.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2f4c00700 1 -- 172.21.15.99:0/2734478671 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8009930 con 0x7fa2f8073810 2024-03-30T00:29:16.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2f4c00700 1 -- 172.21.15.99:0/2734478671 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2e8010460 con 0x7fa2f8073810 2024-03-30T00:29:16.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2f4c00700 1 -- 172.21.15.99:0/2734478671 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e800f620 con 0x7fa2f8073810 2024-03-30T00:29:16.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2fca00700 1 -- 172.21.15.99:0/2734478671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f81a5230 con 0x7fa2f8073810 2024-03-30T00:29:16.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.996+0000 7fa2fca00700 1 -- 172.21.15.99:0/2734478671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f81a56e0 con 0x7fa2f8073810 2024-03-30T00:29:16.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.998+0000 7fa2fca00700 1 -- 172.21.15.99:0/2734478671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f8103460 con 0x7fa2f8073810 2024-03-30T00:29:16.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:15.999+0000 7fa2f4c00700 1 -- 172.21.15.99:0/2734478671 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7fa2e80105d0 con 0x7fa2f8073810 2024-03-30T00:29:16.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:16.001+0000 7fa2f4c00700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2dc0a1110 0x7fa2dc0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:16.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:16.001+0000 7fa2f4c00700 1 -- 172.21.15.99:0/2734478671 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7fa2e80cab50 con 0x7fa2f8073810 2024-03-30T00:29:16.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:16.001+0000 7fa2f7400700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2dc0a1110 0x7fa2dc0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:16.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:16.002+0000 7fa2f7400700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2dc0a1110 0x7fa2dc0a35d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0006010 tx=0x7fa2e000b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:29:16.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:16.005+0000 7fa2f4c00700 1 -- 172.21.15.99:0/2734478671 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa2e80cce60 con 0x7fa2f8073810 2024-03-30T00:29:16.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:29:16.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: osdmap e17: 3 total, 2 up, 3 in 2024-03-30T00:29:16.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:16.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:16.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi161 2024-03-30T00:29:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi161 to 17270M 2024-03-30T00:29:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:15 smithi142 ceph-mon[41779]: from='osd.2 ' entity='osd.2' 2024-03-30T00:29:16.220 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-2[47835]: 2024-03-30T00:29:15.736+0000 7fd1cb800700 -1 osd.2 0 waiting for initial osdmap 2024-03-30T00:29:16.220 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-2[47835]: 2024-03-30T00:29:15.753+0000 7fd1c2000700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:29:16.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:29:16.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: osdmap e17: 3 total, 2 up, 3 in 2024-03-30T00:29:16.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:16.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi161 2024-03-30T00:29:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi161 to 17270M 2024-03-30T00:29:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:16.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.222 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:15 smithi161 ceph-mon[42221]: from='osd.2 ' entity='osd.2' 2024-03-30T00:29:16.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:16.221+0000 7fa2fca00700 1 -- 172.21.15.99:0/2734478671 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2f802cd20 con 0x7fa2dc0a1110 2024-03-30T00:29:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:29:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: osdmap e17: 3 total, 2 up, 3 in 2024-03-30T00:29:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-30T00:29:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi161 2024-03-30T00:29:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:16.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi161 to 17270M 2024-03-30T00:29:16.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:16.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:16.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:16.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:15 smithi099 ceph-mon[38079]: from='osd.2 ' entity='osd.2' 2024-03-30T00:29:17.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:16 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:29:17.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:16 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:29:17.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:17.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:16 smithi142 ceph-mon[41779]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:29:17.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:29:17.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:29:17.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:17.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:16 smithi142 ceph-mon[41779]: osd.2 [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041] boot 2024-03-30T00:29:17.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:16 smithi142 ceph-mon[41779]: osdmap e18: 3 total, 3 up, 3 in 2024-03-30T00:29:17.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:17.185 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:16 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:29:17.185 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:16 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:29:17.185 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:17.185 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:16 smithi099 ceph-mon[38079]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:29:17.186 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:29:17.186 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:29:17.186 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:17.186 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:16 smithi099 ceph-mon[38079]: osd.2 [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041] boot 2024-03-30T00:29:17.186 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:16 smithi099 ceph-mon[38079]: osdmap e18: 3 total, 3 up, 3 in 2024-03-30T00:29:17.186 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:16 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:29:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:16 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:29:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:16 smithi161 ceph-mon[42221]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:29:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:29:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:29:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:16 smithi161 ceph-mon[42221]: osd.2 [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041] boot 2024-03-30T00:29:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:16 smithi161 ceph-mon[42221]: osdmap e18: 3 total, 3 up, 3 in 2024-03-30T00:29:17.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T00:29:17.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:17 smithi099 ceph-mon[38079]: pgmap v72: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:17.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:17 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-30T00:29:18.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:17 smithi142 ceph-mon[41779]: pgmap v72: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:18.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:17 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-30T00:29:18.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:17 smithi161 ceph-mon[42221]: pgmap v72: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:18.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:17 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-30T00:29:19.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:18 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-30T00:29:19.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:18 smithi142 ceph-mon[41779]: osdmap e19: 3 total, 3 up, 3 in 2024-03-30T00:29:19.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:18 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T00:29:19.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:18 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/967531216' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a75bf9a-8f16-4d28-9685-cdbc9a5a9325"}]: dispatch 2024-03-30T00:29:19.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:18 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-30T00:29:19.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:18 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/967531216' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a75bf9a-8f16-4d28-9685-cdbc9a5a9325"}]': finished 2024-03-30T00:29:19.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:18 smithi142 ceph-mon[41779]: osdmap e20: 4 total, 3 up, 4 in 2024-03-30T00:29:19.069 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:18 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:18 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-30T00:29:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:18 smithi161 ceph-mon[42221]: osdmap e19: 3 total, 3 up, 3 in 2024-03-30T00:29:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:18 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T00:29:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:18 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/967531216' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a75bf9a-8f16-4d28-9685-cdbc9a5a9325"}]: dispatch 2024-03-30T00:29:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:18 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-30T00:29:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:18 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/967531216' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a75bf9a-8f16-4d28-9685-cdbc9a5a9325"}]': finished 2024-03-30T00:29:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:18 smithi161 ceph-mon[42221]: osdmap e20: 4 total, 3 up, 4 in 2024-03-30T00:29:19.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:18 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:19.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:18 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-30T00:29:19.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:18 smithi099 ceph-mon[38079]: osdmap e19: 3 total, 3 up, 3 in 2024-03-30T00:29:19.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:18 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T00:29:19.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:18 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/967531216' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a75bf9a-8f16-4d28-9685-cdbc9a5a9325"}]: dispatch 2024-03-30T00:29:19.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:18 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-30T00:29:19.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:18 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/967531216' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a75bf9a-8f16-4d28-9685-cdbc9a5a9325"}]': finished 2024-03-30T00:29:19.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:18 smithi099 ceph-mon[38079]: osdmap e20: 4 total, 3 up, 4 in 2024-03-30T00:29:19.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:18 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:20.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:19 smithi161 ceph-mon[42221]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:20.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:19 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2734738402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:29:20.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:19 smithi099 ceph-mon[38079]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:20.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:19 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2734738402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:29:20.239 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:29:19 smithi099 sudo[61926]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T00:29:20.239 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:29:19 smithi099 sudo[61926]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:20.239 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:29:19 smithi099 sudo[61926]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:20.239 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:29:19 smithi099 sudo[61926]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:20.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:19 smithi142 ceph-mon[41779]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:20.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:19 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2734738402' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:29:20.738 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:29:20 smithi099 sudo[62145]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T00:29:20.738 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:29:20 smithi099 sudo[62145]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:20.738 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:29:20 smithi099 sudo[62145]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:20.738 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:29:20 smithi099 sudo[62145]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:21.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:20 smithi161 ceph-mon[42221]: osdmap e21: 4 total, 3 up, 4 in 2024-03-30T00:29:21.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:21.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:21.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:20 smithi099 ceph-mon[38079]: osdmap e21: 4 total, 3 up, 4 in 2024-03-30T00:29:21.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:21.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:21.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:20 smithi142 ceph-mon[41779]: osdmap e21: 4 total, 3 up, 4 in 2024-03-30T00:29:21.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:21.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:21.318 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:29:20 smithi142 sudo[51542]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T00:29:21.318 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:29:20 smithi142 sudo[51542]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:21.318 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:29:20 smithi142 sudo[51542]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:21.318 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:29:21 smithi142 sudo[51542]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:21.818 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:29:21 smithi142 sudo[51545]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T00:29:21.818 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:29:21 smithi142 sudo[51545]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:21.818 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:29:21 smithi142 sudo[51545]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:21.818 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:29:21 smithi142 sudo[51545]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:21.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:21.872+0000 7fa2f4c00700 1 -- 172.21.15.99:0/2734478671 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa2e8095d60 con 0x7fa2f8073810 2024-03-30T00:29:22.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:21 smithi161 ceph-mon[42221]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:22.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:22.220 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:29:22 smithi161 sudo[53188]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T00:29:22.220 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:29:22 smithi161 sudo[53188]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:22.220 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:29:22 smithi161 sudo[53188]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:22.220 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:29:22 smithi161 sudo[53188]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:22.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:21 smithi099 ceph-mon[38079]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:22.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:22.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:21 smithi142 ceph-mon[41779]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:22.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:22.970 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:29:22 smithi161 sudo[53191]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T00:29:22.970 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:29:22 smithi161 sudo[53191]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:22.970 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:29:22 smithi161 sudo[53191]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:22.970 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:29:22 smithi161 sudo[53191]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:23.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:23 smithi099 sudo[63237]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T00:29:23.524 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:23 smithi099 sudo[63237]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:23.524 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:23 smithi099 sudo[63237]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:23.524 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:23 smithi099 sudo[63237]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:24.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:23 smithi099 sudo[63359]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T00:29:24.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:23 smithi099 sudo[63359]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:24.004 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:23 smithi099 sudo[63359]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:23 smithi099 sudo[63359]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:23 smithi099 ceph-mon[38079]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-03-30T00:29:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:23 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:24.005 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:23 smithi099 ceph-mon[38079]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T00:29:24.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:23 smithi142 ceph-mon[41779]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-03-30T00:29:24.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:23 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:24.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:23 smithi142 ceph-mon[41779]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T00:29:24.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:23 smithi161 ceph-mon[42221]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-03-30T00:29:24.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:23 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:24.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:23 smithi161 ceph-mon[42221]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T00:29:24.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 sudo[53194]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T00:29:24.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 sudo[53194]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:24.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 sudo[53194]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:24.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 sudo[53194]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:24.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:24 smithi099 ceph-mon[38079]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 97 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:24.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:24 smithi099 ceph-mon[38079]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T00:29:24.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:24 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:29:24.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:24 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:29:24.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:24 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:29:24.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:24 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:29:24.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:24 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:29:24.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:24 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:29:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 sudo[53197]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T00:29:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 sudo[53197]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 sudo[53197]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 sudo[53197]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 ceph-mon[42221]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 97 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 ceph-mon[42221]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T00:29:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:29:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:29:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:29:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:29:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:29:25.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:24 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:29:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 sudo[51548]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T00:29:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 sudo[51548]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 sudo[51548]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 sudo[51548]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:26.119 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:25 smithi099 ceph-mon[38079]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T00:29:26.120 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:25 smithi099 ceph-mon[38079]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T00:29:26.120 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:29:26.120 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:29:26.120 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:29:26.120 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:25 smithi099 ceph-mon[38079]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:26.120 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-30T00:29:26.120 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:25 smithi161 ceph-mon[42221]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T00:29:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:25 smithi161 ceph-mon[42221]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T00:29:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:29:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:29:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:29:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:25 smithi161 ceph-mon[42221]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-30T00:29:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:26.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 sudo[51551]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T00:29:26.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 sudo[51551]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T00:29:26.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 sudo[51551]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T00:29:26.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 sudo[51551]: pam_unix(sudo:session): session closed for user root 2024-03-30T00:29:26.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 97 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:26.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T00:29:26.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:29:26.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:29:26.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:29:26.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:29:26.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:29:26.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:29:26.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T00:29:26.321 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T00:29:26.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T00:29:26.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T00:29:26.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T00:29:26.322 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-03-30T00:29:26.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-30T00:29:26.323 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:27.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:26 smithi161 ceph-mon[42221]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T00:29:27.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:26 smithi161 ceph-mon[42221]: Deploying daemon osd.3 on smithi099 2024-03-30T00:29:27.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:26 smithi161 ceph-mon[42221]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T00:29:27.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:26 smithi099 ceph-mon[38079]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T00:29:27.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:26 smithi099 ceph-mon[38079]: Deploying daemon osd.3 on smithi099 2024-03-30T00:29:27.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:26 smithi099 ceph-mon[38079]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T00:29:27.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:26 smithi142 ceph-mon[41779]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T00:29:27.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:26 smithi142 ceph-mon[41779]: Deploying daemon osd.3 on smithi099 2024-03-30T00:29:27.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:26 smithi142 ceph-mon[41779]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T00:29:27.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:27 smithi099 ceph-mon[38079]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 261 KiB/s wr, 16 op/s 2024-03-30T00:29:28.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:27 smithi161 ceph-mon[42221]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 261 KiB/s wr, 16 op/s 2024-03-30T00:29:28.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:27 smithi142 ceph-mon[41779]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 261 KiB/s wr, 16 op/s 2024-03-30T00:29:30.314 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:30 smithi099 ceph-mon[38079]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 196 KiB/s wr, 12 op/s 2024-03-30T00:29:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:30 smithi142 ceph-mon[41779]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 196 KiB/s wr, 12 op/s 2024-03-30T00:29:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:30 smithi161 ceph-mon[42221]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 196 KiB/s wr, 12 op/s 2024-03-30T00:29:31.479 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:31.479 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:31.479 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:31.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:31.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:31.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:31.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:31.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:31.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:32.691 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:32 smithi099 ceph-mon[38079]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-30T00:29:32.691 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:32.691 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:32 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:32.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:32 smithi161 ceph-mon[42221]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-30T00:29:32.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:32 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:32.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:32 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:32.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:32 smithi142 ceph-mon[41779]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-30T00:29:32.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:32 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:32.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:32 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:33.085 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 3 on host 'smithi099' 2024-03-30T00:29:33.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.075+0000 7fa2f4c00700 1 -- 172.21.15.99:0/2734478671 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa2f802cd20 con 0x7fa2dc0a1110 2024-03-30T00:29:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.078+0000 7fa2fca00700 1 -- 172.21.15.99:0/2734478671 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2dc0a1110 msgr2=0x7fa2dc0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.078+0000 7fa2fca00700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2dc0a1110 0x7fa2dc0a35d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0006010 tx=0x7fa2e000b610 comp rx=0 tx=0).stop 2024-03-30T00:29:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.078+0000 7fa2fca00700 1 -- 172.21.15.99:0/2734478671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8073810 msgr2=0x7fa2f819d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.078+0000 7fa2fca00700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8073810 0x7fa2f819d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800d9b0 tx=0x7fa2e800dcc0 comp rx=0 tx=0).stop 2024-03-30T00:29:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.078+0000 7fa2fca00700 1 -- 172.21.15.99:0/2734478671 shutdown_connections 2024-03-30T00:29:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.078+0000 7fa2fca00700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2dc0a1110 0x7fa2dc0a35d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:33.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.078+0000 7fa2fca00700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2f810d200 0x7fa2f81a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:33.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.078+0000 7fa2fca00700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8073810 0x7fa2f819d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:33.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.078+0000 7fa2fca00700 1 --2- 172.21.15.99:0/2734478671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2f8074de0 0x7fa2f819d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:33.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.078+0000 7fa2fca00700 1 -- 172.21.15.99:0/2734478671 >> 172.21.15.99:0/2734478671 conn(0x7fa2f80fc1c0 msgr2=0x7fa2f81005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:29:33.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.079+0000 7fa2fca00700 1 -- 172.21.15.99:0/2734478671 shutdown_connections 2024-03-30T00:29:33.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:29:33.079+0000 7fa2fca00700 1 -- 172.21.15.99:0/2734478671 wait complete. 2024-03-30T00:29:33.447 DEBUG:teuthology.orchestra.run.smithi099:osd.3> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.3.service 2024-03-30T00:29:33.450 INFO:tasks.cephadm:Deploying osd.4 on smithi142 with /dev/vg_nvme/lv_3... 2024-03-30T00:29:33.451 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-30T00:29:33.519 INFO:journalctl@ceph.osd.3.smithi099.stdout:-- Logs begin at Sat 2024-03-30 00:10:25 UTC. -- 2024-03-30T00:29:33.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:33.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:33.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:33.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:33.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:33.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:33.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:33.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:33.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:33.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:33.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:33.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:33.802 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:29:33.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:33.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:33.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:33.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:33.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:33.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:34.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:34 smithi099 ceph-mon[38079]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-30T00:29:34.489 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 30 00:29:34 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-3[64120]: 2024-03-30T00:29:34.391+0000 7f7eb01e2640 -1 osd.3 0 log_to_monitors true 2024-03-30T00:29:34.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:34 smithi142 ceph-mon[41779]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-30T00:29:34.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:34 smithi161 ceph-mon[42221]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-30T00:29:35.470 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-30T00:29:35.470 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-30T00:29:35.470 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-03-30T00:29:35.470 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-03-30T00:29:35.470 INFO:teuthology.orchestra.run.smithi142.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0293749 s, 357 MB/s 2024-03-30T00:29:35.470 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-03-30T00:29:35.676 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:35 smithi142 ceph-mon[41779]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-30T00:29:35.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:35 smithi161 ceph-mon[42221]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-30T00:29:35.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:35 smithi099 ceph-mon[38079]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-30T00:29:35.855 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-03-30T00:29:36.199 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:29:36.692 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-30T00:29:36.692 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-30T00:29:36.692 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: osdmap e22: 4 total, 3 up, 4 in 2024-03-30T00:29:36.692 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:36.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:29:36.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:36.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:36.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:36.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:36.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:36.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:36.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:36.693 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: osdmap e22: 4 total, 3 up, 4 in 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:36.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:36.738 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-3[64120]: 2024-03-30T00:29:36.418+0000 7f7ea3e00700 -1 osd.3 0 waiting for initial osdmap 2024-03-30T00:29:36.738 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-3[64120]: 2024-03-30T00:29:36.425+0000 7f7e9f600700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:29:36.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-30T00:29:36.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-30T00:29:36.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: osdmap e22: 4 total, 3 up, 4 in 2024-03-30T00:29:36.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:36.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:29:36.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:36.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:36.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:36.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:36.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:36.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:36.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:36.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:37.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.299+0000 7ff15be00700 1 -- 172.21.15.142:0/338329337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff154100e00 msgr2=0x7ff154101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:37.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.299+0000 7ff15be00700 1 --2- 172.21.15.142:0/338329337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff154100e00 0x7ff154101220 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff148009990 tx=0x7ff148009ca0 comp rx=0 tx=0).stop 2024-03-30T00:29:37.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.299+0000 7ff15be00700 1 -- 172.21.15.142:0/338329337 shutdown_connections 2024-03-30T00:29:37.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.299+0000 7ff15be00700 1 --2- 172.21.15.142:0/338329337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff154102920 0x7ff154109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:37.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.299+0000 7ff15be00700 1 --2- 172.21.15.142:0/338329337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff154100e00 0x7ff154101220 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:37.308 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.299+0000 7ff15be00700 1 --2- 172.21.15.142:0/338329337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff154101f60 0x7ff1541023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:37.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.299+0000 7ff15be00700 1 -- 172.21.15.142:0/338329337 >> 172.21.15.142:0/338329337 conn(0x7ff1540fc3c0 msgr2=0x7ff1540fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:29:37.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.299+0000 7ff15be00700 1 -- 172.21.15.142:0/338329337 shutdown_connections 2024-03-30T00:29:37.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.299+0000 7ff15be00700 1 -- 172.21.15.142:0/338329337 wait complete. 2024-03-30T00:29:37.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.300+0000 7ff15be00700 1 Processor -- start 2024-03-30T00:29:37.309 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.300+0000 7ff15be00700 1 -- start start 2024-03-30T00:29:37.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.300+0000 7ff15be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff154100e00 0x7ff154196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:37.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.300+0000 7ff15be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff154101f60 0x7ff1541971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:37.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff15be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff154102920 0x7ff15419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:37.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff15be00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15419e7e0 con 0x7ff154100e00 2024-03-30T00:29:37.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff15be00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15419e920 con 0x7ff154101f60 2024-03-30T00:29:37.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff15be00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15419eae0 con 0x7ff154102920 2024-03-30T00:29:37.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff158a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff154101f60 0x7ff1541971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:37.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff158a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff154101f60 0x7ff1541971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55612/0 (socket says 172.21.15.142:55612) 2024-03-30T00:29:37.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff158a00700 1 -- 172.21.15.142:0/308048668 learned_addr learned my addr 172.21.15.142:0/308048668 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-30T00:29:37.311 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff159400700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff154100e00 0x7ff154196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff159e00700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff154102920 0x7ff15419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff158a00700 1 -- 172.21.15.142:0/308048668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff154102920 msgr2=0x7ff15419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff158a00700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff154102920 0x7ff15419e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff158a00700 1 -- 172.21.15.142:0/308048668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff154100e00 msgr2=0x7ff154196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff158a00700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff154100e00 0x7ff154196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.301+0000 7ff158a00700 1 -- 172.21.15.142:0/308048668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1480095f0 con 0x7ff154101f60 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.302+0000 7ff158a00700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff154101f60 0x7ff1541971f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff14000d950 tx=0x7ff14000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.302+0000 7ff152000700 1 -- 172.21.15.142:0/308048668 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000f890 con 0x7ff154101f60 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.302+0000 7ff152000700 1 -- 172.21.15.142:0/308048668 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff14000fe80 con 0x7ff154101f60 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.302+0000 7ff15be00700 1 -- 172.21.15.142:0/308048668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15419edc0 con 0x7ff154101f60 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.302+0000 7ff152000700 1 -- 172.21.15.142:0/308048668 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000e3b0 con 0x7ff154101f60 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.302+0000 7ff15be00700 1 -- 172.21.15.142:0/308048668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15419f240 con 0x7ff154101f60 2024-03-30T00:29:37.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.303+0000 7ff15be00700 1 -- 172.21.15.142:0/308048668 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff154108370 con 0x7ff154101f60 2024-03-30T00:29:37.313 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.305+0000 7ff152000700 1 -- 172.21.15.142:0/308048668 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ff140010460 con 0x7ff154101f60 2024-03-30T00:29:37.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.307+0000 7ff152000700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff13c0a11a0 0x7ff13c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:37.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.307+0000 7ff152000700 1 -- 172.21.15.142:0/308048668 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3389+0+0 (secure 0 0 0) 0x7ff1400cb690 con 0x7ff154101f60 2024-03-30T00:29:37.315 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.308+0000 7ff159400700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff13c0a11a0 0x7ff13c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:37.316 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.308+0000 7ff159400700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff13c0a11a0 0x7ff13c0a3660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff14800b5c0 tx=0x7ff148003720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:29:37.318 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.311+0000 7ff152000700 1 -- 172.21.15.142:0/308048668 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff1400960c0 con 0x7ff154101f60 2024-03-30T00:29:37.532 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:37.524+0000 7ff15be00700 1 -- 172.21.15.142:0/308048668 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff15402cd20 con 0x7ff13c0a11a0 2024-03-30T00:29:37.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:37 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:29:37.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:37 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:29:37.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:37 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi099 2024-03-30T00:29:37.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:37 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi099 to 8635M 2024-03-30T00:29:37.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:37 smithi161 ceph-mon[42221]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:29:37.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:37 smithi161 ceph-mon[42221]: osdmap e23: 4 total, 3 up, 4 in 2024-03-30T00:29:37.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:37 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:37.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:37 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:37.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:37 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:29:37.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:37 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:29:37.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:37 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi099 2024-03-30T00:29:37.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:37 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi099 to 8635M 2024-03-30T00:29:37.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:37 smithi099 ceph-mon[38079]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:29:37.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:37 smithi099 ceph-mon[38079]: osdmap e23: 4 total, 3 up, 4 in 2024-03-30T00:29:37.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:37 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:37.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:37 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:37.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:37 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:29:37.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:37 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:29:37.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:37 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi099 2024-03-30T00:29:37.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:37 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi099 to 8635M 2024-03-30T00:29:37.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:37 smithi142 ceph-mon[41779]: from='osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:29:37.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:37 smithi142 ceph-mon[41779]: osdmap e23: 4 total, 3 up, 4 in 2024-03-30T00:29:37.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:37 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:37.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:37 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:38.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:38 smithi161 ceph-mon[42221]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-30T00:29:38.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:38.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:38 smithi161 ceph-mon[42221]: osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840] boot 2024-03-30T00:29:38.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:38 smithi161 ceph-mon[42221]: osdmap e24: 4 total, 4 up, 4 in 2024-03-30T00:29:38.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:38.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:29:38.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:29:38.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:38.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:38 smithi161 ceph-mon[42221]: osdmap e25: 4 total, 4 up, 4 in 2024-03-30T00:29:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:38 smithi099 ceph-mon[38079]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-30T00:29:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:38 smithi099 ceph-mon[38079]: osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840] boot 2024-03-30T00:29:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:38 smithi099 ceph-mon[38079]: osdmap e24: 4 total, 4 up, 4 in 2024-03-30T00:29:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:29:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:29:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:38.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:38 smithi099 ceph-mon[38079]: osdmap e25: 4 total, 4 up, 4 in 2024-03-30T00:29:38.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:38 smithi142 ceph-mon[41779]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-30T00:29:38.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:38.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:38 smithi142 ceph-mon[41779]: osd.3 [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840] boot 2024-03-30T00:29:38.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:38 smithi142 ceph-mon[41779]: osdmap e24: 4 total, 4 up, 4 in 2024-03-30T00:29:38.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T00:29:38.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:29:38.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:29:38.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:38.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:38 smithi142 ceph-mon[41779]: osdmap e25: 4 total, 4 up, 4 in 2024-03-30T00:29:39.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:39 smithi161 ceph-mon[42221]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:29:39.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:39 smithi099 ceph-mon[38079]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:29:39.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:39 smithi142 ceph-mon[41779]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:29:40.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:40 smithi161 ceph-mon[42221]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:40.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:40 smithi161 ceph-mon[42221]: osdmap e26: 4 total, 4 up, 4 in 2024-03-30T00:29:40.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:40 smithi161 ceph-mon[42221]: osdmap e27: 4 total, 4 up, 4 in 2024-03-30T00:29:40.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:40 smithi161 ceph-mon[42221]: from='client.? 172.21.15.142:0/3395625946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d47704-9d21-440d-a85a-46cd077d50d4"}]: dispatch 2024-03-30T00:29:40.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:40 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d47704-9d21-440d-a85a-46cd077d50d4"}]: dispatch 2024-03-30T00:29:40.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:40 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68d47704-9d21-440d-a85a-46cd077d50d4"}]': finished 2024-03-30T00:29:40.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:40 smithi161 ceph-mon[42221]: osdmap e28: 5 total, 4 up, 5 in 2024-03-30T00:29:40.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:40 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:40.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:40 smithi099 ceph-mon[38079]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:40.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:40 smithi099 ceph-mon[38079]: osdmap e26: 4 total, 4 up, 4 in 2024-03-30T00:29:40.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:40 smithi099 ceph-mon[38079]: osdmap e27: 4 total, 4 up, 4 in 2024-03-30T00:29:40.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:40 smithi099 ceph-mon[38079]: from='client.? 172.21.15.142:0/3395625946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d47704-9d21-440d-a85a-46cd077d50d4"}]: dispatch 2024-03-30T00:29:40.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:40 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d47704-9d21-440d-a85a-46cd077d50d4"}]: dispatch 2024-03-30T00:29:40.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:40 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68d47704-9d21-440d-a85a-46cd077d50d4"}]': finished 2024-03-30T00:29:40.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:40 smithi099 ceph-mon[38079]: osdmap e28: 5 total, 4 up, 5 in 2024-03-30T00:29:40.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:40 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:40 smithi142 ceph-mon[41779]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:40 smithi142 ceph-mon[41779]: osdmap e26: 4 total, 4 up, 4 in 2024-03-30T00:29:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:40 smithi142 ceph-mon[41779]: osdmap e27: 4 total, 4 up, 4 in 2024-03-30T00:29:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:40 smithi142 ceph-mon[41779]: from='client.? 172.21.15.142:0/3395625946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d47704-9d21-440d-a85a-46cd077d50d4"}]: dispatch 2024-03-30T00:29:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:40 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68d47704-9d21-440d-a85a-46cd077d50d4"}]: dispatch 2024-03-30T00:29:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:40 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68d47704-9d21-440d-a85a-46cd077d50d4"}]': finished 2024-03-30T00:29:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:40 smithi142 ceph-mon[41779]: osdmap e28: 5 total, 4 up, 5 in 2024-03-30T00:29:40.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:40 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:41.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:41 smithi161 ceph-mon[42221]: from='client.? 172.21.15.142:0/4212000603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:29:41.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:41 smithi099 ceph-mon[38079]: from='client.? 172.21.15.142:0/4212000603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:29:41.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:41 smithi142 ceph-mon[41779]: from='client.? 172.21.15.142:0/4212000603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:29:42.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:42 smithi161 ceph-mon[42221]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 162 KiB/s, 0 objects/s recovering 2024-03-30T00:29:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:42 smithi099 ceph-mon[38079]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 162 KiB/s, 0 objects/s recovering 2024-03-30T00:29:42.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:42 smithi142 ceph-mon[41779]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 162 KiB/s, 0 objects/s recovering 2024-03-30T00:29:44.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:44 smithi161 ceph-mon[42221]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-30T00:29:44.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:44 smithi099 ceph-mon[38079]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-30T00:29:44.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:44 smithi142 ceph-mon[41779]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-30T00:29:46.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:45 smithi161 ceph-mon[42221]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T00:29:46.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:45 smithi099 ceph-mon[38079]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T00:29:46.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:45 smithi142 ceph-mon[41779]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T00:29:47.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:46 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-30T00:29:47.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:46 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:47.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:46 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-30T00:29:47.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:46 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:47.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:46 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-30T00:29:47.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:46 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:48.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:47 smithi142 ceph-mon[41779]: Deploying daemon osd.4 on smithi142 2024-03-30T00:29:48.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:47 smithi142 ceph-mon[41779]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-30T00:29:48.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:47 smithi161 ceph-mon[42221]: Deploying daemon osd.4 on smithi142 2024-03-30T00:29:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:47 smithi161 ceph-mon[42221]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-30T00:29:48.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:47 smithi099 ceph-mon[38079]: Deploying daemon osd.4 on smithi142 2024-03-30T00:29:48.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:47 smithi099 ceph-mon[38079]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-30T00:29:50.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:50 smithi142 ceph-mon[41779]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-30T00:29:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:50 smithi161 ceph-mon[42221]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-30T00:29:50.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:50 smithi099 ceph-mon[38079]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-30T00:29:52.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:52 smithi142 ceph-mon[41779]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-30T00:29:52.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:52.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:52.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:52 smithi161 ceph-mon[42221]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-30T00:29:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:52.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:52 smithi099 ceph-mon[38079]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-30T00:29:52.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:52.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:52.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:54.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:54 smithi142 ceph-mon[41779]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:54.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:54.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:54.531 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:54.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:54.532 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:54.598 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.591+0000 7ff152000700 1 -- 172.21.15.142:0/308048668 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff15402cd20 con 0x7ff13c0a11a0 2024-03-30T00:29:54.599 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 4 on host 'smithi142' 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 -- 172.21.15.142:0/308048668 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff13c0a11a0 msgr2=0x7ff13c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff13c0a11a0 0x7ff13c0a3660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff14800b5c0 tx=0x7ff148003720 comp rx=0 tx=0).stop 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 -- 172.21.15.142:0/308048668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff154101f60 msgr2=0x7ff1541971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff154101f60 0x7ff1541971f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff14000d950 tx=0x7ff14000dc60 comp rx=0 tx=0).stop 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 -- 172.21.15.142:0/308048668 shutdown_connections 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff154102920 0x7ff15419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff154100e00 0x7ff154196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff13c0a11a0 0x7ff13c0a3660 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 --2- 172.21.15.142:0/308048668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff154101f60 0x7ff1541971f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 -- 172.21.15.142:0/308048668 >> 172.21.15.142:0/308048668 conn(0x7ff1540fc3c0 msgr2=0x7ff1541054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 -- 172.21.15.142:0/308048668 shutdown_connections 2024-03-30T00:29:54.601 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:29:54.594+0000 7ff15be00700 1 -- 172.21.15.142:0/308048668 wait complete. 2024-03-30T00:29:54.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:54 smithi161 ceph-mon[42221]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:54.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:54.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:54.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:54.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:54.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:54 smithi099 ceph-mon[38079]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:54.999 DEBUG:teuthology.orchestra.run.smithi142:osd.4> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.4.service 2024-03-30T00:29:55.005 INFO:tasks.cephadm:Deploying osd.5 on smithi161 with /dev/vg_nvme/lv_3... 2024-03-30T00:29:55.005 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-30T00:29:55.078 INFO:journalctl@ceph.osd.4.smithi142.stdout:-- Logs begin at Sat 2024-03-30 00:10:40 UTC. -- 2024-03-30T00:29:55.362 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:29:55.796 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:55 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:55.796 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:55 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:55.796 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:55 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:55.796 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:55 smithi142 ceph-mon[41779]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:55.909 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:55 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:55.909 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:55 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:55.909 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:55 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:55.909 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:55 smithi161 ceph-mon[42221]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:55.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:55 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:55.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:55 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:55.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:55 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:29:55.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:55 smithi099 ceph-mon[38079]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:56.067 INFO:journalctl@ceph.osd.4.smithi142.stdout:Mar 30 00:29:55 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-4[54766]: 2024-03-30T00:29:55.787+0000 7fc7d2438640 -1 osd.4 0 log_to_monitors true 2024-03-30T00:29:56.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:56 smithi161 ceph-mon[42221]: from='osd.4 [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T00:29:56.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:56 smithi161 ceph-mon[42221]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T00:29:56.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:56 smithi142 ceph-mon[41779]: from='osd.4 [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T00:29:56.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:56 smithi142 ceph-mon[41779]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T00:29:56.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:56 smithi099 ceph-mon[38079]: from='osd.4 [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T00:29:56.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:56 smithi099 ceph-mon[38079]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T00:29:57.109 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-30T00:29:57.109 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-30T00:29:57.109 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-03-30T00:29:57.109 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-03-30T00:29:57.109 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0296775 s, 353 MB/s 2024-03-30T00:29:57.109 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-03-30T00:29:57.511 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-03-30T00:29:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:57 smithi142 ceph-mon[41779]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-30T00:29:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:57 smithi142 ceph-mon[41779]: osdmap e29: 5 total, 4 up, 5 in 2024-03-30T00:29:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:57 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:57 smithi142 ceph-mon[41779]: from='osd.4 [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:29:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:57 smithi142 ceph-mon[41779]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:29:57.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:57 smithi142 ceph-mon[41779]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:57.862 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:29:57.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:57 smithi161 ceph-mon[42221]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-30T00:29:57.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:57 smithi161 ceph-mon[42221]: osdmap e29: 5 total, 4 up, 5 in 2024-03-30T00:29:57.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:57 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:57.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:57 smithi161 ceph-mon[42221]: from='osd.4 [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:29:57.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:57 smithi161 ceph-mon[42221]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:29:57.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:57 smithi161 ceph-mon[42221]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:57.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:57 smithi099 ceph-mon[38079]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-30T00:29:57.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:57 smithi099 ceph-mon[38079]: osdmap e29: 5 total, 4 up, 5 in 2024-03-30T00:29:57.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:57 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:57.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:57 smithi099 ceph-mon[38079]: from='osd.4 [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:29:57.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:57 smithi099 ceph-mon[38079]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:29:57.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:57 smithi099 ceph-mon[38079]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:58.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:29:58.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: osdmap e30: 5 total, 4 up, 5 in 2024-03-30T00:29:58.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:58.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:58.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi142 2024-03-30T00:29:58.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:58.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:58.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi142 to 8635M 2024-03-30T00:29:58.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:58.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:58.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:58 smithi161 ceph-mon[42221]: from='osd.4 ' entity='osd.4' 2024-03-30T00:29:58.817 INFO:journalctl@ceph.osd.4.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-4[54766]: 2024-03-30T00:29:58.468+0000 7fc7c6000700 -1 osd.4 0 waiting for initial osdmap 2024-03-30T00:29:58.817 INFO:journalctl@ceph.osd.4.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-4[54766]: 2024-03-30T00:29:58.495+0000 7fc7bc800700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: osdmap e30: 5 total, 4 up, 5 in 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi142 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi142 to 8635M 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:58.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:58.819 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.819 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:58 smithi142 ceph-mon[41779]: from='osd.4 ' entity='osd.4' 2024-03-30T00:29:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:29:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: osdmap e30: 5 total, 4 up, 5 in 2024-03-30T00:29:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi142 2024-03-30T00:29:58.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:29:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi142 to 8635M 2024-03-30T00:29:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:29:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:29:58.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:58 smithi099 ceph-mon[38079]: from='osd.4 ' entity='osd.4' 2024-03-30T00:29:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.010+0000 7fea4fe00700 1 -- 172.21.15.161:0/1368284738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea48100e00 msgr2=0x7fea48101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.010+0000 7fea4fe00700 1 --2- 172.21.15.161:0/1368284738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea48100e00 0x7fea48101220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009990 tx=0x7fea3c009ca0 comp rx=0 tx=0).stop 2024-03-30T00:29:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.010+0000 7fea4fe00700 1 -- 172.21.15.161:0/1368284738 shutdown_connections 2024-03-30T00:29:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.010+0000 7fea4fe00700 1 --2- 172.21.15.161:0/1368284738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea48102920 0x7fea48109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.010+0000 7fea4fe00700 1 --2- 172.21.15.161:0/1368284738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea48100e00 0x7fea48101220 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.010+0000 7fea4fe00700 1 --2- 172.21.15.161:0/1368284738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea48101f60 0x7fea481023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.010+0000 7fea4fe00700 1 -- 172.21.15.161:0/1368284738 >> 172.21.15.161:0/1368284738 conn(0x7fea480fc3c0 msgr2=0x7fea480fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:29:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.010+0000 7fea4fe00700 1 -- 172.21.15.161:0/1368284738 shutdown_connections 2024-03-30T00:29:59.018 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.010+0000 7fea4fe00700 1 -- 172.21.15.161:0/1368284738 wait complete. 2024-03-30T00:29:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.011+0000 7fea4fe00700 1 Processor -- start 2024-03-30T00:29:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.011+0000 7fea4fe00700 1 -- start start 2024-03-30T00:29:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.011+0000 7fea4fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea48100e00 0x7fea48196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea48101f60 0x7fea48197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea48102920 0x7fea4819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4fe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4819e760 con 0x7fea48100e00 2024-03-30T00:29:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4fe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4819e8a0 con 0x7fea48101f60 2024-03-30T00:29:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4fe00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4819ea30 con 0x7fea48102920 2024-03-30T00:29:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4de00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea48102920 0x7fea4819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4de00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea48102920 0x7fea4819e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39128/0 (socket says 172.21.15.161:39128) 2024-03-30T00:29:59.021 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4de00700 1 -- 172.21.15.161:0/1096480942 learned_addr learned my addr 172.21.15.161:0/1096480942 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4ca00700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea48101f60 0x7fea48197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4d400700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea48100e00 0x7fea48196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4de00700 1 -- 172.21.15.161:0/1096480942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea48101f60 msgr2=0x7fea48197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4de00700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea48101f60 0x7fea48197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4de00700 1 -- 172.21.15.161:0/1096480942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea48100e00 msgr2=0x7fea48196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4de00700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea48100e00 0x7fea48196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.012+0000 7fea4de00700 1 -- 172.21.15.161:0/1096480942 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c0095f0 con 0x7fea48102920 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.013+0000 7fea4de00700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea48102920 0x7fea4819e190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fea4000ef60 tx=0x7fea4000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.013+0000 7fea46000700 1 -- 172.21.15.161:0/1096480942 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40018070 con 0x7fea48102920 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.013+0000 7fea4fe00700 1 -- 172.21.15.161:0/1096480942 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4819ecb0 con 0x7fea48102920 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.013+0000 7fea46000700 1 -- 172.21.15.161:0/1096480942 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea4000f040 con 0x7fea48102920 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.013+0000 7fea46000700 1 -- 172.21.15.161:0/1096480942 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4000cd10 con 0x7fea48102920 2024-03-30T00:29:59.022 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.013+0000 7fea4fe00700 1 -- 172.21.15.161:0/1096480942 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4819f160 con 0x7fea48102920 2024-03-30T00:29:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.014+0000 7fea4fe00700 1 -- 172.21.15.161:0/1096480942 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea481082c0 con 0x7fea48102920 2024-03-30T00:29:59.023 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.016+0000 7fea46000700 1 -- 172.21.15.161:0/1096480942 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fea40004ad0 con 0x7fea48102920 2024-03-30T00:29:59.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.017+0000 7fea46000700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fea300a1120 0x7fea300a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:29:59.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.017+0000 7fea46000700 1 -- 172.21.15.161:0/1096480942 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3821+0+0 (secure 0 0 0) 0x7fea400cb360 con 0x7fea48102920 2024-03-30T00:29:59.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.018+0000 7fea4d400700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fea300a1120 0x7fea300a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:29:59.025 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.018+0000 7fea4d400700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fea300a1120 0x7fea300a35e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00b670 tx=0x7fea3c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:29:59.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.022+0000 7fea46000700 1 -- 172.21.15.161:0/1096480942 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fea40095c70 con 0x7fea48102920 2024-03-30T00:29:59.238 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:29:59.230+0000 7fea4fe00700 1 -- 172.21.15.161:0/1096480942 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fea4802ce10 con 0x7fea300a1120 2024-03-30T00:29:59.861 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:59 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:29:59.861 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:59 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:29:59.862 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:59.862 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:59 smithi161 ceph-mon[42221]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:59.862 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:59 smithi161 ceph-mon[42221]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:29:59.862 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:29:59.862 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:29:59.862 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:59.862 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:59 smithi161 ceph-mon[42221]: osd.4 [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396] boot 2024-03-30T00:29:59.862 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:59 smithi161 ceph-mon[42221]: osdmap e31: 5 total, 5 up, 5 in 2024-03-30T00:29:59.862 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:29:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:59.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:59 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:29:59.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:59 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:29:59.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:29:59.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:59 smithi099 ceph-mon[38079]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:29:59.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:59 smithi099 ceph-mon[38079]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:29:59.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:29:59.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:29:59.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:29:59.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:59 smithi099 ceph-mon[38079]: osd.4 [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396] boot 2024-03-30T00:29:59.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:59 smithi099 ceph-mon[38079]: osdmap e31: 5 total, 5 up, 5 in 2024-03-30T00:29:59.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:29:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:30:00.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:59 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:30:00.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:59 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:30:00.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:30:00.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:59 smithi142 ceph-mon[41779]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-03-30T00:30:00.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:59 smithi142 ceph-mon[41779]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:30:00.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:30:00.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:30:00.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:00.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:59 smithi142 ceph-mon[41779]: osd.4 [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396] boot 2024-03-30T00:30:00.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:59 smithi142 ceph-mon[41779]: osdmap e31: 5 total, 5 up, 5 in 2024-03-30T00:30:00.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:29:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T00:30:00.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:00 smithi161 ceph-mon[42221]: overall HEALTH_OK 2024-03-30T00:30:00.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:00 smithi161 ceph-mon[42221]: osdmap e32: 5 total, 5 up, 5 in 2024-03-30T00:30:01.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:00 smithi099 ceph-mon[38079]: overall HEALTH_OK 2024-03-30T00:30:01.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:00 smithi099 ceph-mon[38079]: osdmap e32: 5 total, 5 up, 5 in 2024-03-30T00:30:01.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:00 smithi142 ceph-mon[41779]: overall HEALTH_OK 2024-03-30T00:30:01.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:00 smithi142 ceph-mon[41779]: osdmap e32: 5 total, 5 up, 5 in 2024-03-30T00:30:02.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:01 smithi161 ceph-mon[42221]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:02.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:01 smithi161 ceph-mon[42221]: from='client.? 172.21.15.161:0/690647017' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64cdf569-3dfe-4792-bcb0-f0031e0dbf7d"}]: dispatch 2024-03-30T00:30:02.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:01 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64cdf569-3dfe-4792-bcb0-f0031e0dbf7d"}]: dispatch 2024-03-30T00:30:02.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:01 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64cdf569-3dfe-4792-bcb0-f0031e0dbf7d"}]': finished 2024-03-30T00:30:02.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:01 smithi161 ceph-mon[42221]: osdmap e33: 6 total, 5 up, 6 in 2024-03-30T00:30:02.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:01 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:01 smithi099 ceph-mon[38079]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:01 smithi099 ceph-mon[38079]: from='client.? 172.21.15.161:0/690647017' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64cdf569-3dfe-4792-bcb0-f0031e0dbf7d"}]: dispatch 2024-03-30T00:30:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:01 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64cdf569-3dfe-4792-bcb0-f0031e0dbf7d"}]: dispatch 2024-03-30T00:30:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:01 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64cdf569-3dfe-4792-bcb0-f0031e0dbf7d"}]': finished 2024-03-30T00:30:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:01 smithi099 ceph-mon[38079]: osdmap e33: 6 total, 5 up, 6 in 2024-03-30T00:30:02.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:01 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:02.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:01 smithi142 ceph-mon[41779]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:02.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:01 smithi142 ceph-mon[41779]: from='client.? 172.21.15.161:0/690647017' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64cdf569-3dfe-4792-bcb0-f0031e0dbf7d"}]: dispatch 2024-03-30T00:30:02.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:01 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64cdf569-3dfe-4792-bcb0-f0031e0dbf7d"}]: dispatch 2024-03-30T00:30:02.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:01 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64cdf569-3dfe-4792-bcb0-f0031e0dbf7d"}]': finished 2024-03-30T00:30:02.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:01 smithi142 ceph-mon[41779]: osdmap e33: 6 total, 5 up, 6 in 2024-03-30T00:30:02.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:01 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:03.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:02 smithi161 ceph-mon[42221]: from='client.? 172.21.15.161:0/286595622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:30:03.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:02 smithi099 ceph-mon[38079]: from='client.? 172.21.15.161:0/286595622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:30:03.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:02 smithi142 ceph-mon[41779]: from='client.? 172.21.15.161:0/286595622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:30:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:03 smithi161 ceph-mon[42221]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:04.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:03 smithi099 ceph-mon[38079]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:04.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:03 smithi142 ceph-mon[41779]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:06.159 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:05 smithi161 ceph-mon[42221]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:06.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:05 smithi099 ceph-mon[38079]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:06.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:05 smithi142 ceph-mon[41779]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:08.316 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:08 smithi161 ceph-mon[42221]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:08.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:08 smithi142 ceph-mon[41779]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:08.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:08 smithi099 ceph-mon[38079]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:09.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-30T00:30:09.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:09.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-30T00:30:09.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:09.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-30T00:30:09.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:10.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:10 smithi142 ceph-mon[41779]: Deploying daemon osd.5 on smithi161 2024-03-30T00:30:10.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:10 smithi142 ceph-mon[41779]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:10.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:10 smithi161 ceph-mon[42221]: Deploying daemon osd.5 on smithi161 2024-03-30T00:30:10.450 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:10 smithi161 ceph-mon[42221]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:10.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:10 smithi099 ceph-mon[38079]: Deploying daemon osd.5 on smithi161 2024-03-30T00:30:10.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:10 smithi099 ceph-mon[38079]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:12.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:12 smithi161 ceph-mon[42221]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:12.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:12 smithi142 ceph-mon[41779]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:12.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:12 smithi099 ceph-mon[38079]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:14.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:14 smithi142 ceph-mon[41779]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:14.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:14.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:14.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:14.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:14 smithi161 ceph-mon[42221]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:14.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:14.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:14.324 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:14.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:14 smithi099 ceph-mon[38079]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:14.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:14.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:14.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:16.554 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.546+0000 7fea46000700 1 -- 172.21.15.161:0/1096480942 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fea4802ce10 con 0x7fea300a1120 2024-03-30T00:30:16.554 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 5 on host 'smithi161' 2024-03-30T00:30:16.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.549+0000 7fea4fe00700 1 -- 172.21.15.161:0/1096480942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fea300a1120 msgr2=0x7fea300a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:30:16.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.549+0000 7fea4fe00700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fea300a1120 0x7fea300a35e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00b670 tx=0x7fea3c005fb0 comp rx=0 tx=0).stop 2024-03-30T00:30:16.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.549+0000 7fea4fe00700 1 -- 172.21.15.161:0/1096480942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea48102920 msgr2=0x7fea4819e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:30:16.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.549+0000 7fea4fe00700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea48102920 0x7fea4819e190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fea4000ef60 tx=0x7fea4000ef90 comp rx=0 tx=0).stop 2024-03-30T00:30:16.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.549+0000 7fea4fe00700 1 -- 172.21.15.161:0/1096480942 shutdown_connections 2024-03-30T00:30:16.557 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.549+0000 7fea4fe00700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea48102920 0x7fea4819e190 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:16.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.549+0000 7fea4fe00700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea48100e00 0x7fea48196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:16.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.549+0000 7fea4fe00700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fea300a1120 0x7fea300a35e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:16.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.549+0000 7fea4fe00700 1 --2- 172.21.15.161:0/1096480942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea48101f60 0x7fea48197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:16.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.549+0000 7fea4fe00700 1 -- 172.21.15.161:0/1096480942 >> 172.21.15.161:0/1096480942 conn(0x7fea480fc3c0 msgr2=0x7fea48105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:30:16.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.550+0000 7fea4fe00700 1 -- 172.21.15.161:0/1096480942 shutdown_connections 2024-03-30T00:30:16.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:30:16.550+0000 7fea4fe00700 1 -- 172.21.15.161:0/1096480942 wait complete. 2024-03-30T00:30:16.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:16 smithi161 ceph-mon[42221]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:16.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:16.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:16.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:16.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:16.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:16.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:16 smithi099 ceph-mon[38079]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:16.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:16.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:16.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:16.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:16.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:16.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:16 smithi142 ceph-mon[41779]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:16.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:16.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:16.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:16.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:16.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:16.975 DEBUG:teuthology.orchestra.run.smithi161:osd.5> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.5.service 2024-03-30T00:30:16.979 INFO:tasks.cephadm:Deploying osd.6 on smithi099 with /dev/vg_nvme/lv_2... 2024-03-30T00:30:16.979 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-30T00:30:17.056 INFO:journalctl@ceph.osd.5.smithi161.stdout:-- Logs begin at Sat 2024-03-30 00:10:26 UTC. -- 2024-03-30T00:30:17.342 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:30:17.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:17 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:17.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:17 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:17.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:17 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:17.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:17 smithi161 ceph-mon[42221]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:17.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:17 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:17.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:17 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:17.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:17 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:17.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:17 smithi142 ceph-mon[41779]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:17.854 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:17 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:17.854 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:17 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:17.854 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:17 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:17.854 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:17 smithi099 ceph-mon[38079]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:17.969 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:30:17 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:30:17.703+0000 7f8c599e7640 -1 osd.5 0 log_to_monitors true 2024-03-30T00:30:18.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:18 smithi142 ceph-mon[41779]: from='osd.5 [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T00:30:18.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:18 smithi142 ceph-mon[41779]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T00:30:18.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:18 smithi161 ceph-mon[42221]: from='osd.5 [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T00:30:18.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:18 smithi161 ceph-mon[42221]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T00:30:18.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:18 smithi099 ceph-mon[38079]: from='osd.5 [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T00:30:18.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:18 smithi099 ceph-mon[38079]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T00:30:19.074 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-30T00:30:19.075 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-30T00:30:19.075 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-03-30T00:30:19.075 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-03-30T00:30:19.075 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0529992 s, 198 MB/s 2024-03-30T00:30:19.075 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-03-30T00:30:19.465 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2024-03-30T00:30:19.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:19 smithi161 ceph-mon[42221]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-30T00:30:19.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:19 smithi161 ceph-mon[42221]: osdmap e34: 6 total, 5 up, 6 in 2024-03-30T00:30:19.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:19 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:19.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:19 smithi161 ceph-mon[42221]: from='osd.5 [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:30:19.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:19 smithi161 ceph-mon[42221]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:30:19.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:19 smithi161 ceph-mon[42221]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:19.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:19 smithi099 ceph-mon[38079]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-30T00:30:19.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:19 smithi099 ceph-mon[38079]: osdmap e34: 6 total, 5 up, 6 in 2024-03-30T00:30:19.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:19 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:19.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:19 smithi099 ceph-mon[38079]: from='osd.5 [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:30:19.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:19 smithi099 ceph-mon[38079]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:30:19.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:19 smithi099 ceph-mon[38079]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:19.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:19 smithi142 ceph-mon[41779]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-30T00:30:19.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:19 smithi142 ceph-mon[41779]: osdmap e34: 6 total, 5 up, 6 in 2024-03-30T00:30:19.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:19 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:19.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:19 smithi142 ceph-mon[41779]: from='osd.5 [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:30:19.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:19 smithi142 ceph-mon[41779]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:30:19.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:19 smithi142 ceph-mon[41779]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:19.820 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:30:20.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:30:20.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: osdmap e35: 6 total, 5 up, 6 in 2024-03-30T00:30:20.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:20.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:20.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi161 2024-03-30T00:30:20.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: osdmap e36: 6 total, 5 up, 6 in 2024-03-30T00:30:20.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:20.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:20.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:20.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi161 to 8635M 2024-03-30T00:30:20.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:20.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:20.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.656 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:20 smithi099 ceph-mon[38079]: from='osd.5 ' entity='osd.5' 2024-03-30T00:30:20.720 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:30:20.445+0000 7f8c4d600700 -1 osd.5 0 waiting for initial osdmap 2024-03-30T00:30:20.720 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:30:20.473+0000 7f8c43e00700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:30:20.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:30:20.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: osdmap e35: 6 total, 5 up, 6 in 2024-03-30T00:30:20.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:20.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:20.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi161 2024-03-30T00:30:20.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: osdmap e36: 6 total, 5 up, 6 in 2024-03-30T00:30:20.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:20.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:20.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:20.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi161 to 8635M 2024-03-30T00:30:20.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:20.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:20.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:20 smithi161 ceph-mon[42221]: from='osd.5 ' entity='osd.5' 2024-03-30T00:30:20.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:30:20.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: osdmap e35: 6 total, 5 up, 6 in 2024-03-30T00:30:20.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:20.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:20.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi161 2024-03-30T00:30:20.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: osdmap e36: 6 total, 5 up, 6 in 2024-03-30T00:30:20.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:20.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:20.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:20.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi161 to 8635M 2024-03-30T00:30:20.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:20.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:20.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:20.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:20 smithi142 ceph-mon[41779]: from='osd.5 ' entity='osd.5' 2024-03-30T00:30:20.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.898+0000 7fd7e8400700 1 -- 172.21.15.99:0/1240521829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e0101f60 msgr2=0x7fd7e01023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:30:20.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.898+0000 7fd7e8400700 1 --2- 172.21.15.99:0/1240521829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e0101f60 0x7fd7e01023e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009990 tx=0x7fd7cc009ca0 comp rx=0 tx=0).stop 2024-03-30T00:30:20.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.899+0000 7fd7e8400700 1 -- 172.21.15.99:0/1240521829 shutdown_connections 2024-03-30T00:30:20.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.899+0000 7fd7e8400700 1 --2- 172.21.15.99:0/1240521829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7e0102920 0x7fd7e0109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:20.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.899+0000 7fd7e8400700 1 --2- 172.21.15.99:0/1240521829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7e0100e00 0x7fd7e0101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:20.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.899+0000 7fd7e8400700 1 --2- 172.21.15.99:0/1240521829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e0101f60 0x7fd7e01023e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:20.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.899+0000 7fd7e8400700 1 -- 172.21.15.99:0/1240521829 >> 172.21.15.99:0/1240521829 conn(0x7fd7e00fc3c0 msgr2=0x7fd7e00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:30:20.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.899+0000 7fd7e8400700 1 -- 172.21.15.99:0/1240521829 shutdown_connections 2024-03-30T00:30:20.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.899+0000 7fd7e8400700 1 -- 172.21.15.99:0/1240521829 wait complete. 2024-03-30T00:30:20.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.900+0000 7fd7e8400700 1 Processor -- start 2024-03-30T00:30:20.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.900+0000 7fd7e8400700 1 -- start start 2024-03-30T00:30:20.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.900+0000 7fd7e8400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7e0100e00 0x7fd7e0198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.900+0000 7fd7e8400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7e0101f60 0x7fd7e01993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.900+0000 7fd7e8400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e0102920 0x7fd7e01a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.900+0000 7fd7e8400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e01a09c0 con 0x7fd7e0102920 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.900+0000 7fd7e8400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e01a0b00 con 0x7fd7e0101f60 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.900+0000 7fd7e8400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e01a0c90 con 0x7fd7e0100e00 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e5a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7e0100e00 0x7fd7e0198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e5a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7e0100e00 0x7fd7e0198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:53106/0 (socket says 172.21.15.99:53106) 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e5a00700 1 -- 172.21.15.99:0/2152379187 learned_addr learned my addr 172.21.15.99:0/2152379187 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e6400700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e0102920 0x7fd7e01a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e5000700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7e0101f60 0x7fd7e01993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e6400700 1 -- 172.21.15.99:0/2152379187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7e0100e00 msgr2=0x7fd7e0198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e6400700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7e0100e00 0x7fd7e0198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:20.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e6400700 1 -- 172.21.15.99:0/2152379187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7e0101f60 msgr2=0x7fd7e01993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:30:20.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e6400700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7e0101f60 0x7fd7e01993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:20.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e6400700 1 -- 172.21.15.99:0/2152379187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0095f0 con 0x7fd7e0102920 2024-03-30T00:30:20.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e5a00700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7e0100e00 0x7fd7e0198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:30:20.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7e6400700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e0102920 0x7fd7e01a03f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009f60 tx=0x7fd7d8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:30:20.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.901+0000 7fd7de000700 1 -- 172.21.15.99:0/2152379187 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8018070 con 0x7fd7e0102920 2024-03-30T00:30:20.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.902+0000 7fd7e8400700 1 -- 172.21.15.99:0/2152379187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e01a0f10 con 0x7fd7e0102920 2024-03-30T00:30:20.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.902+0000 7fd7de000700 1 -- 172.21.15.99:0/2152379187 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7d800f040 con 0x7fd7e0102920 2024-03-30T00:30:20.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.902+0000 7fd7de000700 1 -- 172.21.15.99:0/2152379187 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d800cbf0 con 0x7fd7e0102920 2024-03-30T00:30:20.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.902+0000 7fd7e8400700 1 -- 172.21.15.99:0/2152379187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e01a1330 con 0x7fd7e0102920 2024-03-30T00:30:20.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.903+0000 7fd7e8400700 1 -- 172.21.15.99:0/2152379187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e004ea80 con 0x7fd7e0102920 2024-03-30T00:30:20.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.905+0000 7fd7de000700 1 -- 172.21.15.99:0/2152379187 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fd7d8007500 con 0x7fd7e0102920 2024-03-30T00:30:20.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.906+0000 7fd7de000700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c80a1120 0x7fd7c80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:30:20.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.906+0000 7fd7de000700 1 -- 172.21.15.99:0/2152379187 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4566+0+0 (secure 0 0 0) 0x7fd7d80cb590 con 0x7fd7e0102920 2024-03-30T00:30:20.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.906+0000 7fd7e5a00700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c80a1120 0x7fd7c80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:30:20.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.907+0000 7fd7e5a00700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c80a1120 0x7fd7c80a35e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4011550 tx=0x7fd7d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:30:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:20.911+0000 7fd7de000700 1 -- 172.21.15.99:0/2152379187 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd7d8095bb0 con 0x7fd7e0102920 2024-03-30T00:30:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:21.125+0000 7fd7e8400700 1 -- 172.21.15.99:0/2152379187 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7e0108f80 con 0x7fd7c80a1120 2024-03-30T00:30:21.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:21 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:30:21.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:21 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:30:21.819 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:21.819 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:21 smithi142 ceph-mon[41779]: osd.5 [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367] boot 2024-03-30T00:30:21.819 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:21 smithi142 ceph-mon[41779]: osdmap e37: 6 total, 6 up, 6 in 2024-03-30T00:30:21.819 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:21.819 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:21 smithi142 ceph-mon[41779]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:21.820 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:21 smithi142 ceph-mon[41779]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:30:21.820 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:30:21.820 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:30:21.820 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:21.830 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:21 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:30:21.830 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:21 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:30:21.830 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:21.830 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:21 smithi099 ceph-mon[38079]: osd.5 [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367] boot 2024-03-30T00:30:21.830 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:21 smithi099 ceph-mon[38079]: osdmap e37: 6 total, 6 up, 6 in 2024-03-30T00:30:21.831 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:21.831 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:21 smithi099 ceph-mon[38079]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:21.831 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:21 smithi099 ceph-mon[38079]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:30:21.831 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:30:21.831 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:30:21.831 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:21.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:21 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:30:21.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:21 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:30:21.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:21.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:21 smithi161 ceph-mon[42221]: osd.5 [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367] boot 2024-03-30T00:30:21.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:21 smithi161 ceph-mon[42221]: osdmap e37: 6 total, 6 up, 6 in 2024-03-30T00:30:21.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T00:30:21.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:21 smithi161 ceph-mon[42221]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-03-30T00:30:21.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:21 smithi161 ceph-mon[42221]: from='client.14382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:30:21.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:30:21.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:30:21.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:23.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:22 smithi161 ceph-mon[42221]: osdmap e38: 6 total, 6 up, 6 in 2024-03-30T00:30:23.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:22 smithi099 ceph-mon[38079]: osdmap e38: 6 total, 6 up, 6 in 2024-03-30T00:30:23.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:22 smithi142 ceph-mon[41779]: osdmap e38: 6 total, 6 up, 6 in 2024-03-30T00:30:24.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:23 smithi161 ceph-mon[42221]: osdmap e39: 6 total, 6 up, 6 in 2024-03-30T00:30:24.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:23 smithi161 ceph-mon[42221]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:24.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:23 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1281270034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce11a006-0d8d-45d4-a9a8-e465307c6025"}]: dispatch 2024-03-30T00:30:24.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:23 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1281270034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce11a006-0d8d-45d4-a9a8-e465307c6025"}]': finished 2024-03-30T00:30:24.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:23 smithi161 ceph-mon[42221]: osdmap e40: 7 total, 6 up, 7 in 2024-03-30T00:30:24.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:23 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:24.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:23 smithi099 ceph-mon[38079]: osdmap e39: 6 total, 6 up, 6 in 2024-03-30T00:30:24.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:23 smithi099 ceph-mon[38079]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:24.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:23 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1281270034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce11a006-0d8d-45d4-a9a8-e465307c6025"}]: dispatch 2024-03-30T00:30:24.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:23 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1281270034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce11a006-0d8d-45d4-a9a8-e465307c6025"}]': finished 2024-03-30T00:30:24.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:23 smithi099 ceph-mon[38079]: osdmap e40: 7 total, 6 up, 7 in 2024-03-30T00:30:24.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:23 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:24.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:23 smithi142 ceph-mon[41779]: osdmap e39: 6 total, 6 up, 6 in 2024-03-30T00:30:24.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:23 smithi142 ceph-mon[41779]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:24.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:23 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1281270034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ce11a006-0d8d-45d4-a9a8-e465307c6025"}]: dispatch 2024-03-30T00:30:24.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:23 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1281270034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ce11a006-0d8d-45d4-a9a8-e465307c6025"}]': finished 2024-03-30T00:30:24.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:23 smithi142 ceph-mon[41779]: osdmap e40: 7 total, 6 up, 7 in 2024-03-30T00:30:24.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:23 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:24 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2577255430' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:30:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:24 smithi161 ceph-mon[42221]: osdmap e41: 7 total, 6 up, 7 in 2024-03-30T00:30:25.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:24 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:25.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:24 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2577255430' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:30:25.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:24 smithi099 ceph-mon[38079]: osdmap e41: 7 total, 6 up, 7 in 2024-03-30T00:30:25.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:24 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:25.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:24 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2577255430' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:30:25.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:24 smithi142 ceph-mon[41779]: osdmap e41: 7 total, 6 up, 7 in 2024-03-30T00:30:25.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:24 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:26.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:25 smithi161 ceph-mon[42221]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:25 smithi161 ceph-mon[42221]: osdmap e42: 7 total, 6 up, 7 in 2024-03-30T00:30:26.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:26.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:25 smithi099 ceph-mon[38079]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:26.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:25 smithi099 ceph-mon[38079]: osdmap e42: 7 total, 6 up, 7 in 2024-03-30T00:30:26.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:26.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:25 smithi142 ceph-mon[41779]: pgmap v129: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:26.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:25 smithi142 ceph-mon[41779]: osdmap e42: 7 total, 6 up, 7 in 2024-03-30T00:30:26.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:28.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:28 smithi142 ceph-mon[41779]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-03-30T00:30:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:28 smithi161 ceph-mon[42221]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-03-30T00:30:28.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:28 smithi099 ceph-mon[38079]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-03-30T00:30:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:30 smithi099 ceph-mon[38079]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T00:30:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:30 smithi142 ceph-mon[41779]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T00:30:30.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:30 smithi161 ceph-mon[42221]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T00:30:31.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-30T00:30:31.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:31.226 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:31 smithi099 ceph-mon[38079]: Deploying daemon osd.6 on smithi099 2024-03-30T00:30:31.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-30T00:30:31.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:31.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:31 smithi142 ceph-mon[41779]: Deploying daemon osd.6 on smithi099 2024-03-30T00:30:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-30T00:30:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:31.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:31 smithi161 ceph-mon[42221]: Deploying daemon osd.6 on smithi099 2024-03-30T00:30:32.310 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:32 smithi099 ceph-mon[38079]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-30T00:30:32.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:32 smithi142 ceph-mon[41779]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-30T00:30:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:32 smithi161 ceph-mon[42221]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-30T00:30:34.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:34 smithi142 ceph-mon[41779]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-30T00:30:34.390 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:34 smithi099 ceph-mon[38079]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-30T00:30:34.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:34 smithi161 ceph-mon[42221]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-30T00:30:36.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:36 smithi142 ceph-mon[41779]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-30T00:30:36.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:36.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:36.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:36.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:36 smithi161 ceph-mon[42221]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-30T00:30:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:36.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:36.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:36 smithi099 ceph-mon[38079]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-30T00:30:36.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:36.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:36.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:38.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.137+0000 7fd7de000700 1 -- 172.21.15.99:0/2152379187 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd7e0108f80 con 0x7fd7c80a1120 2024-03-30T00:30:38.144 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 6 on host 'smithi099' 2024-03-30T00:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.139+0000 7fd7e8400700 1 -- 172.21.15.99:0/2152379187 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c80a1120 msgr2=0x7fd7c80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.139+0000 7fd7e8400700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c80a1120 0x7fd7c80a35e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4011550 tx=0x7fd7d4009260 comp rx=0 tx=0).stop 2024-03-30T00:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.139+0000 7fd7e8400700 1 -- 172.21.15.99:0/2152379187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e0102920 msgr2=0x7fd7e01a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.139+0000 7fd7e8400700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e0102920 0x7fd7e01a03f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009f60 tx=0x7fd7d8009f90 comp rx=0 tx=0).stop 2024-03-30T00:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.139+0000 7fd7e8400700 1 -- 172.21.15.99:0/2152379187 shutdown_connections 2024-03-30T00:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.139+0000 7fd7e8400700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e0102920 0x7fd7e01a03f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:38.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.139+0000 7fd7e8400700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7e0100e00 0x7fd7e0198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:38.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.139+0000 7fd7e8400700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c80a1120 0x7fd7c80a35e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:38.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.139+0000 7fd7e8400700 1 --2- 172.21.15.99:0/2152379187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7e0101f60 0x7fd7e01993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:38.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.139+0000 7fd7e8400700 1 -- 172.21.15.99:0/2152379187 >> 172.21.15.99:0/2152379187 conn(0x7fd7e00fc3c0 msgr2=0x7fd7e0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:30:38.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.140+0000 7fd7e8400700 1 -- 172.21.15.99:0/2152379187 shutdown_connections 2024-03-30T00:30:38.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:30:38.140+0000 7fd7e8400700 1 -- 172.21.15.99:0/2152379187 wait complete. 2024-03-30T00:30:38.358 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:38 smithi099 ceph-mon[38079]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:38.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:38.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:38.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.359 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:38 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:38 smithi161 ceph-mon[42221]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:38.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:38.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:38.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:38 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.537 DEBUG:teuthology.orchestra.run.smithi099:osd.6> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.6.service 2024-03-30T00:30:38.541 INFO:tasks.cephadm:Deploying osd.7 on smithi142 with /dev/vg_nvme/lv_2... 2024-03-30T00:30:38.541 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-30T00:30:38.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:38 smithi142 ceph-mon[41779]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:38.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:38.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:38.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:38 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:38.646 INFO:journalctl@ceph.osd.6.smithi099.stdout:-- Logs begin at Sat 2024-03-30 00:10:25 UTC. -- 2024-03-30T00:30:38.916 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:30:39.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:39 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:39.382 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:39 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:39.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:39 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:39.558 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:30:39 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:30:39.297+0000 7f9bb26c8640 -1 osd.6 0 log_to_monitors true 2024-03-30T00:30:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:40 smithi161 ceph-mon[42221]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:40 smithi161 ceph-mon[42221]: from='osd.6 [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-30T00:30:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:40 smithi161 ceph-mon[42221]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-30T00:30:40.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:40 smithi099 ceph-mon[38079]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:40.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:40 smithi099 ceph-mon[38079]: from='osd.6 [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-30T00:30:40.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:40 smithi099 ceph-mon[38079]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-30T00:30:40.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:40 smithi142 ceph-mon[41779]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:40.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:40 smithi142 ceph-mon[41779]: from='osd.6 [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-30T00:30:40.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:40 smithi142 ceph-mon[41779]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-30T00:30:40.598 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-30T00:30:40.598 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-30T00:30:40.598 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-03-30T00:30:40.598 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-03-30T00:30:40.598 INFO:teuthology.orchestra.run.smithi142.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0296127 s, 354 MB/s 2024-03-30T00:30:40.598 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-03-30T00:30:40.976 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-03-30T00:30:41.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:41 smithi142 ceph-mon[41779]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-30T00:30:41.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:41 smithi142 ceph-mon[41779]: osdmap e43: 7 total, 6 up, 7 in 2024-03-30T00:30:41.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:41 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:41.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:41 smithi142 ceph-mon[41779]: from='osd.6 [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:30:41.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:41 smithi142 ceph-mon[41779]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:30:41.369 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:30:41.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:41 smithi161 ceph-mon[42221]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-30T00:30:41.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:41 smithi161 ceph-mon[42221]: osdmap e43: 7 total, 6 up, 7 in 2024-03-30T00:30:41.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:41 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:41.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:41 smithi161 ceph-mon[42221]: from='osd.6 [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:30:41.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:41 smithi161 ceph-mon[42221]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:30:41.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:41 smithi099 ceph-mon[38079]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-30T00:30:41.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:41 smithi099 ceph-mon[38079]: osdmap e43: 7 total, 6 up, 7 in 2024-03-30T00:30:41.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:41 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:41.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:41 smithi099 ceph-mon[38079]: from='osd.6 [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:30:41.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:41 smithi099 ceph-mon[38079]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:30:42.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: osdmap e44: 7 total, 6 up, 7 in 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi099 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi099 to 5756M 2024-03-30T00:30:42.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:42.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:42.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:42 smithi161 ceph-mon[42221]: from='osd.6 ' entity='osd.6' 2024-03-30T00:30:42.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:30:42.084+0000 7f9ba8000700 -1 osd.6 0 waiting for initial osdmap 2024-03-30T00:30:42.488 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:30:42.119+0000 7f9b9ca00700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:30:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:30:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: osdmap e44: 7 total, 6 up, 7 in 2024-03-30T00:30:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi099 2024-03-30T00:30:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:42.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:42.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi099 to 5756M 2024-03-30T00:30:42.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:42.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:42.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:42 smithi099 ceph-mon[38079]: from='osd.6 ' entity='osd.6' 2024-03-30T00:30:42.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.483+0000 7f4a13e00700 1 -- 172.21.15.142:0/2178241802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c102920 msgr2=0x7f4a0c109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:30:42.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.483+0000 7f4a13e00700 1 --2- 172.21.15.142:0/2178241802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c102920 0x7f4a0c109240 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400b0a0 tx=0x7f4a0400b3b0 comp rx=0 tx=0).stop 2024-03-30T00:30:42.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.483+0000 7f4a13e00700 1 -- 172.21.15.142:0/2178241802 shutdown_connections 2024-03-30T00:30:42.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.483+0000 7f4a13e00700 1 --2- 172.21.15.142:0/2178241802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c102920 0x7f4a0c109240 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:42.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.483+0000 7f4a13e00700 1 --2- 172.21.15.142:0/2178241802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a0c100e00 0x7f4a0c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:42.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.483+0000 7f4a13e00700 1 --2- 172.21.15.142:0/2178241802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c101f60 0x7f4a0c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:42.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.483+0000 7f4a13e00700 1 -- 172.21.15.142:0/2178241802 >> 172.21.15.142:0/2178241802 conn(0x7f4a0c0fc3c0 msgr2=0x7f4a0c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:30:42.490 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.483+0000 7f4a13e00700 1 -- 172.21.15.142:0/2178241802 shutdown_connections 2024-03-30T00:30:42.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.483+0000 7f4a13e00700 1 -- 172.21.15.142:0/2178241802 wait complete. 2024-03-30T00:30:42.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.484+0000 7f4a13e00700 1 Processor -- start 2024-03-30T00:30:42.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a13e00700 1 -- start start 2024-03-30T00:30:42.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a13e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c100e00 0x7f4a0c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:30:42.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a13e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c101f60 0x7f4a0c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:30:42.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a13e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a0c102920 0x7f4a0c1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:30:42.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a13e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c1a0a30 con 0x7f4a0c101f60 2024-03-30T00:30:42.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a13e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c1a0b70 con 0x7f4a0c100e00 2024-03-30T00:30:42.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a13e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c1a0d00 con 0x7f4a0c102920 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a11400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c100e00 0x7f4a0c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a11400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c100e00 0x7f4a0c198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44152/0 (socket says 172.21.15.142:44152) 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a11400700 1 -- 172.21.15.142:0/2330594952 learned_addr learned my addr 172.21.15.142:0/2330594952 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a10a00700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c101f60 0x7f4a0c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.485+0000 7f4a11e00700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a0c102920 0x7f4a0c1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.486+0000 7f4a11400700 1 -- 172.21.15.142:0/2330594952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a0c102920 msgr2=0x7f4a0c1a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.486+0000 7f4a11400700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a0c102920 0x7f4a0c1a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.486+0000 7f4a11400700 1 -- 172.21.15.142:0/2330594952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c101f60 msgr2=0x7f4a0c199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.486+0000 7f4a11400700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c101f60 0x7f4a0c199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.486+0000 7f4a11400700 1 -- 172.21.15.142:0/2330594952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f8009520 con 0x7f4a0c100e00 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.486+0000 7f4a11400700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c100e00 0x7f4a0c198ef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c9b0 tx=0x7f4a0000cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:30:42.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.486+0000 7f4a0a000700 1 -- 172.21.15.142:0/2330594952 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00007b00 con 0x7f4a0c100e00 2024-03-30T00:30:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.486+0000 7f4a0a000700 1 -- 172.21.15.142:0/2330594952 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a00007c60 con 0x7f4a0c100e00 2024-03-30T00:30:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.486+0000 7f4a13e00700 1 -- 172.21.15.142:0/2330594952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a04009d00 con 0x7f4a0c100e00 2024-03-30T00:30:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.486+0000 7f4a0a000700 1 -- 172.21.15.142:0/2330594952 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00018660 con 0x7f4a0c100e00 2024-03-30T00:30:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.486+0000 7f4a13e00700 1 -- 172.21.15.142:0/2330594952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0c1a1270 con 0x7f4a0c100e00 2024-03-30T00:30:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.487+0000 7f4a13e00700 1 -- 172.21.15.142:0/2330594952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0c04ea80 con 0x7f4a0c100e00 2024-03-30T00:30:42.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.489+0000 7f4a0a000700 1 -- 172.21.15.142:0/2330594952 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f4a0000f450 con 0x7f4a0c100e00 2024-03-30T00:30:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.490+0000 7f4a0a000700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49f40a11a0 0x7f49f40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:30:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.490+0000 7f4a0a000700 1 -- 172.21.15.142:0/2330594952 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4685+0+0 (secure 0 0 0) 0x7f4a000cbad0 con 0x7f4a0c100e00 2024-03-30T00:30:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.491+0000 7f4a10a00700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49f40a11a0 0x7f49f40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:30:42.498 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.491+0000 7f4a10a00700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49f40a11a0 0x7f49f40a3660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009d80 tx=0x7f49f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:30:42.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.495+0000 7f4a0a000700 1 -- 172.21.15.142:0/2330594952 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4a00095ff0 con 0x7f4a0c100e00 2024-03-30T00:30:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:30:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: osdmap e44: 7 total, 6 up, 7 in 2024-03-30T00:30:42.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi099 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi099 to 5756M 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:42.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:42 smithi142 ceph-mon[41779]: from='osd.6 ' entity='osd.6' 2024-03-30T00:30:42.709 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:30:42.702+0000 7f4a13e00700 1 -- 172.21.15.142:0/2330594952 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a0c074700 con 0x7f49f40a11a0 2024-03-30T00:30:43.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:43 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:30:43.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:43 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:30:43.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:43.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:30:43.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:30:43.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:43.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:43 smithi142 ceph-mon[41779]: osd.6 [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080] boot 2024-03-30T00:30:43.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:43 smithi142 ceph-mon[41779]: osdmap e45: 7 total, 7 up, 7 in 2024-03-30T00:30:43.443 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:43.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:43 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:30:43.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:43 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:30:43.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:43.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:30:43.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:30:43.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:43.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:43 smithi161 ceph-mon[42221]: osd.6 [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080] boot 2024-03-30T00:30:43.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:43 smithi161 ceph-mon[42221]: osdmap e45: 7 total, 7 up, 7 in 2024-03-30T00:30:43.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:43 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:30:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:43 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:30:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:30:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:30:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:43 smithi099 ceph-mon[38079]: osd.6 [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080] boot 2024-03-30T00:30:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:43 smithi099 ceph-mon[38079]: osdmap e45: 7 total, 7 up, 7 in 2024-03-30T00:30:43.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T00:30:44.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:44 smithi161 ceph-mon[42221]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:30:44.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:44 smithi161 ceph-mon[42221]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:44.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:44 smithi099 ceph-mon[38079]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:30:44.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:44 smithi099 ceph-mon[38079]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:44.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:44 smithi142 ceph-mon[41779]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:30:44.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:44 smithi142 ceph-mon[41779]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-03-30T00:30:45.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:45 smithi161 ceph-mon[42221]: osdmap e46: 7 total, 7 up, 7 in 2024-03-30T00:30:45.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:45 smithi099 ceph-mon[38079]: osdmap e46: 7 total, 7 up, 7 in 2024-03-30T00:30:45.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:45 smithi142 ceph-mon[41779]: osdmap e46: 7 total, 7 up, 7 in 2024-03-30T00:30:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:46 smithi161 ceph-mon[42221]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:46 smithi161 ceph-mon[42221]: from='client.? 172.21.15.142:0/662263309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f762698a-4112-4f4e-9658-3f91393ef1a1"}]: dispatch 2024-03-30T00:30:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:46 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f762698a-4112-4f4e-9658-3f91393ef1a1"}]: dispatch 2024-03-30T00:30:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:46 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f762698a-4112-4f4e-9658-3f91393ef1a1"}]': finished 2024-03-30T00:30:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:46 smithi161 ceph-mon[42221]: osdmap e47: 8 total, 7 up, 8 in 2024-03-30T00:30:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:46 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:30:46.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:46 smithi161 ceph-mon[42221]: from='client.? 172.21.15.142:0/3350088325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:30:46.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:46 smithi099 ceph-mon[38079]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:46.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:46 smithi099 ceph-mon[38079]: from='client.? 172.21.15.142:0/662263309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f762698a-4112-4f4e-9658-3f91393ef1a1"}]: dispatch 2024-03-30T00:30:46.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:46 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f762698a-4112-4f4e-9658-3f91393ef1a1"}]: dispatch 2024-03-30T00:30:46.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:46 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f762698a-4112-4f4e-9658-3f91393ef1a1"}]': finished 2024-03-30T00:30:46.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:46 smithi099 ceph-mon[38079]: osdmap e47: 8 total, 7 up, 8 in 2024-03-30T00:30:46.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:46 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:30:46.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:46 smithi099 ceph-mon[38079]: from='client.? 172.21.15.142:0/3350088325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:30:46.570 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:46 smithi142 ceph-mon[41779]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:46.570 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:46 smithi142 ceph-mon[41779]: from='client.? 172.21.15.142:0/662263309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f762698a-4112-4f4e-9658-3f91393ef1a1"}]: dispatch 2024-03-30T00:30:46.570 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:46 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f762698a-4112-4f4e-9658-3f91393ef1a1"}]: dispatch 2024-03-30T00:30:46.570 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:46 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f762698a-4112-4f4e-9658-3f91393ef1a1"}]': finished 2024-03-30T00:30:46.570 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:46 smithi142 ceph-mon[41779]: osdmap e47: 8 total, 7 up, 8 in 2024-03-30T00:30:46.570 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:46 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:30:46.570 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:46 smithi142 ceph-mon[41779]: from='client.? 172.21.15.142:0/3350088325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:30:48.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:48 smithi161 ceph-mon[42221]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:48.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:48 smithi099 ceph-mon[38079]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:48.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:48 smithi142 ceph-mon[41779]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:50.467 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:50 smithi142 ceph-mon[41779]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:50 smithi161 ceph-mon[42221]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:50.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:50 smithi099 ceph-mon[38079]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:52 smithi161 ceph-mon[42221]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-30T00:30:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:52.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:52 smithi099 ceph-mon[38079]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:52.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-30T00:30:52.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:52.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:52 smithi142 ceph-mon[41779]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:52.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-30T00:30:52.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:30:53.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:53 smithi161 ceph-mon[42221]: Deploying daemon osd.7 on smithi142 2024-03-30T00:30:53.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:53 smithi099 ceph-mon[38079]: Deploying daemon osd.7 on smithi142 2024-03-30T00:30:53.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:53 smithi142 ceph-mon[41779]: Deploying daemon osd.7 on smithi142 2024-03-30T00:30:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:54 smithi161 ceph-mon[42221]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:54.487 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:54 smithi142 ceph-mon[41779]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:54.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:54 smithi099 ceph-mon[38079]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:56.116 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:55 smithi142 ceph-mon[41779]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:56.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:55 smithi161 ceph-mon[42221]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:56.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:55 smithi099 ceph-mon[38079]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:58.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:57 smithi142 ceph-mon[41779]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:58.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:57 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:58.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:57 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:58.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:30:57 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:57 smithi161 ceph-mon[42221]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:57 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:58.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:57 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:58.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:30:57 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:30:58.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:57 smithi099 ceph-mon[38079]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:30:58.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:57 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:58.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:57 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:30:58.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:30:57 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:00.024 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.015+0000 7f4a0a000700 1 -- 172.21.15.142:0/2330594952 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4a0c074700 con 0x7f49f40a11a0 2024-03-30T00:31:00.024 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 7 on host 'smithi142' 2024-03-30T00:31:00.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.017+0000 7f4a13e00700 1 -- 172.21.15.142:0/2330594952 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49f40a11a0 msgr2=0x7f49f40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:00.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.017+0000 7f4a13e00700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49f40a11a0 0x7f49f40a3660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009d80 tx=0x7f49f8009260 comp rx=0 tx=0).stop 2024-03-30T00:31:00.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.018+0000 7f4a13e00700 1 -- 172.21.15.142:0/2330594952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c100e00 msgr2=0x7f4a0c198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:00.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.018+0000 7f4a13e00700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c100e00 0x7f4a0c198ef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c9b0 tx=0x7f4a0000cd70 comp rx=0 tx=0).stop 2024-03-30T00:31:00.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.018+0000 7f4a13e00700 1 -- 172.21.15.142:0/2330594952 shutdown_connections 2024-03-30T00:31:00.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.018+0000 7f4a13e00700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a0c102920 0x7f4a0c1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:00.027 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.018+0000 7f4a13e00700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49f40a11a0 0x7f49f40a3660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:00.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.018+0000 7f4a13e00700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a0c100e00 0x7f4a0c198ef0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:00.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.018+0000 7f4a13e00700 1 --2- 172.21.15.142:0/2330594952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c101f60 0x7f4a0c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:00.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.018+0000 7f4a13e00700 1 -- 172.21.15.142:0/2330594952 >> 172.21.15.142:0/2330594952 conn(0x7f4a0c0fc3c0 msgr2=0x7f4a0c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:31:00.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.018+0000 7f4a13e00700 1 -- 172.21.15.142:0/2330594952 shutdown_connections 2024-03-30T00:31:00.028 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:00.018+0000 7f4a13e00700 1 -- 172.21.15.142:0/2330594952 wait complete. 2024-03-30T00:31:00.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:00 smithi142 ceph-mon[41779]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:31:00.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:00.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:00.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.050 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:00 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:00.424 DEBUG:teuthology.orchestra.run.smithi142:osd.7> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.7.service 2024-03-30T00:31:00.428 INFO:tasks.cephadm:Deploying osd.8 on smithi161 with /dev/vg_nvme/lv_2... 2024-03-30T00:31:00.429 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-30T00:31:00.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:00 smithi161 ceph-mon[42221]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:31:00.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:00 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:00 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:00 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:00 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:00 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:00 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:00 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:00 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:00.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:00 smithi099 ceph-mon[38079]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:31:00.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:00 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:00 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:00 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:00.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:00 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:00.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:00 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:00 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:00 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:00.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:00 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:00.501 INFO:journalctl@ceph.osd.7.smithi142.stdout:-- Logs begin at Sat 2024-03-30 00:10:40 UTC. -- 2024-03-30T00:31:00.811 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:31:01.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:01 smithi161 ceph-mon[42221]: from='osd.7 [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T00:31:01.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:01 smithi161 ceph-mon[42221]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T00:31:01.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:01 smithi142 ceph-mon[41779]: from='osd.7 [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T00:31:01.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:01 smithi142 ceph-mon[41779]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T00:31:01.317 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:31:01 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:31:01.006+0000 7f08cb79d640 -1 osd.7 0 log_to_monitors true 2024-03-30T00:31:01.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:01 smithi099 ceph-mon[38079]: from='osd.7 [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T00:31:01.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:01 smithi099 ceph-mon[38079]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T00:31:02.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:02 smithi142 ceph-mon[41779]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:31:02.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:02 smithi142 ceph-mon[41779]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-30T00:31:02.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:02 smithi142 ceph-mon[41779]: osdmap e48: 8 total, 7 up, 8 in 2024-03-30T00:31:02.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:02.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:02 smithi142 ceph-mon[41779]: from='osd.7 [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:31:02.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:02 smithi142 ceph-mon[41779]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:31:02.318 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:31:02 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:31:02.040+0000 7f08bf400700 -1 osd.7 0 waiting for initial osdmap 2024-03-30T00:31:02.318 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:31:02 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:31:02.060+0000 7f08bac00700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:31:02.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:02 smithi161 ceph-mon[42221]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:31:02.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:02 smithi161 ceph-mon[42221]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-30T00:31:02.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:02 smithi161 ceph-mon[42221]: osdmap e48: 8 total, 7 up, 8 in 2024-03-30T00:31:02.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:02.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:02 smithi161 ceph-mon[42221]: from='osd.7 [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:31:02.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:02 smithi161 ceph-mon[42221]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:31:02.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:02 smithi099 ceph-mon[38079]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:31:02.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:02 smithi099 ceph-mon[38079]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-30T00:31:02.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:02 smithi099 ceph-mon[38079]: osdmap e48: 8 total, 7 up, 8 in 2024-03-30T00:31:02.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:02.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:02 smithi099 ceph-mon[38079]: from='osd.7 [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:31:02.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:02 smithi099 ceph-mon[38079]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:31:02.524 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-30T00:31:02.524 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-30T00:31:02.524 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-03-30T00:31:02.524 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-03-30T00:31:02.524 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0307347 s, 341 MB/s 2024-03-30T00:31:02.524 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-03-30T00:31:02.961 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-03-30T00:31:03.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:03 smithi161 ceph-mon[42221]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:31:03.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:03 smithi161 ceph-mon[42221]: osdmap e49: 8 total, 7 up, 8 in 2024-03-30T00:31:03.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:03 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:03.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:03 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:03.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:03 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:03.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:03 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:03.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:03 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:03.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:03 smithi142 ceph-mon[41779]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:31:03.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:03 smithi142 ceph-mon[41779]: osdmap e49: 8 total, 7 up, 8 in 2024-03-30T00:31:03.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:03 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:03.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:03 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:03.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:03 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:03.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:03 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:03.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:03 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:03.360 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:31:03.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:03 smithi099 ceph-mon[38079]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:31:03.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:03 smithi099 ceph-mon[38079]: osdmap e49: 8 total, 7 up, 8 in 2024-03-30T00:31:03.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:03 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:03.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:03 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:03.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:03 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:03.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:03 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:03.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:03 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:04.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi142 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi142 to 5756M 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: osd.7 [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815] boot 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: osdmap e50: 8 total, 8 up, 8 in 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:04.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:04 smithi161 ceph-mon[42221]: osdmap e51: 8 total, 8 up, 8 in 2024-03-30T00:31:04.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:31:04.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:31:04.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:31:04.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi142 2024-03-30T00:31:04.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:04.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:04.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi142 to 5756M 2024-03-30T00:31:04.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: osd.7 [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815] boot 2024-03-30T00:31:04.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: osdmap e50: 8 total, 8 up, 8 in 2024-03-30T00:31:04.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:04.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:04.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:04.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:04.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:04.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:04 smithi142 ceph-mon[41779]: osdmap e51: 8 total, 8 up, 8 in 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 626 GiB / 626 GiB avail 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi142 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi142 to 5756M 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: osd.7 [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815] boot 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: osdmap e50: 8 total, 8 up, 8 in 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:04.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:04.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:04.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:04 smithi099 ceph-mon[38079]: osdmap e51: 8 total, 8 up, 8 in 2024-03-30T00:31:04.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.549+0000 7fa40dc00700 1 -- 172.21.15.161:0/910751373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa408100e00 msgr2=0x7fa408101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:04.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.549+0000 7fa40dc00700 1 --2- 172.21.15.161:0/910751373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa408100e00 0x7fa408101220 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009990 tx=0x7fa3f8009ca0 comp rx=0 tx=0).stop 2024-03-30T00:31:04.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.549+0000 7fa40dc00700 1 -- 172.21.15.161:0/910751373 shutdown_connections 2024-03-30T00:31:04.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.549+0000 7fa40dc00700 1 --2- 172.21.15.161:0/910751373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa408102920 0x7fa408109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:04.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.549+0000 7fa40dc00700 1 --2- 172.21.15.161:0/910751373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa408100e00 0x7fa408101220 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:04.558 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.549+0000 7fa40dc00700 1 --2- 172.21.15.161:0/910751373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa408101f60 0x7fa4081023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:04.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.549+0000 7fa40dc00700 1 -- 172.21.15.161:0/910751373 >> 172.21.15.161:0/910751373 conn(0x7fa4080fc3c0 msgr2=0x7fa4080fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:31:04.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.549+0000 7fa40dc00700 1 -- 172.21.15.161:0/910751373 shutdown_connections 2024-03-30T00:31:04.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.550+0000 7fa40dc00700 1 -- 172.21.15.161:0/910751373 wait complete. 2024-03-30T00:31:04.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.550+0000 7fa40dc00700 1 Processor -- start 2024-03-30T00:31:04.559 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.551+0000 7fa40dc00700 1 -- start start 2024-03-30T00:31:04.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.551+0000 7fa40dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa408100e00 0x7fa408074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:04.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.551+0000 7fa40dc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa408101f60 0x7fa4080731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:04.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.551+0000 7fa40dc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa408102920 0x7fa4080736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:04.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.551+0000 7fa40dc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa408073e80 con 0x7fa408102920 2024-03-30T00:31:04.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.551+0000 7fa40dc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa408073ff0 con 0x7fa408100e00 2024-03-30T00:31:04.562 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.551+0000 7fa40dc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4081a0c90 con 0x7fa408101f60 2024-03-30T00:31:04.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.551+0000 7fa406000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa408101f60 0x7fa4080731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:04.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.551+0000 7fa406000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa408101f60 0x7fa4080731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60020/0 (socket says 172.21.15.161:60020) 2024-03-30T00:31:04.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.551+0000 7fa406000700 1 -- 172.21.15.161:0/2317522841 learned_addr learned my addr 172.21.15.161:0/2317522841 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-30T00:31:04.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.551+0000 7fa406a00700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa408100e00 0x7fa408074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:04.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.552+0000 7fa407400700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa408102920 0x7fa4080736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:04.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.552+0000 7fa406000700 1 -- 172.21.15.161:0/2317522841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa408100e00 msgr2=0x7fa408074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:04.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.552+0000 7fa406000700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa408100e00 0x7fa408074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:04.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.552+0000 7fa406000700 1 -- 172.21.15.161:0/2317522841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa408102920 msgr2=0x7fa4080736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:04.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.552+0000 7fa406000700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa408102920 0x7fa4080736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:04.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.552+0000 7fa406000700 1 -- 172.21.15.161:0/2317522841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f80095f0 con 0x7fa408101f60 2024-03-30T00:31:04.564 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.552+0000 7fa406000700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa408101f60 0x7fa4080731a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400c920 tx=0x7fa3f400cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:31:04.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.552+0000 7fa3ff400700 1 -- 172.21.15.161:0/2317522841 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4004a30 con 0x7fa408101f60 2024-03-30T00:31:04.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.552+0000 7fa3ff400700 1 -- 172.21.15.161:0/2317522841 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3f40054b0 con 0x7fa408101f60 2024-03-30T00:31:04.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.552+0000 7fa3ff400700 1 -- 172.21.15.161:0/2317522841 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4007b50 con 0x7fa408101f60 2024-03-30T00:31:04.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.552+0000 7fa40dc00700 1 -- 172.21.15.161:0/2317522841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4081a0f20 con 0x7fa408101f60 2024-03-30T00:31:04.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.553+0000 7fa40dc00700 1 -- 172.21.15.161:0/2317522841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4081a1340 con 0x7fa408101f60 2024-03-30T00:31:04.565 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.553+0000 7fa40dc00700 1 -- 172.21.15.161:0/2317522841 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4081082c0 con 0x7fa408101f60 2024-03-30T00:31:04.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.557+0000 7fa3ff400700 1 -- 172.21.15.161:0/2317522841 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa3f4007cb0 con 0x7fa408101f60 2024-03-30T00:31:04.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.558+0000 7fa3ff400700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3ec0a1120 0x7fa3ec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:04.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.558+0000 7fa3ff400700 1 -- 172.21.15.161:0/2317522841 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fa3f40cbd00 con 0x7fa408101f60 2024-03-30T00:31:04.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.559+0000 7fa406a00700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3ec0a1120 0x7fa3ec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:04.567 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.560+0000 7fa406a00700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3ec0a1120 0x7fa3ec0a35e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8003780 tx=0x7fa3f8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:31:04.568 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.561+0000 7fa3ff400700 1 -- 172.21.15.161:0/2317522841 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa3f4095f30 con 0x7fa408101f60 2024-03-30T00:31:04.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:04.769+0000 7fa40dc00700 1 -- 172.21.15.161:0/2317522841 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa40802cdc0 con 0x7fa3ec0a1120 2024-03-30T00:31:05.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:31:05.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:31:05.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:05.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:05 smithi142 ceph-mon[41779]: osdmap e52: 8 total, 8 up, 8 in 2024-03-30T00:31:05.356 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:31:05.356 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:31:05.356 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:05.356 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:05 smithi161 ceph-mon[42221]: osdmap e52: 8 total, 8 up, 8 in 2024-03-30T00:31:05.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:31:05.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:31:05.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:05.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:05 smithi099 ceph-mon[38079]: osdmap e52: 8 total, 8 up, 8 in 2024-03-30T00:31:06.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:06 smithi161 ceph-mon[42221]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:31:06.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:06 smithi161 ceph-mon[42221]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:06.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:06 smithi099 ceph-mon[38079]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:31:06.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:06 smithi099 ceph-mon[38079]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:06.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:06 smithi142 ceph-mon[41779]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:31:06.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:06 smithi142 ceph-mon[41779]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:07.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:07 smithi161 ceph-mon[42221]: osdmap e53: 8 total, 8 up, 8 in 2024-03-30T00:31:07.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:07 smithi099 ceph-mon[38079]: osdmap e53: 8 total, 8 up, 8 in 2024-03-30T00:31:07.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:07 smithi142 ceph-mon[41779]: osdmap e53: 8 total, 8 up, 8 in 2024-03-30T00:31:08.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:08 smithi161 ceph-mon[42221]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:08.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:08 smithi161 ceph-mon[42221]: osdmap e54: 8 total, 8 up, 8 in 2024-03-30T00:31:08.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:08 smithi161 ceph-mon[42221]: from='client.? 172.21.15.161:0/690883562' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc"}]: dispatch 2024-03-30T00:31:08.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:08 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc"}]: dispatch 2024-03-30T00:31:08.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:08 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc"}]': finished 2024-03-30T00:31:08.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:08 smithi161 ceph-mon[42221]: osdmap e55: 9 total, 8 up, 9 in 2024-03-30T00:31:08.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:08 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:08.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:08 smithi161 ceph-mon[42221]: from='client.? 172.21.15.161:0/2050300422' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:31:08.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:08 smithi099 ceph-mon[38079]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:08.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:08 smithi099 ceph-mon[38079]: osdmap e54: 8 total, 8 up, 8 in 2024-03-30T00:31:08.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:08 smithi099 ceph-mon[38079]: from='client.? 172.21.15.161:0/690883562' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc"}]: dispatch 2024-03-30T00:31:08.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:08 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc"}]: dispatch 2024-03-30T00:31:08.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:08 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc"}]': finished 2024-03-30T00:31:08.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:08 smithi099 ceph-mon[38079]: osdmap e55: 9 total, 8 up, 9 in 2024-03-30T00:31:08.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:08.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:08 smithi099 ceph-mon[38079]: from='client.? 172.21.15.161:0/2050300422' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:31:08.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:08 smithi142 ceph-mon[41779]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:08.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:08 smithi142 ceph-mon[41779]: osdmap e54: 8 total, 8 up, 8 in 2024-03-30T00:31:08.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:08 smithi142 ceph-mon[41779]: from='client.? 172.21.15.161:0/690883562' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc"}]: dispatch 2024-03-30T00:31:08.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:08 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc"}]: dispatch 2024-03-30T00:31:08.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:08 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc"}]': finished 2024-03-30T00:31:08.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:08 smithi142 ceph-mon[41779]: osdmap e55: 9 total, 8 up, 9 in 2024-03-30T00:31:08.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:08 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:08.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:08 smithi142 ceph-mon[41779]: from='client.? 172.21.15.161:0/2050300422' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:31:10.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:10 smithi161 ceph-mon[42221]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:10.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:10 smithi099 ceph-mon[38079]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:10 smithi142 ceph-mon[41779]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:12.394 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:12 smithi161 ceph-mon[42221]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:12.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:12 smithi099 ceph-mon[38079]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:12.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:12 smithi142 ceph-mon[41779]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:14.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:14 smithi161 ceph-mon[42221]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:14.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-30T00:31:14.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:14 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:14.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:14 smithi099 ceph-mon[38079]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:14.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-30T00:31:14.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:14 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:14.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:14 smithi142 ceph-mon[41779]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:14.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-30T00:31:14.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:14 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:15.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:15 smithi161 ceph-mon[42221]: Deploying daemon osd.8 on smithi161 2024-03-30T00:31:15.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:15 smithi099 ceph-mon[38079]: Deploying daemon osd.8 on smithi161 2024-03-30T00:31:15.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:15 smithi142 ceph-mon[41779]: Deploying daemon osd.8 on smithi161 2024-03-30T00:31:16.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:16 smithi161 ceph-mon[42221]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:16.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:16 smithi099 ceph-mon[38079]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:16 smithi142 ceph-mon[41779]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:18.397 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:18 smithi161 ceph-mon[42221]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:18.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:18 smithi099 ceph-mon[38079]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:18.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:18 smithi142 ceph-mon[41779]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:20.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:20 smithi161 ceph-mon[42221]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:20.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:20.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:20.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:20 smithi099 ceph-mon[38079]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:20 smithi142 ceph-mon[41779]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:20.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:20.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:21.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.985+0000 7fa3ff400700 1 -- 172.21.15.161:0/2317522841 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa40802cdc0 con 0x7fa3ec0a1120 2024-03-30T00:31:21.994 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 8 on host 'smithi161' 2024-03-30T00:31:21.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.987+0000 7fa40dc00700 1 -- 172.21.15.161:0/2317522841 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3ec0a1120 msgr2=0x7fa3ec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:21.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.987+0000 7fa40dc00700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3ec0a1120 0x7fa3ec0a35e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8003780 tx=0x7fa3f8005fb0 comp rx=0 tx=0).stop 2024-03-30T00:31:21.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.987+0000 7fa40dc00700 1 -- 172.21.15.161:0/2317522841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa408101f60 msgr2=0x7fa4080731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:21.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.987+0000 7fa40dc00700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa408101f60 0x7fa4080731a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400c920 tx=0x7fa3f400cc30 comp rx=0 tx=0).stop 2024-03-30T00:31:21.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.988+0000 7fa40dc00700 1 -- 172.21.15.161:0/2317522841 shutdown_connections 2024-03-30T00:31:21.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.988+0000 7fa40dc00700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa408102920 0x7fa4080736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:21.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.988+0000 7fa40dc00700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3ec0a1120 0x7fa3ec0a35e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:21.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.988+0000 7fa40dc00700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa408100e00 0x7fa408074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:21.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.988+0000 7fa40dc00700 1 --2- 172.21.15.161:0/2317522841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa408101f60 0x7fa4080731a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:21.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.988+0000 7fa40dc00700 1 -- 172.21.15.161:0/2317522841 >> 172.21.15.161:0/2317522841 conn(0x7fa4080fc3c0 msgr2=0x7fa408105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:31:21.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.988+0000 7fa40dc00700 1 -- 172.21.15.161:0/2317522841 shutdown_connections 2024-03-30T00:31:21.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:31:21.988+0000 7fa40dc00700 1 -- 172.21.15.161:0/2317522841 wait complete. 2024-03-30T00:31:22.414 DEBUG:teuthology.orchestra.run.smithi161:osd.8> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.8.service 2024-03-30T00:31:22.418 INFO:tasks.cephadm:Deploying osd.9 on smithi099 with /dev/vg_nvme/lv_1... 2024-03-30T00:31:22.419 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-30T00:31:22.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:22 smithi161 ceph-mon[42221]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:22.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:22.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:22.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:22.494 INFO:journalctl@ceph.osd.8.smithi161.stdout:-- Logs begin at Sat 2024-03-30 00:10:26 UTC. -- 2024-03-30T00:31:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:22 smithi142 ceph-mon[41779]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:22.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:22 smithi099 ceph-mon[38079]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:22.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:22.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:22.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:22.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:22.782 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:31:23.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:23 smithi161 ceph-mon[42221]: from='osd.8 [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T00:31:23.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:23 smithi161 ceph-mon[42221]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T00:31:23.470 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:31:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:31:23.188+0000 7ff841ac2640 -1 osd.8 0 log_to_monitors true 2024-03-30T00:31:23.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:23 smithi142 ceph-mon[41779]: from='osd.8 [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T00:31:23.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:23 smithi142 ceph-mon[41779]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T00:31:23.703 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:23 smithi099 ceph-mon[38079]: from='osd.8 [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T00:31:23.703 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:23 smithi099 ceph-mon[38079]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T00:31:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-30T00:31:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-30T00:31:24.487 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10+0 records in 2024-03-30T00:31:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:10+0 records out 2024-03-30T00:31:24.487 INFO:teuthology.orchestra.run.smithi099.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0374787 s, 280 MB/s 2024-03-30T00:31:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:--> Zapping successful for: 2024-03-30T00:31:24.525 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:24 smithi099 ceph-mon[38079]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:24.525 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:24 smithi099 ceph-mon[38079]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-30T00:31:24.526 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:24 smithi099 ceph-mon[38079]: osdmap e56: 9 total, 8 up, 9 in 2024-03-30T00:31:24.526 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:24 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:24.526 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:24 smithi099 ceph-mon[38079]: from='osd.8 [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:31:24.526 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:24 smithi099 ceph-mon[38079]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:31:24.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:24 smithi142 ceph-mon[41779]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:24.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:24 smithi142 ceph-mon[41779]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-30T00:31:24.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:24 smithi142 ceph-mon[41779]: osdmap e56: 9 total, 8 up, 9 in 2024-03-30T00:31:24.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:24 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:24.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:24 smithi142 ceph-mon[41779]: from='osd.8 [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:31:24.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:24 smithi142 ceph-mon[41779]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:31:24.719 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:31:24 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:31:24.260+0000 7ff835600700 -1 osd.8 0 waiting for initial osdmap 2024-03-30T00:31:24.720 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:31:24 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:31:24.283+0000 7ff830e00700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:31:24.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:24 smithi161 ceph-mon[42221]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:24.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:24 smithi161 ceph-mon[42221]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-30T00:31:24.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:24 smithi161 ceph-mon[42221]: osdmap e56: 9 total, 8 up, 9 in 2024-03-30T00:31:24.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:24 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:24.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:24 smithi161 ceph-mon[42221]: from='osd.8 [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:31:24.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:24 smithi161 ceph-mon[42221]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:31:24.918 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2024-03-30T00:31:25.274 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:31:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:25 smithi142 ceph-mon[41779]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:31:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:25 smithi142 ceph-mon[41779]: osdmap e57: 9 total, 8 up, 9 in 2024-03-30T00:31:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:25 smithi142 ceph-mon[41779]: osd.8 [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120] boot 2024-03-30T00:31:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:25 smithi142 ceph-mon[41779]: osdmap e58: 9 total, 9 up, 9 in 2024-03-30T00:31:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:25.569 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:25 smithi099 ceph-mon[38079]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:31:25.569 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:25 smithi099 ceph-mon[38079]: osdmap e57: 9 total, 8 up, 9 in 2024-03-30T00:31:25.569 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:25.569 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:25.569 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:25 smithi099 ceph-mon[38079]: osd.8 [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120] boot 2024-03-30T00:31:25.569 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:25 smithi099 ceph-mon[38079]: osdmap e58: 9 total, 9 up, 9 in 2024-03-30T00:31:25.569 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:25.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:25 smithi161 ceph-mon[42221]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:31:25.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:25 smithi161 ceph-mon[42221]: osdmap e57: 9 total, 8 up, 9 in 2024-03-30T00:31:25.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:25.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:25.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:25 smithi161 ceph-mon[42221]: osd.8 [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120] boot 2024-03-30T00:31:25.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:25 smithi161 ceph-mon[42221]: osdmap e58: 9 total, 9 up, 9 in 2024-03-30T00:31:25.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T00:31:26.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.448+0000 7f37c9000700 1 -- 172.21.15.99:0/2266296890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37c4101f60 msgr2=0x7f37c41023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:26.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.448+0000 7f37c9000700 1 --2- 172.21.15.99:0/2266296890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37c4101f60 0x7f37c41023e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009990 tx=0x7f37b0009ca0 comp rx=0 tx=0).stop 2024-03-30T00:31:26.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.448+0000 7f37c9000700 1 -- 172.21.15.99:0/2266296890 shutdown_connections 2024-03-30T00:31:26.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.448+0000 7f37c9000700 1 --2- 172.21.15.99:0/2266296890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37c4102920 0x7f37c4109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:26.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.448+0000 7f37c9000700 1 --2- 172.21.15.99:0/2266296890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c4100e00 0x7f37c4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:26.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.448+0000 7f37c9000700 1 --2- 172.21.15.99:0/2266296890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37c4101f60 0x7f37c41023e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:26.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.448+0000 7f37c9000700 1 -- 172.21.15.99:0/2266296890 >> 172.21.15.99:0/2266296890 conn(0x7f37c40fc3c0 msgr2=0x7f37c40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:31:26.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.448+0000 7f37c9000700 1 -- 172.21.15.99:0/2266296890 shutdown_connections 2024-03-30T00:31:26.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.448+0000 7f37c9000700 1 -- 172.21.15.99:0/2266296890 wait complete. 2024-03-30T00:31:26.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.449+0000 7f37c9000700 1 Processor -- start 2024-03-30T00:31:26.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.449+0000 7f37c9000700 1 -- start start 2024-03-30T00:31:26.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c9000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37c4100e00 0x7f37c4074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:26.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37c4101f60 0x7f37c40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:26.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c9000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c4102920 0x7f37c40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:26.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c9000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4073dc0 con 0x7f37c4100e00 2024-03-30T00:31:26.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c9000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4073f30 con 0x7f37c4101f60 2024-03-30T00:31:26.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c9000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c41a0db0 con 0x7f37c4102920 2024-03-30T00:31:26.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37c4100e00 0x7f37c4074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37c4100e00 0x7f37c4074ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56650/0 (socket says 172.21.15.99:56650) 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c1e00700 1 -- 172.21.15.99:0/2964925763 learned_addr learned my addr 172.21.15.99:0/2964925763 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c1400700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37c4101f60 0x7f37c40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c2800700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c4102920 0x7f37c40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c1e00700 1 -- 172.21.15.99:0/2964925763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c4102920 msgr2=0x7f37c40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c1e00700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c4102920 0x7f37c40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c1e00700 1 -- 172.21.15.99:0/2964925763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37c4101f60 msgr2=0x7f37c40731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c1e00700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37c4101f60 0x7f37c40731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.450+0000 7f37c1e00700 1 -- 172.21.15.99:0/2964925763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00095f0 con 0x7f37c4100e00 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.451+0000 7f37c1e00700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37c4100e00 0x7f37c4074ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f37b400b530 tx=0x7f37b400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.451+0000 7f37baa00700 1 -- 172.21.15.99:0/2964925763 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4011830 con 0x7f37c4100e00 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.451+0000 7f37baa00700 1 -- 172.21.15.99:0/2964925763 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37b4011e20 con 0x7f37c4100e00 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.451+0000 7f37c9000700 1 -- 172.21.15.99:0/2964925763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c41a0fb0 con 0x7f37c4100e00 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.451+0000 7f37baa00700 1 -- 172.21.15.99:0/2964925763 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4010dd0 con 0x7f37c4100e00 2024-03-30T00:31:26.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.451+0000 7f37c9000700 1 -- 172.21.15.99:0/2964925763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c41a1470 con 0x7f37c4100e00 2024-03-30T00:31:26.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.452+0000 7f37c9000700 1 -- 172.21.15.99:0/2964925763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c4108320 con 0x7f37c4100e00 2024-03-30T00:31:26.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.454+0000 7f37baa00700 1 -- 172.21.15.99:0/2964925763 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f37b4011990 con 0x7f37c4100e00 2024-03-30T00:31:26.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.455+0000 7f37baa00700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37a80a11a0 0x7f37a80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:26.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.456+0000 7f37baa00700 1 -- 172.21.15.99:0/2964925763 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f37b40cbe20 con 0x7f37c4100e00 2024-03-30T00:31:26.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.456+0000 7f37c1400700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37a80a11a0 0x7f37a80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:26.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.456+0000 7f37c1400700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37a80a11a0 0x7f37a80a3660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f37b0006010 tx=0x7f37b0003740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.458+0000 7f37baa00700 1 -- 172.21.15.99:0/2964925763 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f37b4095ea0 con 0x7f37c4100e00 2024-03-30T00:31:26.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:31:26.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:31:26.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:26.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi161 2024-03-30T00:31:26.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:26.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:26.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:26.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi161 to 5756M 2024-03-30T00:31:26.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:26.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:26.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:26 smithi099 ceph-mon[38079]: osdmap e59: 9 total, 9 up, 9 in 2024-03-30T00:31:26.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:26.670+0000 7f37c9000700 1 -- 172.21.15.99:0/2964925763 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f37c402cdc0 con 0x7f37a80a11a0 2024-03-30T00:31:26.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:31:26.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:31:26.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:26.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi161 2024-03-30T00:31:26.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:26.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:26.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:26.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi161 to 5756M 2024-03-30T00:31:26.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:26.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:26.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:26 smithi161 ceph-mon[42221]: osdmap e59: 9 total, 9 up, 9 in 2024-03-30T00:31:26.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:31:26.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:31:26.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-03-30T00:31:26.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi161 2024-03-30T00:31:26.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:26.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:26.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:26.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi161 to 5756M 2024-03-30T00:31:26.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:26.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:26.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:26.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:26 smithi142 ceph-mon[41779]: osdmap e59: 9 total, 9 up, 9 in 2024-03-30T00:31:27.588 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:31:27.588 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:31:27.589 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:27.589 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:27 smithi099 ceph-mon[38079]: osdmap e60: 9 total, 9 up, 9 in 2024-03-30T00:31:27.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:31:27.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:31:27.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:27.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:27 smithi161 ceph-mon[42221]: osdmap e60: 9 total, 9 up, 9 in 2024-03-30T00:31:27.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:31:27.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:31:27.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:27.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:27 smithi142 ceph-mon[41779]: osdmap e60: 9 total, 9 up, 9 in 2024-03-30T00:31:28.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:28 smithi161 ceph-mon[42221]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:31:28.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:28 smithi161 ceph-mon[42221]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:28.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:28 smithi161 ceph-mon[42221]: osdmap e61: 9 total, 9 up, 9 in 2024-03-30T00:31:28.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:28 smithi099 ceph-mon[38079]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:31:28.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:28 smithi099 ceph-mon[38079]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:28.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:28 smithi099 ceph-mon[38079]: osdmap e61: 9 total, 9 up, 9 in 2024-03-30T00:31:28.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:28 smithi142 ceph-mon[41779]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:31:28.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:28 smithi142 ceph-mon[41779]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:28.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:28 smithi142 ceph-mon[41779]: osdmap e61: 9 total, 9 up, 9 in 2024-03-30T00:31:30.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:29 smithi161 ceph-mon[42221]: osdmap e62: 9 total, 9 up, 9 in 2024-03-30T00:31:30.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:29 smithi161 ceph-mon[42221]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:30.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:29 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3158641969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cb6f263-248a-417e-b8a4-37afed246071"}]: dispatch 2024-03-30T00:31:30.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:29 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3158641969' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cb6f263-248a-417e-b8a4-37afed246071"}]': finished 2024-03-30T00:31:30.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:29 smithi161 ceph-mon[42221]: osdmap e63: 10 total, 9 up, 10 in 2024-03-30T00:31:30.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:29 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:30.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:29 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3361926440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:31:30.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:29 smithi099 ceph-mon[38079]: osdmap e62: 9 total, 9 up, 9 in 2024-03-30T00:31:30.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:29 smithi099 ceph-mon[38079]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:29 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3158641969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cb6f263-248a-417e-b8a4-37afed246071"}]: dispatch 2024-03-30T00:31:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:29 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3158641969' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cb6f263-248a-417e-b8a4-37afed246071"}]': finished 2024-03-30T00:31:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:29 smithi099 ceph-mon[38079]: osdmap e63: 10 total, 9 up, 10 in 2024-03-30T00:31:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:29 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:29 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3361926440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:31:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:29 smithi142 ceph-mon[41779]: osdmap e62: 9 total, 9 up, 9 in 2024-03-30T00:31:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:29 smithi142 ceph-mon[41779]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:29 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3158641969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cb6f263-248a-417e-b8a4-37afed246071"}]: dispatch 2024-03-30T00:31:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:29 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3158641969' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cb6f263-248a-417e-b8a4-37afed246071"}]': finished 2024-03-30T00:31:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:29 smithi142 ceph-mon[41779]: osdmap e63: 10 total, 9 up, 10 in 2024-03-30T00:31:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:29 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:29 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3361926440' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:31:32.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:32 smithi142 ceph-mon[41779]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-03-30T00:31:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:32 smithi161 ceph-mon[42221]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-03-30T00:31:32.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:32 smithi099 ceph-mon[38079]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-03-30T00:31:34.316 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:34 smithi099 ceph-mon[38079]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T00:31:34.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:34 smithi142 ceph-mon[41779]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T00:31:34.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:34 smithi161 ceph-mon[42221]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T00:31:36.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:35 smithi161 ceph-mon[42221]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-30T00:31:36.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:35 smithi099 ceph-mon[38079]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-30T00:31:36.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:35 smithi142 ceph-mon[41779]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-30T00:31:37.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-30T00:31:37.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:36 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:37.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:36 smithi161 ceph-mon[42221]: Deploying daemon osd.9 on smithi099 2024-03-30T00:31:37.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-30T00:31:37.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:36 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:37.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:36 smithi099 ceph-mon[38079]: Deploying daemon osd.9 on smithi099 2024-03-30T00:31:37.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-30T00:31:37.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:36 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:37.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:36 smithi142 ceph-mon[41779]: Deploying daemon osd.9 on smithi099 2024-03-30T00:31:37.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:37 smithi099 ceph-mon[38079]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-30T00:31:38.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:37 smithi161 ceph-mon[42221]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-30T00:31:38.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:37 smithi142 ceph-mon[41779]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-30T00:31:40.219 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:40 smithi099 ceph-mon[38079]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-30T00:31:40.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:40 smithi142 ceph-mon[41779]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-30T00:31:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:40 smithi161 ceph-mon[42221]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-30T00:31:42.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:41 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:42.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:41 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:42.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:41 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:42.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:41 smithi161 ceph-mon[42221]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T00:31:42.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:41 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:42.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:41 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:42.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:41 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:42.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:41 smithi099 ceph-mon[38079]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T00:31:42.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:41 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:42.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:41 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:42.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:41 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:42.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:41 smithi142 ceph-mon[41779]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T00:31:43.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.778+0000 7f37baa00700 1 -- 172.21.15.99:0/2964925763 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f37c402cdc0 con 0x7f37a80a11a0 2024-03-30T00:31:43.788 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 9 on host 'smithi099' 2024-03-30T00:31:43.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.781+0000 7f37c9000700 1 -- 172.21.15.99:0/2964925763 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37a80a11a0 msgr2=0x7f37a80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:43.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.781+0000 7f37c9000700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37a80a11a0 0x7f37a80a3660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f37b0006010 tx=0x7f37b0003740 comp rx=0 tx=0).stop 2024-03-30T00:31:43.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.781+0000 7f37c9000700 1 -- 172.21.15.99:0/2964925763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37c4100e00 msgr2=0x7f37c4074ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:43.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.781+0000 7f37c9000700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37c4100e00 0x7f37c4074ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f37b400b530 tx=0x7f37b400b8f0 comp rx=0 tx=0).stop 2024-03-30T00:31:43.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.781+0000 7f37c9000700 1 -- 172.21.15.99:0/2964925763 shutdown_connections 2024-03-30T00:31:43.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.781+0000 7f37c9000700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37c4102920 0x7f37c40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:43.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.781+0000 7f37c9000700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37c4100e00 0x7f37c4074ab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:43.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.781+0000 7f37c9000700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37a80a11a0 0x7f37a80a3660 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:43.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.781+0000 7f37c9000700 1 --2- 172.21.15.99:0/2964925763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37c4101f60 0x7f37c40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:43.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.781+0000 7f37c9000700 1 -- 172.21.15.99:0/2964925763 >> 172.21.15.99:0/2964925763 conn(0x7f37c40fc3c0 msgr2=0x7f37c4105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:31:43.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.782+0000 7f37c9000700 1 -- 172.21.15.99:0/2964925763 shutdown_connections 2024-03-30T00:31:43.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:31:43.782+0000 7f37c9000700 1 -- 172.21.15.99:0/2964925763 wait complete. 2024-03-30T00:31:43.976 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:43.976 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:43.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:43 smithi099 ceph-mon[38079]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:43.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:43.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:43.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:43.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:43.977 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:43 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:44.163 DEBUG:teuthology.orchestra.run.smithi099:osd.9> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.9.service 2024-03-30T00:31:44.168 INFO:tasks.cephadm:Deploying osd.10 on smithi142 with /dev/vg_nvme/lv_1... 2024-03-30T00:31:44.168 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-30T00:31:44.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:44.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:44.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:43 smithi161 ceph-mon[42221]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:44.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:44.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:44.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:44.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:44.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:43 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:44.234 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:44.234 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:44.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:43 smithi142 ceph-mon[41779]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:44.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:44.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:44.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:44.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:44.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:43 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:44.246 INFO:journalctl@ceph.osd.9.smithi099.stdout:-- Logs begin at Sat 2024-03-30 00:10:25 UTC. -- 2024-03-30T00:31:44.526 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:31:44.980 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:44 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:44.980 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:44 smithi142 ceph-mon[41779]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-30T00:31:45.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:44 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:45.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:44 smithi161 ceph-mon[42221]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-30T00:31:45.237 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:31:44 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:31:44.874+0000 7f3f14fba640 -1 osd.9 0 log_to_monitors true 2024-03-30T00:31:45.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:44 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:31:45.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:44 smithi099 ceph-mon[38079]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-30T00:31:46.202 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-30T00:31:46.202 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-30T00:31:46.202 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-03-30T00:31:46.202 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-03-30T00:31:46.202 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0292676 s, 358 MB/s 2024-03-30T00:31:46.202 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-03-30T00:31:46.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:45 smithi161 ceph-mon[42221]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:46.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:45 smithi161 ceph-mon[42221]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-30T00:31:46.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:45 smithi161 ceph-mon[42221]: osdmap e64: 10 total, 9 up, 10 in 2024-03-30T00:31:46.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:46.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:45 smithi161 ceph-mon[42221]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:31:46.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:45 smithi099 ceph-mon[38079]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:46.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:45 smithi099 ceph-mon[38079]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-30T00:31:46.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:45 smithi099 ceph-mon[38079]: osdmap e64: 10 total, 9 up, 10 in 2024-03-30T00:31:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:45 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:46.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:45 smithi099 ceph-mon[38079]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:31:46.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:45 smithi142 ceph-mon[41779]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:46.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:45 smithi142 ceph-mon[41779]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-30T00:31:46.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:45 smithi142 ceph-mon[41779]: osdmap e64: 10 total, 9 up, 10 in 2024-03-30T00:31:46.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:45 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:46.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:45 smithi142 ceph-mon[41779]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-03-30T00:31:46.627 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2024-03-30T00:31:46.895 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:31:46 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:31:46.870+0000 7f3f08c00700 -1 osd.9 0 waiting for initial osdmap 2024-03-30T00:31:46.976 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:31:47.238 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:31:46 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:31:46.888+0000 7f3f04400700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:31:48.114 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.107+0000 7f494ac00700 1 -- 172.21.15.142:0/3196969648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944100e00 msgr2=0x7f4944101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:48.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.107+0000 7f494ac00700 1 --2- 172.21.15.142:0/3196969648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944100e00 0x7f4944101220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4934009990 tx=0x7f4934009ca0 comp rx=0 tx=0).stop 2024-03-30T00:31:48.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.107+0000 7f494ac00700 1 -- 172.21.15.142:0/3196969648 shutdown_connections 2024-03-30T00:31:48.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.107+0000 7f494ac00700 1 --2- 172.21.15.142:0/3196969648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4944102920 0x7f4944109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:48.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.107+0000 7f494ac00700 1 --2- 172.21.15.142:0/3196969648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944100e00 0x7f4944101220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:48.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.107+0000 7f494ac00700 1 --2- 172.21.15.142:0/3196969648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4944101f60 0x7f49441023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:48.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.107+0000 7f494ac00700 1 -- 172.21.15.142:0/3196969648 >> 172.21.15.142:0/3196969648 conn(0x7f49440fc3c0 msgr2=0x7f49440fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:31:48.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.108+0000 7f494ac00700 1 -- 172.21.15.142:0/3196969648 shutdown_connections 2024-03-30T00:31:48.115 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.108+0000 7f494ac00700 1 -- 172.21.15.142:0/3196969648 wait complete. 2024-03-30T00:31:48.116 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.109+0000 7f494ac00700 1 Processor -- start 2024-03-30T00:31:48.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.109+0000 7f494ac00700 1 -- start start 2024-03-30T00:31:48.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.109+0000 7f494ac00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4944100e00 0x7f4944074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:48.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.109+0000 7f494ac00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4944101f60 0x7f49440731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:48.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.109+0000 7f494ac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944102920 0x7f49440736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:48.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.109+0000 7f494ac00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944073e80 con 0x7f4944101f60 2024-03-30T00:31:48.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.109+0000 7f494ac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944073ff0 con 0x7f4944102920 2024-03-30T00:31:48.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.109+0000 7f494ac00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49441a0c70 con 0x7f4944100e00 2024-03-30T00:31:48.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4948c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944102920 0x7f49440736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:48.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4948c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944102920 0x7f49440736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:55818/0 (socket says 172.21.15.142:55818) 2024-03-30T00:31:48.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4948c00700 1 -- 172.21.15.142:0/1316672523 learned_addr learned my addr 172.21.15.142:0/1316672523 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-03-30T00:31:48.117 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4943400700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4944101f60 0x7f49440731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:48.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4943e00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4944100e00 0x7f4944074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:48.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4948c00700 1 -- 172.21.15.142:0/1316672523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4944100e00 msgr2=0x7f4944074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:48.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4948c00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4944100e00 0x7f4944074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:48.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4948c00700 1 -- 172.21.15.142:0/1316672523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4944101f60 msgr2=0x7f49440731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:31:48.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4948c00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4944101f60 0x7f49440731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:31:48.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4948c00700 1 -- 172.21.15.142:0/1316672523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340095f0 con 0x7f4944102920 2024-03-30T00:31:48.118 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4948c00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944102920 0x7f49440736e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f493c007d70 tx=0x7f493c00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:31:48.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.110+0000 7f4940c00700 1 -- 172.21.15.142:0/1316672523 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c00ad40 con 0x7f4944102920 2024-03-30T00:31:48.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.111+0000 7f4940c00700 1 -- 172.21.15.142:0/1316672523 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f493c0128b0 con 0x7f4944102920 2024-03-30T00:31:48.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.111+0000 7f494ac00700 1 -- 172.21.15.142:0/1316672523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49441a0f00 con 0x7f4944102920 2024-03-30T00:31:48.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.111+0000 7f4940c00700 1 -- 172.21.15.142:0/1316672523 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c0115a0 con 0x7f4944102920 2024-03-30T00:31:48.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.111+0000 7f494ac00700 1 -- 172.21.15.142:0/1316672523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49441a1320 con 0x7f4944102920 2024-03-30T00:31:48.119 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.112+0000 7f494ac00700 1 -- 172.21.15.142:0/1316672523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4944108280 con 0x7f4944102920 2024-03-30T00:31:48.121 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.115+0000 7f4940c00700 1 -- 172.21.15.142:0/1316672523 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f493c012a20 con 0x7f4944102920 2024-03-30T00:31:48.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.116+0000 7f4940c00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49280a1120 0x7f49280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:31:48.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.116+0000 7f4940c00700 1 -- 172.21.15.142:0/1316672523 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f493c0cc760 con 0x7f4944102920 2024-03-30T00:31:48.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.116+0000 7f4943e00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49280a1120 0x7f49280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:31:48.123 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.117+0000 7f4943e00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49280a1120 0x7f49280a35e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f493400b610 tx=0x7f4934005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:31:48.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.119+0000 7f4940c00700 1 -- 172.21.15.142:0/1316672523 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f493c096650 con 0x7f4944102920 2024-03-30T00:31:48.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: osdmap e65: 10 total, 9 up, 10 in 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi099 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi099 to 4317M 2024-03-30T00:31:48.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:48.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:48.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:31:48.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: osdmap e65: 10 total, 9 up, 10 in 2024-03-30T00:31:48.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:48.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:48.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:48.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi099 2024-03-30T00:31:48.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi099 to 4317M 2024-03-30T00:31:48.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:48.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:48.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-03-30T00:31:48.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: osdmap e65: 10 total, 9 up, 10 in 2024-03-30T00:31:48.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:48.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:48.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-03-30T00:31:48.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi099 2024-03-30T00:31:48.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:31:48.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi099 to 4317M 2024-03-30T00:31:48.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:48.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:31:48.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:31:48.338 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:31:48.330+0000 7f494ac00700 1 -- 172.21.15.142:0/1316672523 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f494402cd20 con 0x7f49280a1120 2024-03-30T00:31:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:48 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:31:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:48 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:31:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:48 smithi161 ceph-mon[42221]: osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896] boot 2024-03-30T00:31:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:48 smithi161 ceph-mon[42221]: osdmap e66: 10 total, 10 up, 10 in 2024-03-30T00:31:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:48 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:49.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:48 smithi161 ceph-mon[42221]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:31:49.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:48 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:31:49.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:48 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:31:49.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:48 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:49.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:48 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:31:49.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:48 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:31:49.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:48 smithi099 ceph-mon[38079]: osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896] boot 2024-03-30T00:31:49.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:48 smithi099 ceph-mon[38079]: osdmap e66: 10 total, 10 up, 10 in 2024-03-30T00:31:49.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:48 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:49.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:48 smithi099 ceph-mon[38079]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:31:49.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:48 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:31:49.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:48 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:31:49.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:48 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:49.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:48 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:31:49.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:48 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:31:49.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:48 smithi142 ceph-mon[41779]: osd.9 [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896] boot 2024-03-30T00:31:49.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:48 smithi142 ceph-mon[41779]: osdmap e66: 10 total, 10 up, 10 in 2024-03-30T00:31:49.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:48 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T00:31:49.271 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:48 smithi142 ceph-mon[41779]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:31:49.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:48 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:31:49.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:48 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:31:49.272 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:48 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:50.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:49 smithi142 ceph-mon[41779]: osdmap e67: 10 total, 10 up, 10 in 2024-03-30T00:31:50.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:49 smithi142 ceph-mon[41779]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:50.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:49 smithi161 ceph-mon[42221]: osdmap e67: 10 total, 10 up, 10 in 2024-03-30T00:31:50.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:49 smithi161 ceph-mon[42221]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:50.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:49 smithi099 ceph-mon[38079]: osdmap e67: 10 total, 10 up, 10 in 2024-03-30T00:31:50.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:49 smithi099 ceph-mon[38079]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:51.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:50 smithi161 ceph-mon[42221]: from='client.? 172.21.15.142:0/2988216465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2d0d43f-b79c-4830-926f-8d51fb41eccd"}]: dispatch 2024-03-30T00:31:51.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:50 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2d0d43f-b79c-4830-926f-8d51fb41eccd"}]: dispatch 2024-03-30T00:31:51.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:50 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2d0d43f-b79c-4830-926f-8d51fb41eccd"}]': finished 2024-03-30T00:31:51.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:50 smithi161 ceph-mon[42221]: osdmap e68: 11 total, 10 up, 11 in 2024-03-30T00:31:51.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:31:51.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:50 smithi099 ceph-mon[38079]: from='client.? 172.21.15.142:0/2988216465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2d0d43f-b79c-4830-926f-8d51fb41eccd"}]: dispatch 2024-03-30T00:31:51.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:50 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2d0d43f-b79c-4830-926f-8d51fb41eccd"}]: dispatch 2024-03-30T00:31:51.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:50 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2d0d43f-b79c-4830-926f-8d51fb41eccd"}]': finished 2024-03-30T00:31:51.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:50 smithi099 ceph-mon[38079]: osdmap e68: 11 total, 10 up, 11 in 2024-03-30T00:31:51.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:31:51.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:50 smithi142 ceph-mon[41779]: from='client.? 172.21.15.142:0/2988216465' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2d0d43f-b79c-4830-926f-8d51fb41eccd"}]: dispatch 2024-03-30T00:31:51.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:50 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2d0d43f-b79c-4830-926f-8d51fb41eccd"}]: dispatch 2024-03-30T00:31:51.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:50 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2d0d43f-b79c-4830-926f-8d51fb41eccd"}]': finished 2024-03-30T00:31:51.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:50 smithi142 ceph-mon[41779]: osdmap e68: 11 total, 10 up, 11 in 2024-03-30T00:31:51.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:50 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:31:52.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:51 smithi099 ceph-mon[38079]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:52.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:51 smithi099 ceph-mon[38079]: from='client.? 172.21.15.142:0/3826416129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:31:52.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:51 smithi142 ceph-mon[41779]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:52.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:51 smithi142 ceph-mon[41779]: from='client.? 172.21.15.142:0/3826416129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:31:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:51 smithi161 ceph-mon[42221]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:51 smithi161 ceph-mon[42221]: from='client.? 172.21.15.142:0/3826416129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:31:54.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:54 smithi142 ceph-mon[41779]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:54 smithi161 ceph-mon[42221]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:54.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:54 smithi099 ceph-mon[38079]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:56.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:55 smithi161 ceph-mon[42221]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:56.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:55 smithi099 ceph-mon[38079]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:56.257 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:55 smithi142 ceph-mon[41779]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:58.313 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:58 smithi142 ceph-mon[41779]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:58.314 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-30T00:31:58.314 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:58 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:58 smithi161 ceph-mon[42221]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-30T00:31:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:58 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:58.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:58 smithi099 ceph-mon[38079]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:31:58.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-30T00:31:58.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:58 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:31:59.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:31:59 smithi142 ceph-mon[41779]: Deploying daemon osd.10 on smithi142 2024-03-30T00:31:59.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:31:59 smithi161 ceph-mon[42221]: Deploying daemon osd.10 on smithi142 2024-03-30T00:31:59.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:31:59 smithi099 ceph-mon[38079]: Deploying daemon osd.10 on smithi142 2024-03-30T00:32:00.309 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:00 smithi142 ceph-mon[41779]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:00.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:00 smithi161 ceph-mon[42221]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:00.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:00 smithi099 ceph-mon[38079]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:02.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:02 smithi142 ceph-mon[41779]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:02.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:02 smithi161 ceph-mon[42221]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:02.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:02 smithi099 ceph-mon[38079]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:03.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:03 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:03.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:03 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:03.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:03 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:03.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:03 smithi142 ceph-mon[41779]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:03.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:03 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:03.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:03 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:03.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:03 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:03.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:03 smithi161 ceph-mon[42221]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:03.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:03 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:03.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:03 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:03.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:03 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:03.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:03 smithi099 ceph-mon[38079]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:05.455 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.447+0000 7f4940c00700 1 -- 172.21.15.142:0/1316672523 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f494402cd20 con 0x7f49280a1120 2024-03-30T00:32:05.455 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 10 on host 'smithi142' 2024-03-30T00:32:05.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 -- 172.21.15.142:0/1316672523 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49280a1120 msgr2=0x7f49280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:05.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49280a1120 0x7f49280a35e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f493400b610 tx=0x7f4934005fb0 comp rx=0 tx=0).stop 2024-03-30T00:32:05.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 -- 172.21.15.142:0/1316672523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944102920 msgr2=0x7f49440736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:05.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944102920 0x7f49440736e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f493c007d70 tx=0x7f493c00a340 comp rx=0 tx=0).stop 2024-03-30T00:32:05.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 -- 172.21.15.142:0/1316672523 shutdown_connections 2024-03-30T00:32:05.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f49280a1120 0x7f49280a35e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:05.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944102920 0x7f49440736e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:05.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4944100e00 0x7f4944074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:05.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 --2- 172.21.15.142:0/1316672523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4944101f60 0x7f49440731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:05.459 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 -- 172.21.15.142:0/1316672523 >> 172.21.15.142:0/1316672523 conn(0x7f49440fc3c0 msgr2=0x7f49441053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:05.459 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 -- 172.21.15.142:0/1316672523 shutdown_connections 2024-03-30T00:32:05.459 INFO:teuthology.orchestra.run.smithi142.stderr:2024-03-30T00:32:05.450+0000 7f494ac00700 1 -- 172.21.15.142:0/1316672523 wait complete. 2024-03-30T00:32:05.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:05.669 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:05.669 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.669 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.669 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.669 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:05.853 DEBUG:teuthology.orchestra.run.smithi142:osd.10> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.10.service 2024-03-30T00:32:05.858 INFO:tasks.cephadm:Deploying osd.11 on smithi161 with /dev/vg_nvme/lv_1... 2024-03-30T00:32:05.858 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-30T00:32:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.928 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:05.940 INFO:journalctl@ceph.osd.10.smithi142.stdout:-- Logs begin at Sat 2024-03-30 00:10:40 UTC. -- 2024-03-30T00:32:05.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:05.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:05.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:05.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:06.226 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:32:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:06 smithi161 ceph-mon[42221]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:06.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:06 smithi142 ceph-mon[41779]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:06.818 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:32:06 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:32:06.511+0000 7f592473d640 -1 osd.10 0 log_to_monitors true 2024-03-30T00:32:06.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:06 smithi099 ceph-mon[38079]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:07.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:07 smithi142 ceph-mon[41779]: from='osd.10 [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T00:32:07.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:07 smithi142 ceph-mon[41779]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T00:32:07.962 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-30T00:32:07.962 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-30T00:32:07.962 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2024-03-30T00:32:07.962 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2024-03-30T00:32:07.962 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0379022 s, 277 MB/s 2024-03-30T00:32:07.962 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2024-03-30T00:32:07.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:07 smithi161 ceph-mon[42221]: from='osd.10 [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T00:32:07.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:07 smithi161 ceph-mon[42221]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T00:32:07.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:07 smithi099 ceph-mon[38079]: from='osd.10 [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T00:32:07.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:07 smithi099 ceph-mon[38079]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T00:32:08.507 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-03-30T00:32:08.548 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:32:08 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:32:08.511+0000 7f5918200700 -1 osd.10 0 waiting for initial osdmap 2024-03-30T00:32:08.548 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:32:08 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:32:08.540+0000 7f5913a00700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:32:08.549 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:08 smithi142 ceph-mon[41779]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:08.549 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:08 smithi142 ceph-mon[41779]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-30T00:32:08.549 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:08 smithi142 ceph-mon[41779]: osdmap e69: 11 total, 10 up, 11 in 2024-03-30T00:32:08.549 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:08 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:08.549 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:08 smithi142 ceph-mon[41779]: from='osd.10 [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:32:08.549 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:08 smithi142 ceph-mon[41779]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:32:08.864 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:32:08.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:08 smithi161 ceph-mon[42221]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:08.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:08 smithi161 ceph-mon[42221]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-30T00:32:08.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:08 smithi161 ceph-mon[42221]: osdmap e69: 11 total, 10 up, 11 in 2024-03-30T00:32:08.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:08 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:08.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:08 smithi161 ceph-mon[42221]: from='osd.10 [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:32:08.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:08 smithi161 ceph-mon[42221]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:32:08.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:08 smithi099 ceph-mon[38079]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:08.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:08 smithi099 ceph-mon[38079]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-30T00:32:08.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:08 smithi099 ceph-mon[38079]: osdmap e69: 11 total, 10 up, 11 in 2024-03-30T00:32:08.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:08 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:08.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:08 smithi099 ceph-mon[38079]: from='osd.10 [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:32:08.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:08 smithi099 ceph-mon[38079]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-03-30T00:32:09.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: osdmap e70: 11 total, 10 up, 11 in 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi142 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi142 to 4317M 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:09.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.721 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:09 smithi161 ceph-mon[42221]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:32:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: osdmap e70: 11 total, 10 up, 11 in 2024-03-30T00:32:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi142 2024-03-30T00:32:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi142 to 4317M 2024-03-30T00:32:09.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:09.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:09.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:09 smithi142 ceph-mon[41779]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:09.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: osdmap e70: 11 total, 10 up, 11 in 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi142 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi142 to 4317M 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:09.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:09.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:09.989 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:09 smithi099 ceph-mon[38079]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-03-30T00:32:09.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.984+0000 7f32be000700 1 -- 172.21.15.161:0/120738462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b8102920 msgr2=0x7f32b8109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:09.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.985+0000 7f32be000700 1 --2- 172.21.15.161:0/120738462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b8102920 0x7f32b8109240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32b000b0a0 tx=0x7f32b000b3b0 comp rx=0 tx=0).stop 2024-03-30T00:32:09.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.985+0000 7f32be000700 1 -- 172.21.15.161:0/120738462 shutdown_connections 2024-03-30T00:32:09.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.985+0000 7f32be000700 1 --2- 172.21.15.161:0/120738462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b8102920 0x7f32b8109240 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:09.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.985+0000 7f32be000700 1 --2- 172.21.15.161:0/120738462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32b8100e00 0x7f32b8101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:09.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.985+0000 7f32be000700 1 --2- 172.21.15.161:0/120738462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b8101f60 0x7f32b81023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:09.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.985+0000 7f32be000700 1 -- 172.21.15.161:0/120738462 >> 172.21.15.161:0/120738462 conn(0x7f32b80fc3c0 msgr2=0x7f32b80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:09.992 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.985+0000 7f32be000700 1 -- 172.21.15.161:0/120738462 shutdown_connections 2024-03-30T00:32:09.993 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.985+0000 7f32be000700 1 -- 172.21.15.161:0/120738462 wait complete. 2024-03-30T00:32:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.986+0000 7f32be000700 1 Processor -- start 2024-03-30T00:32:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.986+0000 7f32be000700 1 -- start start 2024-03-30T00:32:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.986+0000 7f32be000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b8100e00 0x7f32b8198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.986+0000 7f32be000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b8101f60 0x7f32b8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.986+0000 7f32be000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32b8102920 0x7f32b81a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.986+0000 7f32be000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b81a0850 con 0x7f32b8102920 2024-03-30T00:32:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.986+0000 7f32be000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b81a0990 con 0x7f32b8101f60 2024-03-30T00:32:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.986+0000 7f32be000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b81a0b50 con 0x7f32b8100e00 2024-03-30T00:32:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32b7400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b8100e00 0x7f32b8198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:09.995 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32b7400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b8100e00 0x7f32b8198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:40814/0 (socket says 172.21.15.161:40814) 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32b7400700 1 -- 172.21.15.161:0/4085304751 learned_addr learned my addr 172.21.15.161:0/4085304751 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32b7e00700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32b8102920 0x7f32b81a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32b6a00700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b8101f60 0x7f32b8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32b7400700 1 -- 172.21.15.161:0/4085304751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b8101f60 msgr2=0x7f32b8199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32b7400700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b8101f60 0x7f32b8199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32b7400700 1 -- 172.21.15.161:0/4085304751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32b8102920 msgr2=0x7f32b81a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32b7400700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32b8102920 0x7f32b81a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32b7400700 1 -- 172.21.15.161:0/4085304751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b0009d00 con 0x7f32b8100e00 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32b7400700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b8100e00 0x7f32b8198d50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d950 tx=0x7f32a800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32afe00700 1 -- 172.21.15.161:0/4085304751 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8009940 con 0x7f32b8100e00 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.987+0000 7f32afe00700 1 -- 172.21.15.161:0/4085304751 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32a8010460 con 0x7f32b8100e00 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.988+0000 7f32afe00700 1 -- 172.21.15.161:0/4085304751 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a800f620 con 0x7f32b8100e00 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.988+0000 7f32be000700 1 -- 172.21.15.161:0/4085304751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b81a0e30 con 0x7f32b8100e00 2024-03-30T00:32:09.996 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.988+0000 7f32be000700 1 -- 172.21.15.161:0/4085304751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b81a11f0 con 0x7f32b8100e00 2024-03-30T00:32:09.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.989+0000 7f32be000700 1 -- 172.21.15.161:0/4085304751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b804ea80 con 0x7f32b8100e00 2024-03-30T00:32:09.997 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.991+0000 7f32afe00700 1 -- 172.21.15.161:0/4085304751 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f32a80105d0 con 0x7f32b8100e00 2024-03-30T00:32:09.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.992+0000 7f32afe00700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f329c0a1120 0x7f329c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:09.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.992+0000 7f32afe00700 1 -- 172.21.15.161:0/4085304751 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f32a80cc1a0 con 0x7f32b8100e00 2024-03-30T00:32:09.999 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.992+0000 7f32b6a00700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f329c0a1120 0x7f329c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:10.000 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.993+0000 7f32b6a00700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f329c0a1120 0x7f329c0a35e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009bf0 tx=0x7f32a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:10.001 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:09.994+0000 7f32afe00700 1 -- 172.21.15.161:0/4085304751 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f32a8095ec0 con 0x7f32b8100e00 2024-03-30T00:32:10.214 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:10.206+0000 7f32be000700 1 -- 172.21.15.161:0/4085304751 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f32b8108ee0 con 0x7f329c0a1120 2024-03-30T00:32:10.798 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:10 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:32:10.798 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:10 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:32:10.798 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:10.798 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:10 smithi161 ceph-mon[42221]: osd.10 [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760] boot 2024-03-30T00:32:10.798 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:10 smithi161 ceph-mon[42221]: osdmap e71: 11 total, 11 up, 11 in 2024-03-30T00:32:10.798 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:10.798 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:10 smithi161 ceph-mon[42221]: osdmap e72: 11 total, 11 up, 11 in 2024-03-30T00:32:10.799 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:10 smithi161 ceph-mon[42221]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:32:10.799 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:32:10.799 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:32:10.799 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:10 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:32:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:10 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:32:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:10 smithi142 ceph-mon[41779]: osd.10 [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760] boot 2024-03-30T00:32:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:10 smithi142 ceph-mon[41779]: osdmap e71: 11 total, 11 up, 11 in 2024-03-30T00:32:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:10 smithi142 ceph-mon[41779]: osdmap e72: 11 total, 11 up, 11 in 2024-03-30T00:32:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:10 smithi142 ceph-mon[41779]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:32:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:32:10.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:32:10.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:10.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:10 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:32:10.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:10 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:32:10.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:10.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:10 smithi099 ceph-mon[38079]: osd.10 [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760] boot 2024-03-30T00:32:10.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:10 smithi099 ceph-mon[38079]: osdmap e71: 11 total, 11 up, 11 in 2024-03-30T00:32:10.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T00:32:10.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:10 smithi099 ceph-mon[38079]: osdmap e72: 11 total, 11 up, 11 in 2024-03-30T00:32:10.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:10 smithi099 ceph-mon[38079]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:32:10.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T00:32:10.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T00:32:10.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:12.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:11 smithi161 ceph-mon[42221]: osdmap e73: 11 total, 11 up, 11 in 2024-03-30T00:32:12.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:11 smithi161 ceph-mon[42221]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:12.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:11 smithi099 ceph-mon[38079]: osdmap e73: 11 total, 11 up, 11 in 2024-03-30T00:32:12.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:11 smithi099 ceph-mon[38079]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:12.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:11 smithi142 ceph-mon[41779]: osdmap e73: 11 total, 11 up, 11 in 2024-03-30T00:32:12.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:11 smithi142 ceph-mon[41779]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:13.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:12 smithi161 ceph-mon[42221]: osdmap e74: 11 total, 11 up, 11 in 2024-03-30T00:32:13.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:12 smithi161 ceph-mon[42221]: from='client.? 172.21.15.161:0/2673631625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7a3b48-85d3-41d2-8a24-288b8250e803"}]: dispatch 2024-03-30T00:32:13.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:12 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7a3b48-85d3-41d2-8a24-288b8250e803"}]: dispatch 2024-03-30T00:32:13.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:12 smithi161 ceph-mon[42221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d7a3b48-85d3-41d2-8a24-288b8250e803"}]': finished 2024-03-30T00:32:13.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:12 smithi161 ceph-mon[42221]: osdmap e75: 12 total, 11 up, 12 in 2024-03-30T00:32:13.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:13.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:12 smithi099 ceph-mon[38079]: osdmap e74: 11 total, 11 up, 11 in 2024-03-30T00:32:13.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:12 smithi099 ceph-mon[38079]: from='client.? 172.21.15.161:0/2673631625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7a3b48-85d3-41d2-8a24-288b8250e803"}]: dispatch 2024-03-30T00:32:13.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:12 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7a3b48-85d3-41d2-8a24-288b8250e803"}]: dispatch 2024-03-30T00:32:13.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:12 smithi099 ceph-mon[38079]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d7a3b48-85d3-41d2-8a24-288b8250e803"}]': finished 2024-03-30T00:32:13.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:12 smithi099 ceph-mon[38079]: osdmap e75: 12 total, 11 up, 12 in 2024-03-30T00:32:13.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:13.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:12 smithi142 ceph-mon[41779]: osdmap e74: 11 total, 11 up, 11 in 2024-03-30T00:32:13.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:12 smithi142 ceph-mon[41779]: from='client.? 172.21.15.161:0/2673631625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7a3b48-85d3-41d2-8a24-288b8250e803"}]: dispatch 2024-03-30T00:32:13.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:12 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d7a3b48-85d3-41d2-8a24-288b8250e803"}]: dispatch 2024-03-30T00:32:13.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:12 smithi142 ceph-mon[41779]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d7a3b48-85d3-41d2-8a24-288b8250e803"}]': finished 2024-03-30T00:32:13.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:12 smithi142 ceph-mon[41779]: osdmap e75: 12 total, 11 up, 12 in 2024-03-30T00:32:13.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:14.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:13 smithi161 ceph-mon[42221]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:14.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:13 smithi161 ceph-mon[42221]: from='client.? 172.21.15.161:0/1265508214' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:32:14.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:13 smithi099 ceph-mon[38079]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:14.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:13 smithi099 ceph-mon[38079]: from='client.? 172.21.15.161:0/1265508214' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:32:14.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:13 smithi142 ceph-mon[41779]: pgmap v217: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:14.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:13 smithi142 ceph-mon[41779]: from='client.? 172.21.15.161:0/1265508214' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T00:32:16.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:15 smithi161 ceph-mon[42221]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-30T00:32:16.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:15 smithi099 ceph-mon[38079]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-30T00:32:16.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:15 smithi142 ceph-mon[41779]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-30T00:32:18.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:18 smithi161 ceph-mon[42221]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-30T00:32:18.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:18 smithi142 ceph-mon[41779]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-30T00:32:18.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:18 smithi099 ceph-mon[38079]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-30T00:32:20.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:20 smithi142 ceph-mon[41779]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T00:32:20.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-30T00:32:20.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:20.330 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:20 smithi161 ceph-mon[42221]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T00:32:20.330 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-30T00:32:20.331 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:20.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:20 smithi099 ceph-mon[38079]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T00:32:20.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-30T00:32:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:21.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:21 smithi161 ceph-mon[42221]: Deploying daemon osd.11 on smithi161 2024-03-30T00:32:21.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:21 smithi099 ceph-mon[38079]: Deploying daemon osd.11 on smithi161 2024-03-30T00:32:21.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:21 smithi142 ceph-mon[41779]: Deploying daemon osd.11 on smithi161 2024-03-30T00:32:22.130 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:22 smithi161 ceph-mon[42221]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-30T00:32:22.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:22 smithi099 ceph-mon[38079]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-30T00:32:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:22 smithi142 ceph-mon[41779]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-30T00:32:24.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:24 smithi161 ceph-mon[42221]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-30T00:32:24.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:24 smithi099 ceph-mon[38079]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-30T00:32:24.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:24 smithi142 ceph-mon[41779]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-03-30T00:32:25.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:25.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:25.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:25 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:25.971 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:25 smithi161 ceph-mon[42221]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T00:32:25.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:25.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:25.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:25 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:25.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:25 smithi099 ceph-mon[38079]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T00:32:26.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:26.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:26.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:25 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:26.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:25 smithi142 ceph-mon[41779]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T00:32:27.880 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 11 on host 'smithi161' 2024-03-30T00:32:27.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.871+0000 7f32afe00700 1 -- 172.21.15.161:0/4085304751 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f32b8108ee0 con 0x7f329c0a1120 2024-03-30T00:32:27.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.874+0000 7f32be000700 1 -- 172.21.15.161:0/4085304751 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f329c0a1120 msgr2=0x7f329c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:27.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.874+0000 7f32be000700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f329c0a1120 0x7f329c0a35e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009bf0 tx=0x7f32a4009260 comp rx=0 tx=0).stop 2024-03-30T00:32:27.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.874+0000 7f32be000700 1 -- 172.21.15.161:0/4085304751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b8100e00 msgr2=0x7f32b8198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:27.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.874+0000 7f32be000700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b8100e00 0x7f32b8198d50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32a800d950 tx=0x7f32a800dc60 comp rx=0 tx=0).stop 2024-03-30T00:32:27.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.874+0000 7f32be000700 1 -- 172.21.15.161:0/4085304751 shutdown_connections 2024-03-30T00:32:27.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.874+0000 7f32be000700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32b8102920 0x7f32b81a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:27.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.874+0000 7f32be000700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32b8100e00 0x7f32b8198d50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:27.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.874+0000 7f32be000700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f329c0a1120 0x7f329c0a35e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:27.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.874+0000 7f32be000700 1 --2- 172.21.15.161:0/4085304751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b8101f60 0x7f32b8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:27.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.875+0000 7f32be000700 1 -- 172.21.15.161:0/4085304751 >> 172.21.15.161:0/4085304751 conn(0x7f32b80fc3c0 msgr2=0x7f32b81077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:27.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.875+0000 7f32be000700 1 -- 172.21.15.161:0/4085304751 shutdown_connections 2024-03-30T00:32:27.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:27.875+0000 7f32be000700 1 -- 172.21.15.161:0/4085304751 wait complete. 2024-03-30T00:32:28.077 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.077 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.077 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:28.077 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:28.077 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.077 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:27 smithi161 ceph-mon[42221]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:28.077 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.077 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.077 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:27 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:28.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:28.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:28.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:27 smithi099 ceph-mon[38079]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:28.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:27 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:28.259 DEBUG:teuthology.orchestra.run.smithi161:osd.11> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.11.service 2024-03-30T00:32:28.263 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-30T00:32:28.263 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-03-30T00:32:28.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:28.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:28.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:27 smithi142 ceph-mon[41779]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:28.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:28.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:27 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:28.333 INFO:journalctl@ceph.osd.11.smithi161.stdout:-- Logs begin at Sat 2024-03-30 00:10:26 UTC. -- 2024-03-30T00:32:28.629 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:32:28.876 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:32:28 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:32:28.671+0000 7fb35ee60640 -1 osd.11 0 log_to_monitors true 2024-03-30T00:32:29.099 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:28 smithi099 ceph-mon[38079]: from='osd.11 [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T00:32:29.099 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:28 smithi099 ceph-mon[38079]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T00:32:29.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:28 smithi161 ceph-mon[42221]: from='osd.11 [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T00:32:29.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:28 smithi161 ceph-mon[42221]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T00:32:29.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:28 smithi142 ceph-mon[41779]: from='osd.11 [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T00:32:29.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:28 smithi142 ceph-mon[41779]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T00:32:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.722+0000 7f1bcfa00700 1 -- 172.21.15.99:0/1234360896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc81041b0 msgr2=0x7f1bc81065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.722+0000 7f1bcfa00700 1 --2- 172.21.15.99:0/1234360896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc81041b0 0x7f1bc81065a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009990 tx=0x7f1bb4009ca0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.722+0000 7f1bcfa00700 1 -- 172.21.15.99:0/1234360896 shutdown_connections 2024-03-30T00:32:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.722+0000 7f1bcfa00700 1 --2- 172.21.15.99:0/1234360896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bc8106ae0 0x7f1bc8108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.722+0000 7f1bcfa00700 1 --2- 172.21.15.99:0/1234360896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc8101880 0x7f1bc8103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.722+0000 7f1bcfa00700 1 --2- 172.21.15.99:0/1234360896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc81041b0 0x7f1bc81065a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.722+0000 7f1bcfa00700 1 -- 172.21.15.99:0/1234360896 >> 172.21.15.99:0/1234360896 conn(0x7f1bc80fb2b0 msgr2=0x7f1bc80fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.722+0000 7f1bcfa00700 1 -- 172.21.15.99:0/1234360896 shutdown_connections 2024-03-30T00:32:29.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.722+0000 7f1bcfa00700 1 -- 172.21.15.99:0/1234360896 wait complete. 2024-03-30T00:32:29.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.723+0000 7f1bcfa00700 1 Processor -- start 2024-03-30T00:32:29.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.724+0000 7f1bcfa00700 1 -- start start 2024-03-30T00:32:29.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.724+0000 7f1bcfa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc8101880 0x7f1bc8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:29.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.724+0000 7f1bcfa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bc81041b0 0x7f1bc81993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:29.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.724+0000 7f1bcfa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc8106ae0 0x7f1bc81a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:29.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.724+0000 7f1bcfa00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc81a09e0 con 0x7f1bc8106ae0 2024-03-30T00:32:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.724+0000 7f1bcfa00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc81a0b20 con 0x7f1bc81041b0 2024-03-30T00:32:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.724+0000 7f1bcfa00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc81a0cb0 con 0x7f1bc8101880 2024-03-30T00:32:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.724+0000 7f1bcd000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc8101880 0x7f1bc8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.724+0000 7f1bcd000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc8101880 0x7f1bc8198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:52942/0 (socket says 172.21.15.99:52942) 2024-03-30T00:32:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.724+0000 7f1bcd000700 1 -- 172.21.15.99:0/2037007191 learned_addr learned my addr 172.21.15.99:0/2037007191 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.724+0000 7f1bcda00700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc8106ae0 0x7f1bc81a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bc7e00700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bc81041b0 0x7f1bc81993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bcd000700 1 -- 172.21.15.99:0/2037007191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bc81041b0 msgr2=0x7f1bc81993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bcd000700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bc81041b0 0x7f1bc81993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bcd000700 1 -- 172.21.15.99:0/2037007191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc8106ae0 msgr2=0x7f1bc81a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bcd000700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc8106ae0 0x7f1bc81a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bcd000700 1 -- 172.21.15.99:0/2037007191 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40095f0 con 0x7f1bc8101880 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bcd000700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc8101880 0x7f1bc8198ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00d9b0 tx=0x7f1bbc00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bc5600700 1 -- 172.21.15.99:0/2037007191 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc004a30 con 0x7f1bc8101880 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bcfa00700 1 -- 172.21.15.99:0/2037007191 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc81a0f90 con 0x7f1bc8101880 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bc5600700 1 -- 172.21.15.99:0/2037007191 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bbc0054b0 con 0x7f1bc8101880 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bc5600700 1 -- 172.21.15.99:0/2037007191 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc009bc0 con 0x7f1bc8101880 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.725+0000 7f1bcfa00700 1 -- 172.21.15.99:0/2037007191 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc81a13a0 con 0x7f1bc8101880 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.726+0000 7f1bcfa00700 1 -- 172.21.15.99:0/2037007191 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc81086b0 con 0x7f1bc8101880 2024-03-30T00:32:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.728+0000 7f1bc5600700 1 -- 172.21.15.99:0/2037007191 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1bbc005620 con 0x7f1bc8101880 2024-03-30T00:32:29.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.729+0000 7f1bc5600700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bb00a11a0 0x7f1bb00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:29.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.730+0000 7f1bc5600700 1 -- 172.21.15.99:0/2037007191 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f1bbc0cc580 con 0x7f1bc8101880 2024-03-30T00:32:29.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.730+0000 7f1bc7e00700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bb00a11a0 0x7f1bb00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:29.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.730+0000 7f1bc7e00700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bb00a11a0 0x7f1bb00a3660 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4003780 tx=0x7f1bb4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:29.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.734+0000 7f1bc5600700 1 -- 172.21.15.99:0/2037007191 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1bbc0962c0 con 0x7f1bc8101880 2024-03-30T00:32:29.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.940+0000 7f1bcfa00700 1 -- 172.21.15.99:0/2037007191 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1bc804ea80 con 0x7f1bc8101880 2024-03-30T00:32:29.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.941+0000 7f1bc5600700 1 -- 172.21.15.99:0/2037007191 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1bbc017020 con 0x7f1bc8101880 2024-03-30T00:32:29.948 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:29.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.943+0000 7f1bcfa00700 1 -- 172.21.15.99:0/2037007191 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bb00a11a0 msgr2=0x7f1bb00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:29.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.943+0000 7f1bcfa00700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bb00a11a0 0x7f1bb00a3660 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4003780 tx=0x7f1bb4005fb0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.943+0000 7f1bcfa00700 1 -- 172.21.15.99:0/2037007191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc8101880 msgr2=0x7f1bc8198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:29.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.943+0000 7f1bcfa00700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc8101880 0x7f1bc8198ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00d9b0 tx=0x7f1bbc00dcc0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.943+0000 7f1bcfa00700 1 -- 172.21.15.99:0/2037007191 shutdown_connections 2024-03-30T00:32:29.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.943+0000 7f1bcfa00700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bc8106ae0 0x7f1bc81a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.943+0000 7f1bcfa00700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bc8101880 0x7f1bc8198ea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.943+0000 7f1bcfa00700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bb00a11a0 0x7f1bb00a3660 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.943+0000 7f1bcfa00700 1 --2- 172.21.15.99:0/2037007191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bc81041b0 0x7f1bc81993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:29.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.943+0000 7f1bcfa00700 1 -- 172.21.15.99:0/2037007191 >> 172.21.15.99:0/2037007191 conn(0x7f1bc80fb2b0 msgr2=0x7f1bc8101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:29.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.943+0000 7f1bcfa00700 1 -- 172.21.15.99:0/2037007191 shutdown_connections 2024-03-30T00:32:29.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:29.944+0000 7f1bcfa00700 1 -- 172.21.15.99:0/2037007191 wait complete. 2024-03-30T00:32:30.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:29 smithi099 ceph-mon[38079]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-30T00:32:30.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:29 smithi099 ceph-mon[38079]: osdmap e76: 12 total, 11 up, 12 in 2024-03-30T00:32:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:29 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:29 smithi099 ceph-mon[38079]: from='osd.11 [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:32:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:29 smithi099 ceph-mon[38079]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:32:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:29 smithi099 ceph-mon[38079]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:30.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:29 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2037007191' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T00:32:30.311 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1711758729,"num_in_osds":12,"osd_in_since":1711758732,"num_remapped_pgs":0} 2024-03-30T00:32:30.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:29 smithi142 ceph-mon[41779]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-30T00:32:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:29 smithi142 ceph-mon[41779]: osdmap e76: 12 total, 11 up, 12 in 2024-03-30T00:32:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:29 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:29 smithi142 ceph-mon[41779]: from='osd.11 [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:32:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:29 smithi142 ceph-mon[41779]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:32:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:29 smithi142 ceph-mon[41779]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:30.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:29 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2037007191' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T00:32:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:29 smithi161 ceph-mon[42221]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-30T00:32:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:29 smithi161 ceph-mon[42221]: osdmap e76: 12 total, 11 up, 12 in 2024-03-30T00:32:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:29 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:29 smithi161 ceph-mon[42221]: from='osd.11 [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:32:30.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:29 smithi161 ceph-mon[42221]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-03-30T00:32:30.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:29 smithi161 ceph-mon[42221]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:30.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:29 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2037007191' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T00:32:31.135 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:30 smithi161 ceph-mon[42221]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:32:31.135 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:30 smithi161 ceph-mon[42221]: osdmap e77: 12 total, 11 up, 12 in 2024-03-30T00:32:31.135 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:30 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:31.136 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:30 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:31.136 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:32:31.034+0000 7fb352a00700 -1 osd.11 0 waiting for initial osdmap 2024-03-30T00:32:31.137 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:32:31.070+0000 7fb349200700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T00:32:31.312 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-03-30T00:32:31.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:30 smithi142 ceph-mon[41779]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:32:31.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:30 smithi142 ceph-mon[41779]: osdmap e77: 12 total, 11 up, 12 in 2024-03-30T00:32:31.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:30 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:31.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:30 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:31.381 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:30 smithi099 ceph-mon[38079]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-03-30T00:32:31.382 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:30 smithi099 ceph-mon[38079]: osdmap e77: 12 total, 11 up, 12 in 2024-03-30T00:32:31.382 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:30 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:31.382 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:30 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:31.681 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:32:32.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: purged_snaps scrub starts 2024-03-30T00:32:32.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: purged_snaps scrub ok 2024-03-30T00:32:32.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:32.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:32.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='osd.11 ' entity='osd.11' 2024-03-30T00:32:32.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: Detected new or changed devices on smithi161 2024-03-30T00:32:32.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.192 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: Adjusting osd_memory_target on smithi161 to 4317M 2024-03-30T00:32:32.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:32.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:32.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.193 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:31 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:32.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: purged_snaps scrub starts 2024-03-30T00:32:32.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: purged_snaps scrub ok 2024-03-30T00:32:32.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:32.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:32.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='osd.11 ' entity='osd.11' 2024-03-30T00:32:32.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: Detected new or changed devices on smithi161 2024-03-30T00:32:32.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: Adjusting osd_memory_target on smithi161 to 4317M 2024-03-30T00:32:32.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:32.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:32.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.319 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:31 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: purged_snaps scrub starts 2024-03-30T00:32:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: purged_snaps scrub ok 2024-03-30T00:32:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-03-30T00:32:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='osd.11 ' entity='osd.11' 2024-03-30T00:32:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: Detected new or changed devices on smithi161 2024-03-30T00:32:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T00:32:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: Adjusting osd_memory_target on smithi161 to 4317M 2024-03-30T00:32:32.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:32.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:32.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:32.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:31 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.798+0000 7f1514400700 1 -- 172.21.15.99:0/4293604225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c100e00 msgr2=0x7f150c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.798+0000 7f1514400700 1 --2- 172.21.15.99:0/4293604225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c100e00 0x7f150c101220 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1500009990 tx=0x7f1500009ca0 comp rx=0 tx=0).stop 2024-03-30T00:32:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.798+0000 7f1514400700 1 -- 172.21.15.99:0/4293604225 shutdown_connections 2024-03-30T00:32:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.798+0000 7f1514400700 1 --2- 172.21.15.99:0/4293604225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f150c102920 0x7f150c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.798+0000 7f1514400700 1 --2- 172.21.15.99:0/4293604225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c100e00 0x7f150c101220 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.798+0000 7f1514400700 1 --2- 172.21.15.99:0/4293604225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f150c101f60 0x7f150c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.798+0000 7f1514400700 1 -- 172.21.15.99:0/4293604225 >> 172.21.15.99:0/4293604225 conn(0x7f150c0fc3c0 msgr2=0x7f150c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:32.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.798+0000 7f1514400700 1 -- 172.21.15.99:0/4293604225 shutdown_connections 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.798+0000 7f1514400700 1 -- 172.21.15.99:0/4293604225 wait complete. 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.799+0000 7f1514400700 1 Processor -- start 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.799+0000 7f1514400700 1 -- start start 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.799+0000 7f1514400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f150c100e00 0x7f150c074ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.799+0000 7f1514400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f150c101f60 0x7f150c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.799+0000 7f1514400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c102920 0x7f150c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.799+0000 7f1514400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c073dc0 con 0x7f150c102920 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.799+0000 7f1514400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c073f30 con 0x7f150c100e00 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.799+0000 7f1514400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c1a0d40 con 0x7f150c101f60 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1511a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f150c100e00 0x7f150c074ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1511a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f150c100e00 0x7f150c074ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:47012/0 (socket says 172.21.15.99:47012) 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1511a00700 1 -- 172.21.15.99:0/1214754647 learned_addr learned my addr 172.21.15.99:0/1214754647 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:32.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1512400700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c102920 0x7f150c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1511000700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f150c101f60 0x7f150c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1512400700 1 -- 172.21.15.99:0/1214754647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f150c101f60 msgr2=0x7f150c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1512400700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f150c101f60 0x7f150c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1512400700 1 -- 172.21.15.99:0/1214754647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f150c100e00 msgr2=0x7f150c074ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1512400700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f150c100e00 0x7f150c074ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1512400700 1 -- 172.21.15.99:0/1214754647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000095f0 con 0x7f150c102920 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1511a00700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f150c100e00 0x7f150c074ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f1512400700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c102920 0x7f150c0736e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1504007f10 tx=0x7f150400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.800+0000 7f150a000700 1 -- 172.21.15.99:0/1214754647 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150400ddd0 con 0x7f150c102920 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.801+0000 7f150a000700 1 -- 172.21.15.99:0/1214754647 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f150400f040 con 0x7f150c102920 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.801+0000 7f1514400700 1 -- 172.21.15.99:0/1214754647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c1a0fa0 con 0x7f150c102920 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.801+0000 7f150a000700 1 -- 172.21.15.99:0/1214754647 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15040059c0 con 0x7f150c102920 2024-03-30T00:32:32.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.801+0000 7f1514400700 1 -- 172.21.15.99:0/1214754647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c1a1460 con 0x7f150c102920 2024-03-30T00:32:32.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.802+0000 7f1514400700 1 -- 172.21.15.99:0/1214754647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c108370 con 0x7f150c102920 2024-03-30T00:32:32.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.804+0000 7f150a000700 1 -- 172.21.15.99:0/1214754647 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1504005000 con 0x7f150c102920 2024-03-30T00:32:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.805+0000 7f150a000700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14f40a11a0 0x7f14f40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.805+0000 7f150a000700 1 -- 172.21.15.99:0/1214754647 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f15040cc430 con 0x7f150c102920 2024-03-30T00:32:32.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.805+0000 7f1511a00700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14f40a11a0 0x7f14f40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:32.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.806+0000 7f1511a00700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14f40a11a0 0x7f14f40a3660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f150000c010 tx=0x7f1500003740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:32.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:32.808+0000 7f150a000700 1 -- 172.21.15.99:0/1214754647 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1504095fc0 con 0x7f150c102920 2024-03-30T00:32:33.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.013+0000 7f1514400700 1 -- 172.21.15.99:0/1214754647 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f150c04ea80 con 0x7f150c102920 2024-03-30T00:32:33.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.013+0000 7f150a000700 1 -- 172.21.15.99:0/1214754647 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1504023090 con 0x7f150c102920 2024-03-30T00:32:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.015+0000 7f1514400700 1 -- 172.21.15.99:0/1214754647 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14f40a11a0 msgr2=0x7f14f40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.015+0000 7f1514400700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14f40a11a0 0x7f14f40a3660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f150000c010 tx=0x7f1500003740 comp rx=0 tx=0).stop 2024-03-30T00:32:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.015+0000 7f1514400700 1 -- 172.21.15.99:0/1214754647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c102920 msgr2=0x7f150c0736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.015+0000 7f1514400700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c102920 0x7f150c0736e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1504007f10 tx=0x7f150400d3b0 comp rx=0 tx=0).stop 2024-03-30T00:32:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.016+0000 7f1514400700 1 -- 172.21.15.99:0/1214754647 shutdown_connections 2024-03-30T00:32:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.016+0000 7f1514400700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c102920 0x7f150c0736e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.016+0000 7f1514400700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14f40a11a0 0x7f14f40a3660 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:33.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.016+0000 7f1514400700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f150c100e00 0x7f150c074ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:33.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.016+0000 7f1514400700 1 --2- 172.21.15.99:0/1214754647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f150c101f60 0x7f150c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:33.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.016+0000 7f1514400700 1 -- 172.21.15.99:0/1214754647 >> 172.21.15.99:0/1214754647 conn(0x7f150c0fc3c0 msgr2=0x7f150c1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:33.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.016+0000 7f1514400700 1 -- 172.21.15.99:0/1214754647 shutdown_connections 2024-03-30T00:32:33.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:33.016+0000 7f1514400700 1 -- 172.21.15.99:0/1214754647 wait complete. 2024-03-30T00:32:33.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:33 smithi142 ceph-mon[41779]: osd.11 [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097] boot 2024-03-30T00:32:33.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:33 smithi142 ceph-mon[41779]: osdmap e78: 12 total, 12 up, 12 in 2024-03-30T00:32:33.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:33 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:33.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:33 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1214754647' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T00:32:33.356 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:33 smithi099 ceph-mon[38079]: osd.11 [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097] boot 2024-03-30T00:32:33.357 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:33 smithi099 ceph-mon[38079]: osdmap e78: 12 total, 12 up, 12 in 2024-03-30T00:32:33.357 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:33 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:33.357 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:33 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1214754647' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T00:32:33.387 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1711758752,"num_in_osds":12,"osd_in_since":1711758732,"num_remapped_pgs":0} 2024-03-30T00:32:33.387 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-03-30T00:32:33.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:33 smithi161 ceph-mon[42221]: osd.11 [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097] boot 2024-03-30T00:32:33.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:33 smithi161 ceph-mon[42221]: osdmap e78: 12 total, 12 up, 12 in 2024-03-30T00:32:33.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:33 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T00:32:33.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:33 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1214754647' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T00:32:33.738 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:32:34.215 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:34 smithi099 ceph-mon[38079]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:34.216 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:34 smithi099 ceph-mon[38079]: osdmap e79: 12 total, 12 up, 12 in 2024-03-30T00:32:34.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:34 smithi142 ceph-mon[41779]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:34.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:34 smithi142 ceph-mon[41779]: osdmap e79: 12 total, 12 up, 12 in 2024-03-30T00:32:34.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:34 smithi161 ceph-mon[42221]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:34.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:34 smithi161 ceph-mon[42221]: osdmap e79: 12 total, 12 up, 12 in 2024-03-30T00:32:34.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.848+0000 7f5660a00700 1 -- 172.21.15.99:0/3883606352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c106ef0 msgr2=0x7f565c1072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:34.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.848+0000 7f5660a00700 1 --2- 172.21.15.99:0/3883606352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c106ef0 0x7f565c1072d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f564c0099c0 tx=0x7f564c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:32:34.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.848+0000 7f5660a00700 1 -- 172.21.15.99:0/3883606352 shutdown_connections 2024-03-30T00:32:34.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.848+0000 7f5660a00700 1 --2- 172.21.15.99:0/3883606352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f565c101820 0x7f565c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:34.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.848+0000 7f5660a00700 1 --2- 172.21.15.99:0/3883606352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c106ef0 0x7f565c1072d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:34.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.848+0000 7f5660a00700 1 --2- 172.21.15.99:0/3883606352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565c100e60 0x7f565c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:34.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.848+0000 7f5660a00700 1 -- 172.21.15.99:0/3883606352 >> 172.21.15.99:0/3883606352 conn(0x7f565c077fa0 msgr2=0x7f565c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:34.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.848+0000 7f5660a00700 1 -- 172.21.15.99:0/3883606352 shutdown_connections 2024-03-30T00:32:34.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.848+0000 7f5660a00700 1 -- 172.21.15.99:0/3883606352 wait complete. 2024-03-30T00:32:34.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.849+0000 7f5660a00700 1 Processor -- start 2024-03-30T00:32:34.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.849+0000 7f5660a00700 1 -- start start 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5660a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c100e60 0x7f565c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5660a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565c101820 0x7f565c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5660a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f565c106ef0 0x7f565c19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5660a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c19e350 con 0x7f565c100e60 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5660a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c19e4c0 con 0x7f565c106ef0 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5660a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c19e680 con 0x7f565c101820 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5659e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c100e60 0x7f565c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5659e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c100e60 0x7f565c199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60312/0 (socket says 172.21.15.99:60312) 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5659e00700 1 -- 172.21.15.99:0/1320319464 learned_addr learned my addr 172.21.15.99:0/1320319464 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f565a800700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f565c106ef0 0x7f565c19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5653e00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565c101820 0x7f565c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:34.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5659e00700 1 -- 172.21.15.99:0/1320319464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565c101820 msgr2=0x7f565c199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:34.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.850+0000 7f5659e00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565c101820 0x7f565c199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:34.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.851+0000 7f5659e00700 1 -- 172.21.15.99:0/1320319464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f565c106ef0 msgr2=0x7f565c19dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:34.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.851+0000 7f5659e00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f565c106ef0 0x7f565c19dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:34.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.851+0000 7f5659e00700 1 -- 172.21.15.99:0/1320319464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c0095f0 con 0x7f565c100e60 2024-03-30T00:32:34.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.851+0000 7f5659e00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c100e60 0x7f565c199450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f564c009f50 tx=0x7f564c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:34.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.851+0000 7f5652a00700 1 -- 172.21.15.99:0/1320319464 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c01d070 con 0x7f565c100e60 2024-03-30T00:32:34.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.851+0000 7f5652a00700 1 -- 172.21.15.99:0/1320319464 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f564c00fe10 con 0x7f565c100e60 2024-03-30T00:32:34.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.851+0000 7f5660a00700 1 -- 172.21.15.99:0/1320319464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565c19e900 con 0x7f565c100e60 2024-03-30T00:32:34.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.851+0000 7f5652a00700 1 -- 172.21.15.99:0/1320319464 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c0173f0 con 0x7f565c100e60 2024-03-30T00:32:34.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.851+0000 7f5660a00700 1 -- 172.21.15.99:0/1320319464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565c1a61c0 con 0x7f565c100e60 2024-03-30T00:32:34.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.852+0000 7f5660a00700 1 -- 172.21.15.99:0/1320319464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565c104180 con 0x7f565c100e60 2024-03-30T00:32:34.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.854+0000 7f5652a00700 1 -- 172.21.15.99:0/1320319464 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f564c022650 con 0x7f565c100e60 2024-03-30T00:32:34.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.856+0000 7f5652a00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f56400a11a0 0x7f56400a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:34.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.856+0000 7f5652a00700 1 -- 172.21.15.99:0/1320319464 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f564c0cd560 con 0x7f565c100e60 2024-03-30T00:32:34.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.856+0000 7f5653e00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f56400a11a0 0x7f56400a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:34.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.857+0000 7f5653e00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f56400a11a0 0x7f56400a3660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5644006fd0 tx=0x7f5644008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:34.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:34.860+0000 7f5652a00700 1 -- 172.21.15.99:0/1320319464 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f564c097180 con 0x7f565c100e60 2024-03-30T00:32:35.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.066+0000 7f5660a00700 1 -- 172.21.15.99:0/1320319464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f565c1024f0 con 0x7f565c100e60 2024-03-30T00:32:35.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.068+0000 7f5652a00700 1 -- 172.21.15.99:0/1320319464 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18423 (secure 0 0 0) 0x7f564c09a7a0 con 0x7f565c100e60 2024-03-30T00:32:35.075 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:35.075 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":79,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:32:33.039895+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:32:35.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 -- 172.21.15.99:0/1320319464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f56400a11a0 msgr2=0x7f56400a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:35.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f56400a11a0 0x7f56400a3660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5644006fd0 tx=0x7f5644008040 comp rx=0 tx=0).stop 2024-03-30T00:32:35.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 -- 172.21.15.99:0/1320319464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c100e60 msgr2=0x7f565c199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:35.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c100e60 0x7f565c199450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f564c009f50 tx=0x7f564c00f510 comp rx=0 tx=0).stop 2024-03-30T00:32:35.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 -- 172.21.15.99:0/1320319464 shutdown_connections 2024-03-30T00:32:35.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f56400a11a0 0x7f56400a3660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:35.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f565c106ef0 0x7f565c19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:35.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c100e60 0x7f565c199450 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:35.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 --2- 172.21.15.99:0/1320319464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f565c101820 0x7f565c199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:35.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 -- 172.21.15.99:0/1320319464 >> 172.21.15.99:0/1320319464 conn(0x7f565c077fa0 msgr2=0x7f565c10c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:35.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 -- 172.21.15.99:0/1320319464 shutdown_connections 2024-03-30T00:32:35.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:35.070+0000 7f5660a00700 1 -- 172.21.15.99:0/1320319464 wait complete. 2024-03-30T00:32:35.458 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:32:35.459 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-03-30T00:32:35.813 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:32:36.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:35 smithi161 ceph-mon[42221]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:36.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:35 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1320319464' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:32:36.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:35 smithi099 ceph-mon[38079]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:36.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:35 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1320319464' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:32:36.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:35 smithi142 ceph-mon[41779]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:36.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:35 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1320319464' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:32:36.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.907+0000 7f825d800700 1 -- 172.21.15.99:0/1320034772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f825810d3f0 msgr2=0x7f825810f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:36.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.907+0000 7f825d800700 1 --2- 172.21.15.99:0/1320034772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f825810d3f0 0x7f825810f860 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f825000b0a0 tx=0x7f825000b3b0 comp rx=0 tx=0).stop 2024-03-30T00:32:36.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.908+0000 7f825d800700 1 -- 172.21.15.99:0/1320034772 shutdown_connections 2024-03-30T00:32:36.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.908+0000 7f825d800700 1 --2- 172.21.15.99:0/1320034772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f825810d3f0 0x7f825810f860 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:36.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.908+0000 7f825d800700 1 --2- 172.21.15.99:0/1320034772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82580731e0 0x7f82580735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:36.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.908+0000 7f825d800700 1 --2- 172.21.15.99:0/1320034772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8258073b00 0x7f825810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:36.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.908+0000 7f825d800700 1 -- 172.21.15.99:0/1320034772 >> 172.21.15.99:0/1320034772 conn(0x7f82580fc6e0 msgr2=0x7f82580feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:36.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.908+0000 7f825d800700 1 -- 172.21.15.99:0/1320034772 shutdown_connections 2024-03-30T00:32:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.908+0000 7f825d800700 1 -- 172.21.15.99:0/1320034772 wait complete. 2024-03-30T00:32:36.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.909+0000 7f825d800700 1 Processor -- start 2024-03-30T00:32:36.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.909+0000 7f825d800700 1 -- start start 2024-03-30T00:32:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.909+0000 7f825d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82580731e0 0x7f82581993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.909+0000 7f825d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8258073b00 0x7f8258199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.909+0000 7f825d800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f825810d3f0 0x7f825819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.909+0000 7f825d800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825819e2a0 con 0x7f82580731e0 2024-03-30T00:32:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.909+0000 7f825d800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825819e410 con 0x7f8258073b00 2024-03-30T00:32:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.909+0000 7f825d800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825819e5d0 con 0x7f825810d3f0 2024-03-30T00:32:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.909+0000 7f8257e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82580731e0 0x7f82581993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.909+0000 7f8257e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82580731e0 0x7f82581993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48942/0 (socket says 172.21.15.99:48942) 2024-03-30T00:32:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.909+0000 7f8257e00700 1 -- 172.21.15.99:0/4019289235 learned_addr learned my addr 172.21.15.99:0/4019289235 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f8257400700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8258073b00 0x7f8258199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f825ce00700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f825810d3f0 0x7f825819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f8257e00700 1 -- 172.21.15.99:0/4019289235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f825810d3f0 msgr2=0x7f825819dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f8257e00700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f825810d3f0 0x7f825819dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f8257e00700 1 -- 172.21.15.99:0/4019289235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8258073b00 msgr2=0x7f8258199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f8257e00700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8258073b00 0x7f8258199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f8257e00700 1 -- 172.21.15.99:0/4019289235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8250009d00 con 0x7f82580731e0 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f8257e00700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82580731e0 0x7f82581993f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f824800b530 tx=0x7f824800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f8254c00700 1 -- 172.21.15.99:0/4019289235 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248011830 con 0x7f82580731e0 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f8254c00700 1 -- 172.21.15.99:0/4019289235 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8248011e20 con 0x7f82580731e0 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f8254c00700 1 -- 172.21.15.99:0/4019289235 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248010dd0 con 0x7f82580731e0 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f825d800700 1 -- 172.21.15.99:0/4019289235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825819e8b0 con 0x7f82580731e0 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.910+0000 7f825d800700 1 -- 172.21.15.99:0/4019289235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82581a6110 con 0x7f82580731e0 2024-03-30T00:32:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.911+0000 7f825d800700 1 -- 172.21.15.99:0/4019289235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8258108600 con 0x7f82580731e0 2024-03-30T00:32:36.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.913+0000 7f8254c00700 1 -- 172.21.15.99:0/4019289235 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f8248011990 con 0x7f82580731e0 2024-03-30T00:32:36.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.915+0000 7f8254c00700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f823c0a11a0 0x7f823c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:36.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.915+0000 7f8254c00700 1 -- 172.21.15.99:0/4019289235 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f82480cc360 con 0x7f82580731e0 2024-03-30T00:32:36.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.915+0000 7f8257400700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f823c0a11a0 0x7f823c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:36.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.915+0000 7f8257400700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f823c0a11a0 0x7f823c0a3660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8244006fd0 tx=0x7f8244008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:36.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:36.917+0000 7f8254c00700 1 -- 172.21.15.99:0/4019289235 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8248095ef0 con 0x7f82580731e0 2024-03-30T00:32:37.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.121+0000 7f825d800700 1 -- 172.21.15.99:0/4019289235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f825819a750 con 0x7f82580731e0 2024-03-30T00:32:37.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.122+0000 7f8254c00700 1 -- 172.21.15.99:0/4019289235 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8248099510 con 0x7f82580731e0 2024-03-30T00:32:37.129 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:32:37.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.124+0000 7f825d800700 1 -- 172.21.15.99:0/4019289235 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f823c0a11a0 msgr2=0x7f823c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:37.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.124+0000 7f825d800700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f823c0a11a0 0x7f823c0a3660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8244006fd0 tx=0x7f8244008040 comp rx=0 tx=0).stop 2024-03-30T00:32:37.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.124+0000 7f825d800700 1 -- 172.21.15.99:0/4019289235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82580731e0 msgr2=0x7f82581993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:37.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.124+0000 7f825d800700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82580731e0 0x7f82581993f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f824800b530 tx=0x7f824800b8f0 comp rx=0 tx=0).stop 2024-03-30T00:32:37.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.125+0000 7f825d800700 1 -- 172.21.15.99:0/4019289235 shutdown_connections 2024-03-30T00:32:37.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.125+0000 7f825d800700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f825810d3f0 0x7f825819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:37.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.125+0000 7f825d800700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82580731e0 0x7f82581993f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:37.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.125+0000 7f825d800700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f823c0a11a0 0x7f823c0a3660 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:37.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.125+0000 7f825d800700 1 --2- 172.21.15.99:0/4019289235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8258073b00 0x7f8258199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:37.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.125+0000 7f825d800700 1 -- 172.21.15.99:0/4019289235 >> 172.21.15.99:0/4019289235 conn(0x7f82580fc6e0 msgr2=0x7f82580fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:37.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.125+0000 7f825d800700 1 -- 172.21.15.99:0/4019289235 shutdown_connections 2024-03-30T00:32:37.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:37.125+0000 7f825d800700 1 -- 172.21.15.99:0/4019289235 wait complete. 2024-03-30T00:32:37.492 INFO:tasks.cephadm:Adding mds.a on smithi099 2024-03-30T00:32:37.492 INFO:tasks.cephadm:Adding mds.d on smithi099 2024-03-30T00:32:37.492 INFO:tasks.cephadm:Adding mds.g on smithi099 2024-03-30T00:32:37.492 INFO:tasks.cephadm:Adding mds.j on smithi099 2024-03-30T00:32:37.493 INFO:tasks.cephadm:Adding mds.b on smithi142 2024-03-30T00:32:37.493 INFO:tasks.cephadm:Adding mds.e on smithi142 2024-03-30T00:32:37.493 INFO:tasks.cephadm:Adding mds.h on smithi142 2024-03-30T00:32:37.493 INFO:tasks.cephadm:Adding mds.k on smithi142 2024-03-30T00:32:37.493 INFO:tasks.cephadm:Adding mds.c on smithi161 2024-03-30T00:32:37.493 INFO:tasks.cephadm:Adding mds.f on smithi161 2024-03-30T00:32:37.493 INFO:tasks.cephadm:Adding mds.i on smithi161 2024-03-30T00:32:37.493 INFO:tasks.cephadm:Adding mds.l on smithi161 2024-03-30T00:32:37.494 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph orch apply mds all '12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l' 2024-03-30T00:32:37.861 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:32:38.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:38 smithi142 ceph-mon[41779]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:38.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:38 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4019289235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:32:38.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:38 smithi161 ceph-mon[42221]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:38.429 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:38 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4019289235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:32:38.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:38 smithi099 ceph-mon[38079]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:38.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:38 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4019289235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:32:39.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.125+0000 7f064de00700 1 -- 172.21.15.161:0/4231386493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0648100e00 msgr2=0x7f0648101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:39.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.125+0000 7f064de00700 1 --2- 172.21.15.161:0/4231386493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0648100e00 0x7f0648101220 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0638009990 tx=0x7f0638009ca0 comp rx=0 tx=0).stop 2024-03-30T00:32:39.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.125+0000 7f064de00700 1 -- 172.21.15.161:0/4231386493 shutdown_connections 2024-03-30T00:32:39.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.125+0000 7f064de00700 1 --2- 172.21.15.161:0/4231386493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0648102920 0x7f0648109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:39.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.125+0000 7f064de00700 1 --2- 172.21.15.161:0/4231386493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0648100e00 0x7f0648101220 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:39.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.125+0000 7f064de00700 1 --2- 172.21.15.161:0/4231386493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0648101f60 0x7f06481023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:39.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.125+0000 7f064de00700 1 -- 172.21.15.161:0/4231386493 >> 172.21.15.161:0/4231386493 conn(0x7f06480fc3c0 msgr2=0x7f06480fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:39.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.125+0000 7f064de00700 1 -- 172.21.15.161:0/4231386493 shutdown_connections 2024-03-30T00:32:39.133 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.125+0000 7f064de00700 1 -- 172.21.15.161:0/4231386493 wait complete. 2024-03-30T00:32:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.126+0000 7f064de00700 1 Processor -- start 2024-03-30T00:32:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f064de00700 1 -- start start 2024-03-30T00:32:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f064de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0648100e00 0x7f0648198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f064de00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0648101f60 0x7f0648199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f064de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0648102920 0x7f06481a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f064de00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481a08e0 con 0x7f0648100e00 2024-03-30T00:32:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f064de00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481a0a20 con 0x7f0648102920 2024-03-30T00:32:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f064de00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481a0be0 con 0x7f0648101f60 2024-03-30T00:32:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f0646000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0648101f60 0x7f0648199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:39.135 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f0646a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0648100e00 0x7f0648198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f0646a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0648100e00 0x7f0648198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.161:57434/0 (socket says 172.21.15.161:57434) 2024-03-30T00:32:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f0646000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0648101f60 0x7f0648199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41160/0 (socket says 172.21.15.161:41160) 2024-03-30T00:32:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f0646a00700 1 -- 172.21.15.161:0/1758624803 learned_addr learned my addr 172.21.15.161:0/1758624803 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-30T00:32:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.127+0000 7f0647400700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0648102920 0x7f06481a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.128+0000 7f0646a00700 1 -- 172.21.15.161:0/1758624803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0648101f60 msgr2=0x7f0648199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.128+0000 7f0646a00700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0648101f60 0x7f0648199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.128+0000 7f0646a00700 1 -- 172.21.15.161:0/1758624803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0648102920 msgr2=0x7f06481a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.128+0000 7f0646a00700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0648102920 0x7f06481a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.128+0000 7f0646a00700 1 -- 172.21.15.161:0/1758624803 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06380095f0 con 0x7f0648100e00 2024-03-30T00:32:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.128+0000 7f0646000700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0648101f60 0x7f0648199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:32:39.136 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.128+0000 7f0646a00700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0648100e00 0x7f0648198de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0638000c00 tx=0x7f0638004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:39.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.128+0000 7f064ca00700 1 -- 172.21.15.161:0/1758624803 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063801d070 con 0x7f0648100e00 2024-03-30T00:32:39.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.128+0000 7f064de00700 1 -- 172.21.15.161:0/1758624803 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06481a0e60 con 0x7f0648100e00 2024-03-30T00:32:39.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.128+0000 7f064ca00700 1 -- 172.21.15.161:0/1758624803 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0638003ec0 con 0x7f0648100e00 2024-03-30T00:32:39.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.128+0000 7f064ca00700 1 -- 172.21.15.161:0/1758624803 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638022bc0 con 0x7f0648100e00 2024-03-30T00:32:39.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.129+0000 7f064de00700 1 -- 172.21.15.161:0/1758624803 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06481a1220 con 0x7f0648100e00 2024-03-30T00:32:39.137 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.129+0000 7f064de00700 1 -- 172.21.15.161:0/1758624803 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064804ea80 con 0x7f0648100e00 2024-03-30T00:32:39.138 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.132+0000 7f064ca00700 1 -- 172.21.15.161:0/1758624803 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f0638003a30 con 0x7f0648100e00 2024-03-30T00:32:39.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.133+0000 7f064ca00700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f062c0a1120 0x7f062c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:39.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.133+0000 7f064ca00700 1 -- 172.21.15.161:0/1758624803 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06380cddc0 con 0x7f0648100e00 2024-03-30T00:32:39.140 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.133+0000 7f0646000700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f062c0a1120 0x7f062c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:39.141 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.134+0000 7f0646000700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f062c0a1120 0x7f062c0a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0634005950 tx=0x7f063400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:39.142 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.135+0000 7f064ca00700 1 -- 172.21.15.161:0/1758624803 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f06380979e0 con 0x7f0648100e00 2024-03-30T00:32:39.353 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.346+0000 7f064de00700 1 -- 172.21.15.161:0/1758624803 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f0648074700 con 0x7f062c0a1120 2024-03-30T00:32:39.362 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.355+0000 7f064ca00700 1 -- 172.21.15.161:0/1758624803 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f0648074700 con 0x7f062c0a1120 2024-03-30T00:32:39.362 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mds.all update... 2024-03-30T00:32:39.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 -- 172.21.15.161:0/1758624803 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f062c0a1120 msgr2=0x7f062c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:39.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f062c0a1120 0x7f062c0a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0634005950 tx=0x7f063400b410 comp rx=0 tx=0).stop 2024-03-30T00:32:39.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 -- 172.21.15.161:0/1758624803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0648100e00 msgr2=0x7f0648198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:39.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0648100e00 0x7f0648198de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0638000c00 tx=0x7f0638004730 comp rx=0 tx=0).stop 2024-03-30T00:32:39.365 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 -- 172.21.15.161:0/1758624803 shutdown_connections 2024-03-30T00:32:39.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f062c0a1120 0x7f062c0a35e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:39.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0648102920 0x7f06481a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:39.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0648100e00 0x7f0648198de0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:39.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 --2- 172.21.15.161:0/1758624803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0648101f60 0x7f0648199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:39.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 -- 172.21.15.161:0/1758624803 >> 172.21.15.161:0/1758624803 conn(0x7f06480fc3c0 msgr2=0x7f0648072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:39.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 -- 172.21.15.161:0/1758624803 shutdown_connections 2024-03-30T00:32:39.366 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-30T00:32:39.358+0000 7f064de00700 1 -- 172.21.15.161:0/1758624803 wait complete. 2024-03-30T00:32:39.817 DEBUG:teuthology.orchestra.run.smithi099:mds.a> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.a.service 2024-03-30T00:32:39.821 DEBUG:teuthology.orchestra.run.smithi099:mds.d> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.d.service 2024-03-30T00:32:39.865 DEBUG:teuthology.orchestra.run.smithi099:mds.g> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.g.service 2024-03-30T00:32:39.898 INFO:journalctl@ceph.mds.a.smithi099.stdout:-- Logs begin at Sat 2024-03-30 00:10:25 UTC. -- 2024-03-30T00:32:39.901 DEBUG:teuthology.orchestra.run.smithi099:mds.j> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.j.service 2024-03-30T00:32:39.936 INFO:journalctl@ceph.mds.d.smithi099.stdout:-- Logs begin at Sat 2024-03-30 00:10:25 UTC. -- 2024-03-30T00:32:39.939 DEBUG:teuthology.orchestra.run.smithi142:mds.b> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.b.service 2024-03-30T00:32:39.944 DEBUG:teuthology.orchestra.run.smithi142:mds.e> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.e.service 2024-03-30T00:32:39.968 INFO:journalctl@ceph.mds.g.smithi099.stdout:-- Logs begin at Sat 2024-03-30 00:10:25 UTC. -- 2024-03-30T00:32:39.989 DEBUG:teuthology.orchestra.run.smithi142:mds.h> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.h.service 2024-03-30T00:32:40.002 INFO:journalctl@ceph.mds.j.smithi099.stdout:-- Logs begin at Sat 2024-03-30 00:10:25 UTC. -- 2024-03-30T00:32:40.022 INFO:journalctl@ceph.mds.b.smithi142.stdout:-- Logs begin at Sat 2024-03-30 00:10:40 UTC. -- 2024-03-30T00:32:40.025 DEBUG:teuthology.orchestra.run.smithi142:mds.k> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.k.service 2024-03-30T00:32:40.059 INFO:journalctl@ceph.mds.e.smithi142.stdout:-- Logs begin at Sat 2024-03-30 00:10:40 UTC. -- 2024-03-30T00:32:40.062 DEBUG:teuthology.orchestra.run.smithi161:mds.c> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.c.service 2024-03-30T00:32:40.067 DEBUG:teuthology.orchestra.run.smithi161:mds.f> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.f.service 2024-03-30T00:32:40.091 INFO:journalctl@ceph.mds.h.smithi142.stdout:-- Logs begin at Sat 2024-03-30 00:10:40 UTC. -- 2024-03-30T00:32:40.111 DEBUG:teuthology.orchestra.run.smithi161:mds.i> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.i.service 2024-03-30T00:32:40.125 INFO:journalctl@ceph.mds.k.smithi142.stdout:-- Logs begin at Sat 2024-03-30 00:10:40 UTC. -- 2024-03-30T00:32:40.155 DEBUG:teuthology.orchestra.run.smithi161:mds.l> sudo journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.l.service 2024-03-30T00:32:40.156 INFO:journalctl@ceph.mds.c.smithi161.stdout:-- Logs begin at Sat 2024-03-30 00:10:26 UTC. -- 2024-03-30T00:32:40.183 INFO:journalctl@ceph.mds.f.smithi161.stdout:-- Logs begin at Sat 2024-03-30 00:10:26 UTC. -- 2024-03-30T00:32:40.186 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-30T00:32:40.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:32:40.223 INFO:journalctl@ceph.mds.i.smithi161.stdout:-- Logs begin at Sat 2024-03-30 00:10:26 UTC. -- 2024-03-30T00:32:40.253 INFO:journalctl@ceph.mds.l.smithi161.stdout:-- Logs begin at Sat 2024-03-30 00:10:26 UTC. -- 2024-03-30T00:32:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.376+0000 7f1795e00700 1 -- 172.21.15.99:0/218236315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1790100c80 msgr2=0x7f17901010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.376+0000 7f1795e00700 1 --2- 172.21.15.99:0/218236315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1790100c80 0x7f17901010a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f17800099c0 tx=0x7f1780009cd0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.376+0000 7f1795e00700 1 -- 172.21.15.99:0/218236315 shutdown_connections 2024-03-30T00:32:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.376+0000 7f1795e00700 1 --2- 172.21.15.99:0/218236315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1790102810 0x7f17901090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.376+0000 7f1795e00700 1 --2- 172.21.15.99:0/218236315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1790100c80 0x7f17901010a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.376+0000 7f1795e00700 1 --2- 172.21.15.99:0/218236315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1790101e70 0x7f17901022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.376+0000 7f1795e00700 1 -- 172.21.15.99:0/218236315 >> 172.21.15.99:0/218236315 conn(0x7f17900fc1f0 msgr2=0x7f17900fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.376+0000 7f1795e00700 1 -- 172.21.15.99:0/218236315 shutdown_connections 2024-03-30T00:32:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.376+0000 7f1795e00700 1 -- 172.21.15.99:0/218236315 wait complete. 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.377+0000 7f1795e00700 1 Processor -- start 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.377+0000 7f1795e00700 1 -- start start 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f1795e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1790100c80 0x7f1790198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f1795e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1790101e70 0x7f1790199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f1795e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1790102810 0x7f17901a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f1795e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901a08e0 con 0x7f1790101e70 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f1795e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901a0a50 con 0x7f1790100c80 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f1795e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901a0c10 con 0x7f1790102810 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f178e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1790101e70 0x7f1790199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f178e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1790101e70 0x7f1790199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48970/0 (socket says 172.21.15.99:48970) 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f178e000700 1 -- 172.21.15.99:0/1286388067 learned_addr learned my addr 172.21.15.99:0/1286388067 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:40.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f178f400700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1790102810 0x7f17901a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f178ea00700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1790100c80 0x7f1790198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f178e000700 1 -- 172.21.15.99:0/1286388067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1790102810 msgr2=0x7f17901a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f178e000700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1790102810 0x7f17901a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f178e000700 1 -- 172.21.15.99:0/1286388067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1790100c80 msgr2=0x7f1790198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f178e000700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1790100c80 0x7f1790198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.378+0000 7f178e000700 1 -- 172.21.15.99:0/1286388067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17800095f0 con 0x7f1790101e70 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.379+0000 7f178e000700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1790101e70 0x7f1790199330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f177c00d980 tx=0x7f177c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.379+0000 7f1794a00700 1 -- 172.21.15.99:0/1286388067 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c004220 con 0x7f1790101e70 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.379+0000 7f1794a00700 1 -- 172.21.15.99:0/1286388067 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f177c007640 con 0x7f1790101e70 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.379+0000 7f1795e00700 1 -- 172.21.15.99:0/1286388067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17901a0ef0 con 0x7f1790101e70 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.379+0000 7f1794a00700 1 -- 172.21.15.99:0/1286388067 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c005660 con 0x7f1790101e70 2024-03-30T00:32:40.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.379+0000 7f1795e00700 1 -- 172.21.15.99:0/1286388067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901a1310 con 0x7f1790101e70 2024-03-30T00:32:40.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.380+0000 7f1795e00700 1 -- 172.21.15.99:0/1286388067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1790067a10 con 0x7f1790101e70 2024-03-30T00:32:40.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.382+0000 7f1794a00700 1 -- 172.21.15.99:0/1286388067 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f177c004750 con 0x7f1790101e70 2024-03-30T00:32:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.383+0000 7f1794a00700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f17740a1120 0x7f17740a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.383+0000 7f1794a00700 1 -- 172.21.15.99:0/1286388067 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f177c0cc280 con 0x7f1790101e70 2024-03-30T00:32:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.383+0000 7f178ea00700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f17740a1120 0x7f17740a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.384+0000 7f178ea00700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f17740a1120 0x7f17740a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1780000c00 tx=0x7f1780003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.387+0000 7f1794a00700 1 -- 172.21.15.99:0/1286388067 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f177c095e10 con 0x7f1790101e70 2024-03-30T00:32:40.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.627+0000 7f1795e00700 1 -- 172.21.15.99:0/1286388067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f179010c390 con 0x7f1790101e70 2024-03-30T00:32:40.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.628+0000 7f1794a00700 1 -- 172.21.15.99:0/1286388067 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18423 (secure 0 0 0) 0x7f177c099430 con 0x7f1790101e70 2024-03-30T00:32:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:40.635 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":79,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:32:33.039895+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:32:40.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 -- 172.21.15.99:0/1286388067 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f17740a1120 msgr2=0x7f17740a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:40.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f17740a1120 0x7f17740a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1780000c00 tx=0x7f1780003680 comp rx=0 tx=0).stop 2024-03-30T00:32:40.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 -- 172.21.15.99:0/1286388067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1790101e70 msgr2=0x7f1790199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:40.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1790101e70 0x7f1790199330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f177c00d980 tx=0x7f177c00dc90 comp rx=0 tx=0).stop 2024-03-30T00:32:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 -- 172.21.15.99:0/1286388067 shutdown_connections 2024-03-30T00:32:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1790102810 0x7f17901a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f17740a1120 0x7f17740a35e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1790100c80 0x7f1790198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 --2- 172.21.15.99:0/1286388067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1790101e70 0x7f1790199330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 -- 172.21.15.99:0/1286388067 >> 172.21.15.99:0/1286388067 conn(0x7f17900fc1f0 msgr2=0x7f1790107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 -- 172.21.15.99:0/1286388067 shutdown_connections 2024-03-30T00:32:40.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.630+0000 7f1795e00700 1 -- 172.21.15.99:0/1286388067 wait complete. 2024-03-30T00:32:40.654 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:40.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:32:40.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-03-30T00:32:40.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:40.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:40.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:40.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:40.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:40.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:40.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:40.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:40.655 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:40 smithi099 ceph-mon[38079]: Deploying daemon mds.l on smithi161 2024-03-30T00:32:40.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:32:40.657 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:32:40.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:40.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:32:40.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-03-30T00:32:40.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:40.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:40.705 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:40.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:40.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:40.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:40.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:40.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:40.706 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:40 smithi161 ceph-mon[42221]: Deploying daemon mds.l on smithi161 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: Saving service mds.all spec with placement smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:40 smithi142 ceph-mon[41779]: Deploying daemon mds.l on smithi161 2024-03-30T00:32:40.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.845+0000 7f8783000700 1 -- 172.21.15.99:0/2587881319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f877c102910 msgr2=0x7f877c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:40.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.845+0000 7f8783000700 1 --2- 172.21.15.99:0/2587881319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f877c102910 0x7f877c1091c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f877400b0d0 tx=0x7f877400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.845+0000 7f8783000700 1 -- 172.21.15.99:0/2587881319 shutdown_connections 2024-03-30T00:32:40.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.845+0000 7f8783000700 1 --2- 172.21.15.99:0/2587881319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f877c102910 0x7f877c1091c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.845+0000 7f8783000700 1 --2- 172.21.15.99:0/2587881319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f877c100d80 0x7f877c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.845+0000 7f8783000700 1 --2- 172.21.15.99:0/2587881319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f877c101f70 0x7f877c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.845+0000 7f8783000700 1 -- 172.21.15.99:0/2587881319 >> 172.21.15.99:0/2587881319 conn(0x7f877c0fc2f0 msgr2=0x7f877c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:40.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.845+0000 7f8783000700 1 -- 172.21.15.99:0/2587881319 shutdown_connections 2024-03-30T00:32:40.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.845+0000 7f8783000700 1 -- 172.21.15.99:0/2587881319 wait complete. 2024-03-30T00:32:40.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.846+0000 7f8783000700 1 Processor -- start 2024-03-30T00:32:40.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.846+0000 7f8783000700 1 -- start start 2024-03-30T00:32:40.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8783000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f877c100d80 0x7f877c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:40.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8783000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f877c101f70 0x7f877c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8783000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f877c102910 0x7f877c1a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8783000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c1a0a80 con 0x7f877c102910 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8783000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c1a0bc0 con 0x7f877c101f70 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8783000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c1a0d50 con 0x7f877c100d80 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8781000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f877c102910 0x7f877c1a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f877be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f877c100d80 0x7f877c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f877be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f877c100d80 0x7f877c198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:37718/0 (socket says 172.21.15.99:37718) 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8781000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f877c102910 0x7f877c1a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48988/0 (socket says 172.21.15.99:48988) 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f877be00700 1 -- 172.21.15.99:0/1242720005 learned_addr learned my addr 172.21.15.99:0/1242720005 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f877b400700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f877c101f70 0x7f877c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8781000700 1 -- 172.21.15.99:0/1242720005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f877c100d80 msgr2=0x7f877c198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8781000700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f877c100d80 0x7f877c198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8781000700 1 -- 172.21.15.99:0/1242720005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f877c101f70 msgr2=0x7f877c1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:40.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8781000700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f877c101f70 0x7f877c1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.847+0000 7f8781000700 1 -- 172.21.15.99:0/1242720005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8774009d00 con 0x7f877c102910 2024-03-30T00:32:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.848+0000 7f8781000700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f877c102910 0x7f877c1a04b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f877400b770 tx=0x7f877400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.848+0000 7f8778c00700 1 -- 172.21.15.99:0/1242720005 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877400e040 con 0x7f877c102910 2024-03-30T00:32:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.848+0000 7f8778c00700 1 -- 172.21.15.99:0/1242720005 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8774004c50 con 0x7f877c102910 2024-03-30T00:32:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.848+0000 7f8778c00700 1 -- 172.21.15.99:0/1242720005 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877401b6b0 con 0x7f877c102910 2024-03-30T00:32:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.848+0000 7f8783000700 1 -- 172.21.15.99:0/1242720005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c1a0fd0 con 0x7f877c102910 2024-03-30T00:32:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.848+0000 7f8783000700 1 -- 172.21.15.99:0/1242720005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c1a13a0 con 0x7f877c102910 2024-03-30T00:32:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.849+0000 7f8783000700 1 -- 172.21.15.99:0/1242720005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877c067af0 con 0x7f877c102910 2024-03-30T00:32:40.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.851+0000 7f8778c00700 1 -- 172.21.15.99:0/1242720005 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f877401d480 con 0x7f877c102910 2024-03-30T00:32:40.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.853+0000 7f8778c00700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f87600a1120 0x7f87600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:40.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.853+0000 7f8778c00700 1 -- 172.21.15.99:0/1242720005 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f87740cddf0 con 0x7f877c102910 2024-03-30T00:32:40.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.853+0000 7f877be00700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f87600a1120 0x7f87600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:40.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.854+0000 7f877be00700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f87600a1120 0x7f87600a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f876c005950 tx=0x7f876c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:40.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:40.857+0000 7f8778c00700 1 -- 172.21.15.99:0/1242720005 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8774097a10 con 0x7f877c102910 2024-03-30T00:32:41.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.065+0000 7f8783000700 1 -- 172.21.15.99:0/1242720005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f877c1a1670 con 0x7f877c102910 2024-03-30T00:32:41.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.066+0000 7f8778c00700 1 -- 172.21.15.99:0/1242720005 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f877409b030 con 0x7f877c102910 2024-03-30T00:32:41.073 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:32:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 -- 172.21.15.99:0/1242720005 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f87600a1120 msgr2=0x7f87600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f87600a1120 0x7f87600a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f876c005950 tx=0x7f876c009260 comp rx=0 tx=0).stop 2024-03-30T00:32:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 -- 172.21.15.99:0/1242720005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f877c102910 msgr2=0x7f877c1a04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f877c102910 0x7f877c1a04b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f877400b770 tx=0x7f877400cab0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 -- 172.21.15.99:0/1242720005 shutdown_connections 2024-03-30T00:32:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f877c102910 0x7f877c1a04b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f877c100d80 0x7f877c198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f87600a1120 0x7f87600a35e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 --2- 172.21.15.99:0/1242720005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f877c101f70 0x7f877c1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 -- 172.21.15.99:0/1242720005 >> 172.21.15.99:0/1242720005 conn(0x7f877c0fc2f0 msgr2=0x7f877c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 -- 172.21.15.99:0/1242720005 shutdown_connections 2024-03-30T00:32:41.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.068+0000 7f8783000700 1 -- 172.21.15.99:0/1242720005 wait complete. 2024-03-30T00:32:41.094 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:32:41.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.279+0000 7f93eb800700 1 -- 172.21.15.99:0/2214820427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e4102810 msgr2=0x7f93e41090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:41.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.279+0000 7f93eb800700 1 --2- 172.21.15.99:0/2214820427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e4102810 0x7f93e41090c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00b0d0 tx=0x7f93dc00b3e0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.279+0000 7f93eb800700 1 -- 172.21.15.99:0/2214820427 shutdown_connections 2024-03-30T00:32:41.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.279+0000 7f93eb800700 1 --2- 172.21.15.99:0/2214820427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e4102810 0x7f93e41090c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.279+0000 7f93eb800700 1 --2- 172.21.15.99:0/2214820427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e4100c80 0x7f93e41010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.279+0000 7f93eb800700 1 --2- 172.21.15.99:0/2214820427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4101e70 0x7f93e41022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.279+0000 7f93eb800700 1 -- 172.21.15.99:0/2214820427 >> 172.21.15.99:0/2214820427 conn(0x7f93e40fc1f0 msgr2=0x7f93e40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.279+0000 7f93eb800700 1 -- 172.21.15.99:0/2214820427 shutdown_connections 2024-03-30T00:32:41.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.279+0000 7f93eb800700 1 -- 172.21.15.99:0/2214820427 wait complete. 2024-03-30T00:32:41.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.280+0000 7f93eb800700 1 Processor -- start 2024-03-30T00:32:41.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.281+0000 7f93eb800700 1 -- start start 2024-03-30T00:32:41.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.281+0000 7f93eb800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e4100c80 0x7f93e4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:41.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.281+0000 7f93eb800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4101e70 0x7f93e4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:41.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.281+0000 7f93eb800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e4102810 0x7f93e41a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:41.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.281+0000 7f93eb800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41a08f0 con 0x7f93e4102810 2024-03-30T00:32:41.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.281+0000 7f93eb800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41a0a60 con 0x7f93e4101e70 2024-03-30T00:32:41.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.281+0000 7f93eb800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41a0c20 con 0x7f93e4100c80 2024-03-30T00:32:41.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.281+0000 7f93e9800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e4102810 0x7f93e41a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.281+0000 7f93e9800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e4102810 0x7f93e41a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49000/0 (socket says 172.21.15.99:49000) 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.281+0000 7f93e9800700 1 -- 172.21.15.99:0/1784742390 learned_addr learned my addr 172.21.15.99:0/1784742390 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.281+0000 7f93e8e00700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e4100c80 0x7f93e4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93e3e00700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4101e70 0x7f93e4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93e9800700 1 -- 172.21.15.99:0/1784742390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e4100c80 msgr2=0x7f93e4198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93e9800700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e4100c80 0x7f93e4198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93e9800700 1 -- 172.21.15.99:0/1784742390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4101e70 msgr2=0x7f93e4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93e9800700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4101e70 0x7f93e4199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93e9800700 1 -- 172.21.15.99:0/1784742390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc009d00 con 0x7f93e4102810 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93e9800700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e4102810 0x7f93e41a0320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0079c0 tx=0x7f93dc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93e1600700 1 -- 172.21.15.99:0/1784742390 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc00e040 con 0x7f93e4102810 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93e1600700 1 -- 172.21.15.99:0/1784742390 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93dc004e60 con 0x7f93e4102810 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93eb800700 1 -- 172.21.15.99:0/1784742390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e41a0ea0 con 0x7f93e4102810 2024-03-30T00:32:41.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93e1600700 1 -- 172.21.15.99:0/1784742390 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc01b6b0 con 0x7f93e4102810 2024-03-30T00:32:41.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.282+0000 7f93eb800700 1 -- 172.21.15.99:0/1784742390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e41a1260 con 0x7f93e4102810 2024-03-30T00:32:41.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.283+0000 7f93eb800700 1 -- 172.21.15.99:0/1784742390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e4067a10 con 0x7f93e4102810 2024-03-30T00:32:41.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.285+0000 7f93e1600700 1 -- 172.21.15.99:0/1784742390 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f93dc01d9a0 con 0x7f93e4102810 2024-03-30T00:32:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.286+0000 7f93e1600700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93cc0a11a0 0x7f93cc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.286+0000 7f93e1600700 1 -- 172.21.15.99:0/1784742390 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f93dc0cdf10 con 0x7f93e4102810 2024-03-30T00:32:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.287+0000 7f93e8e00700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93cc0a11a0 0x7f93cc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.287+0000 7f93e8e00700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93cc0a11a0 0x7f93cc0a3660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f93e419a280 tx=0x7f93d8007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:41.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.290+0000 7f93e1600700 1 -- 172.21.15.99:0/1784742390 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f93dc097b30 con 0x7f93e4102810 2024-03-30T00:32:41.368 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 30 00:32:41 smithi161 systemd[1]: Starting Ceph mds.l for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:32:41.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.540+0000 7f93eb800700 1 -- 172.21.15.99:0/1784742390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93e410c350 con 0x7f93e4102810 2024-03-30T00:32:41.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.541+0000 7f93e1600700 1 -- 172.21.15.99:0/1784742390 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f93dc09b150 con 0x7f93e4102810 2024-03-30T00:32:41.548 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:41.548 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-30T00:32:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 -- 172.21.15.99:0/1784742390 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93cc0a11a0 msgr2=0x7f93cc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93cc0a11a0 0x7f93cc0a3660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f93e419a280 tx=0x7f93d8007400 comp rx=0 tx=0).stop 2024-03-30T00:32:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 -- 172.21.15.99:0/1784742390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e4102810 msgr2=0x7f93e41a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e4102810 0x7f93e41a0320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0079c0 tx=0x7f93dc00ccc0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 -- 172.21.15.99:0/1784742390 shutdown_connections 2024-03-30T00:32:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e4102810 0x7f93e41a0320 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93e4100c80 0x7f93e4198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93cc0a11a0 0x7f93cc0a3660 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 --2- 172.21.15.99:0/1784742390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4101e70 0x7f93e4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:41.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 -- 172.21.15.99:0/1784742390 >> 172.21.15.99:0/1784742390 conn(0x7f93e40fc1f0 msgr2=0x7f93e4072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:41.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 -- 172.21.15.99:0/1784742390 shutdown_connections 2024-03-30T00:32:41.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:41.543+0000 7f93eb800700 1 -- 172.21.15.99:0/1784742390 wait complete. 2024-03-30T00:32:41.551 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 1 2024-03-30T00:32:41.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:41 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1286388067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:32:41.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:41 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1242720005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:32:41.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:41 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1286388067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:32:41.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:41 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1242720005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:32:41.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:41 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1286388067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:32:41.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:41 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1242720005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:32:42.219 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 30 00:32:41 smithi161 bash[78883]: e77ed2d3f8088e27665a83cee8323cdfc0eb902307e52f8f51fe92400bbc7d87 2024-03-30T00:32:42.219 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 30 00:32:41 smithi161 ceph-mds[78921]: -- 172.21.15.161:0/283537571 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562c2af101a0 con 0x562c2aee8c00 2024-03-30T00:32:42.219 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 30 00:32:41 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-l[78898]: starting mds.l at 2024-03-30T00:32:42.219 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 30 00:32:41 smithi161 systemd[1]: Started Ceph mds.l for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:32:42.674 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:42 smithi142 ceph-mon[41779]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:42.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:42 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1784742390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:42.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:42 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.161:6832/3055946850,v1:172.21.15.161:6833/3055946850] up:boot 2024-03-30T00:32:42.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:42 smithi142 ceph-mon[41779]: fsmap 1 up:standby 2024-03-30T00:32:42.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T00:32:42.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:42.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:42.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:42.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:42.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:42.675 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:42 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:42 smithi161 ceph-mon[42221]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:42 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1784742390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:42 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.161:6832/3055946850,v1:172.21.15.161:6833/3055946850] up:boot 2024-03-30T00:32:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:42 smithi161 ceph-mon[42221]: fsmap 1 up:standby 2024-03-30T00:32:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T00:32:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:42.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:42.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:42.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:42 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:42.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:42 smithi099 ceph-mon[38079]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:42 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1784742390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:42 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.161:6832/3055946850,v1:172.21.15.161:6833/3055946850] up:boot 2024-03-30T00:32:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:42 smithi099 ceph-mon[38079]: fsmap 1 up:standby 2024-03-30T00:32:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T00:32:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:42 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:43.570 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:32:43.634 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:43 smithi099 ceph-mon[38079]: Deploying daemon mds.k on smithi142 2024-03-30T00:32:43.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:43 smithi142 ceph-mon[41779]: Deploying daemon mds.k on smithi142 2024-03-30T00:32:43.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:43 smithi161 ceph-mon[42221]: Deploying daemon mds.k on smithi142 2024-03-30T00:32:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.765+0000 7fddc4a00700 1 -- 172.21.15.99:0/91283335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc0100c80 msgr2=0x7fddc01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.765+0000 7fddc4a00700 1 --2- 172.21.15.99:0/91283335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc0100c80 0x7fddc01010a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fddb00099c0 tx=0x7fddb0009cd0 comp rx=0 tx=0).stop 2024-03-30T00:32:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.766+0000 7fddc4a00700 1 -- 172.21.15.99:0/91283335 shutdown_connections 2024-03-30T00:32:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.766+0000 7fddc4a00700 1 --2- 172.21.15.99:0/91283335 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddc0102810 0x7fddc01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.766+0000 7fddc4a00700 1 --2- 172.21.15.99:0/91283335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc0100c80 0x7fddc01010a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.766+0000 7fddc4a00700 1 --2- 172.21.15.99:0/91283335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddc0101e70 0x7fddc01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.766+0000 7fddc4a00700 1 -- 172.21.15.99:0/91283335 >> 172.21.15.99:0/91283335 conn(0x7fddc00fc1f0 msgr2=0x7fddc00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.766+0000 7fddc4a00700 1 -- 172.21.15.99:0/91283335 shutdown_connections 2024-03-30T00:32:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.766+0000 7fddc4a00700 1 -- 172.21.15.99:0/91283335 wait complete. 2024-03-30T00:32:43.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddc4a00700 1 Processor -- start 2024-03-30T00:32:43.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddc4a00700 1 -- start start 2024-03-30T00:32:43.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddc4a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddc0100c80 0x7fddc0198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:43.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddc4a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc0101e70 0x7fddc01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:43.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddc4a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddc0102810 0x7fddc01a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:43.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddc4a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc01a0880 con 0x7fddc0101e70 2024-03-30T00:32:43.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddc4a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc01a09f0 con 0x7fddc0100c80 2024-03-30T00:32:43.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddc4a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc01a0bb0 con 0x7fddc0102810 2024-03-30T00:32:43.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddbd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc0101e70 0x7fddc01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:43.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddbde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddc0100c80 0x7fddc0198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:43.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddbd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc0101e70 0x7fddc01992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:49022/0 (socket says 172.21.15.99:49022) 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.767+0000 7fddbd400700 1 -- 172.21.15.99:0/3323072356 learned_addr learned my addr 172.21.15.99:0/3323072356 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.768+0000 7fddbd400700 1 -- 172.21.15.99:0/3323072356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddc0102810 msgr2=0x7fddc01a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.768+0000 7fddbd400700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddc0102810 0x7fddc01a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.768+0000 7fddbd400700 1 -- 172.21.15.99:0/3323072356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddc0100c80 msgr2=0x7fddc0198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.768+0000 7fddbd400700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddc0100c80 0x7fddc0198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.768+0000 7fddbd400700 1 -- 172.21.15.99:0/3323072356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb00095f0 con 0x7fddc0101e70 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.768+0000 7fddbd400700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc0101e70 0x7fddc01992d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fddac00e9a0 tx=0x7fddac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.768+0000 7fddb6a00700 1 -- 172.21.15.99:0/3323072356 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac009790 con 0x7fddc0101e70 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.768+0000 7fddb6a00700 1 -- 172.21.15.99:0/3323072356 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fddac00cbd0 con 0x7fddc0101e70 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.768+0000 7fddc4a00700 1 -- 172.21.15.99:0/3323072356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc01a0e90 con 0x7fddc0101e70 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.768+0000 7fddb6a00700 1 -- 172.21.15.99:0/3323072356 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac018680 con 0x7fddc0101e70 2024-03-30T00:32:43.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.768+0000 7fddc4a00700 1 -- 172.21.15.99:0/3323072356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc01a1300 con 0x7fddc0101e70 2024-03-30T00:32:43.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.769+0000 7fddc4a00700 1 -- 172.21.15.99:0/3323072356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc0067a10 con 0x7fddc0101e70 2024-03-30T00:32:43.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.771+0000 7fddb6a00700 1 -- 172.21.15.99:0/3323072356 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fddac00cd40 con 0x7fddc0101e70 2024-03-30T00:32:43.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.773+0000 7fddb6a00700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdda40a11a0 0x7fdda40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:43.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.774+0000 7fddb6a00700 1 -- 172.21.15.99:0/3323072356 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fddac014070 con 0x7fddc0101e70 2024-03-30T00:32:43.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.774+0000 7fddbde00700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdda40a11a0 0x7fdda40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:43.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.774+0000 7fddbde00700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdda40a11a0 0x7fdda40a3660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fddb0000c00 tx=0x7fddb0003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:43.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:43.777+0000 7fddb6a00700 1 -- 172.21.15.99:0/3323072356 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fddac0529f0 con 0x7fddc0101e70 2024-03-30T00:32:44.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.030+0000 7fddc4a00700 1 -- 172.21.15.99:0/3323072356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddc010c390 con 0x7fddc0101e70 2024-03-30T00:32:44.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.030+0000 7fddb6a00700 1 -- 172.21.15.99:0/3323072356 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7fddac096270 con 0x7fddc0101e70 2024-03-30T00:32:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:44.038 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-30T00:32:44.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 -- 172.21.15.99:0/3323072356 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdda40a11a0 msgr2=0x7fdda40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:44.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdda40a11a0 0x7fdda40a3660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fddb0000c00 tx=0x7fddb0003680 comp rx=0 tx=0).stop 2024-03-30T00:32:44.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 -- 172.21.15.99:0/3323072356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc0101e70 msgr2=0x7fddc01992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:44.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc0101e70 0x7fddc01992d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fddac00e9a0 tx=0x7fddac00ecb0 comp rx=0 tx=0).stop 2024-03-30T00:32:44.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 -- 172.21.15.99:0/3323072356 shutdown_connections 2024-03-30T00:32:44.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddc0102810 0x7fddc01a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdda40a11a0 0x7fdda40a3660 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddc0100c80 0x7fddc0198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 --2- 172.21.15.99:0/3323072356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddc0101e70 0x7fddc01992d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 -- 172.21.15.99:0/3323072356 >> 172.21.15.99:0/3323072356 conn(0x7fddc00fc1f0 msgr2=0x7fddc0107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 -- 172.21.15.99:0/3323072356 shutdown_connections 2024-03-30T00:32:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:44.033+0000 7fddc4a00700 1 -- 172.21.15.99:0/3323072356 wait complete. 2024-03-30T00:32:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 2 2024-03-30T00:32:44.067 INFO:journalctl@ceph.mds.k.smithi142.stdout:Mar 30 00:32:43 smithi142 systemd[1]: Starting Ceph mds.k for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:32:44.567 INFO:journalctl@ceph.mds.k.smithi142.stdout:Mar 30 00:32:44 smithi142 bash[76851]: 72f5314df4c435577a01565a470be432aac469903fc2fafb6e4141f040833491 2024-03-30T00:32:44.567 INFO:journalctl@ceph.mds.k.smithi142.stdout:Mar 30 00:32:44 smithi142 systemd[1]: Started Ceph mds.k for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:32:44.567 INFO:journalctl@ceph.mds.k.smithi142.stdout:Mar 30 00:32:44 smithi142 ceph-mds[76889]: -- 172.21.15.142:0/3353628261 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ba634941a0 con 0x55ba6346cc00 2024-03-30T00:32:44.567 INFO:journalctl@ceph.mds.k.smithi142.stdout:Mar 30 00:32:44 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-k[76866]: starting mds.k at 2024-03-30T00:32:44.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:44 smithi142 ceph-mon[41779]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:44.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:44 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3323072356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:44.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:44 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:44.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:44 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:44.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:44 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:44.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:44 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:44.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:44 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:44.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:44 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:44.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:44 smithi161 ceph-mon[42221]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:44.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:44 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3323072356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:44.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:44 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:44.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:44 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:44.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:44 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:44.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:44 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:44.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:44 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:44.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:44 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:44.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:44 smithi099 ceph-mon[38079]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:44.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:44 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3323072356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:44.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:44 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:44.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:44 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:44.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:44 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:44.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:44 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:44.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:44 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:44.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:44 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:45.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:45 smithi099 ceph-mon[38079]: Deploying daemon mds.j on smithi099 2024-03-30T00:32:45.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:45 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.142:6832/780858221,v1:172.21.15.142:6833/780858221] up:boot 2024-03-30T00:32:45.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:45 smithi099 ceph-mon[38079]: fsmap 2 up:standby 2024-03-30T00:32:45.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:45 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T00:32:45.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:45 smithi161 ceph-mon[42221]: Deploying daemon mds.j on smithi099 2024-03-30T00:32:45.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:45 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.142:6832/780858221,v1:172.21.15.142:6833/780858221] up:boot 2024-03-30T00:32:45.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:45 smithi161 ceph-mon[42221]: fsmap 2 up:standby 2024-03-30T00:32:45.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:45 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T00:32:45.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:45 smithi142 ceph-mon[41779]: Deploying daemon mds.j on smithi099 2024-03-30T00:32:45.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:45 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.142:6832/780858221,v1:172.21.15.142:6833/780858221] up:boot 2024-03-30T00:32:45.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:45 smithi142 ceph-mon[41779]: fsmap 2 up:standby 2024-03-30T00:32:45.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:45 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T00:32:46.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:32:46.123 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 30 00:32:46 smithi099 systemd[1]: Starting Ceph mds.j for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:32:46.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.266+0000 7f5330c00700 1 -- 172.21.15.99:0/2443752762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c074df0 msgr2=0x7f532c073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:46.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.266+0000 7f5330c00700 1 --2- 172.21.15.99:0/2443752762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c074df0 0x7f532c073240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f531c0098f0 tx=0x7f531c009c00 comp rx=0 tx=0).stop 2024-03-30T00:32:46.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.266+0000 7f5330c00700 1 -- 172.21.15.99:0/2443752762 shutdown_connections 2024-03-30T00:32:46.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.266+0000 7f5330c00700 1 --2- 172.21.15.99:0/2443752762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c10d250 0x7f532c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:46.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.266+0000 7f5330c00700 1 --2- 172.21.15.99:0/2443752762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c074df0 0x7f532c073240 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:46.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.266+0000 7f5330c00700 1 --2- 172.21.15.99:0/2443752762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f532c073810 0x7f532c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:46.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.266+0000 7f5330c00700 1 -- 172.21.15.99:0/2443752762 >> 172.21.15.99:0/2443752762 conn(0x7f532c0fc1f0 msgr2=0x7f532c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:46.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.266+0000 7f5330c00700 1 -- 172.21.15.99:0/2443752762 shutdown_connections 2024-03-30T00:32:46.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.266+0000 7f5330c00700 1 -- 172.21.15.99:0/2443752762 wait complete. 2024-03-30T00:32:46.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.267+0000 7f5330c00700 1 Processor -- start 2024-03-30T00:32:46.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.267+0000 7f5330c00700 1 -- start start 2024-03-30T00:32:46.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.268+0000 7f5330c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c073810 0x7f532c19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:46.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.268+0000 7f5330c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f532c074df0 0x7f532c19d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:46.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.268+0000 7f5330c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c10d250 0x7f532c1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:46.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.268+0000 7f5330c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1a4cf0 con 0x7f532c073810 2024-03-30T00:32:46.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.268+0000 7f5330c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1a4e30 con 0x7f532c10d250 2024-03-30T00:32:46.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.268+0000 7f5330c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1a4fc0 con 0x7f532c074df0 2024-03-30T00:32:46.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.268+0000 7f5329400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f532c074df0 0x7f532c19d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.268+0000 7f5329e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c073810 0x7f532c19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.268+0000 7f5329400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f532c074df0 0x7f532c19d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:35752/0 (socket says 172.21.15.99:35752) 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.268+0000 7f5329400700 1 -- 172.21.15.99:0/1780685680 learned_addr learned my addr 172.21.15.99:0/1780685680 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.269+0000 7f532a800700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c10d250 0x7f532c1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.269+0000 7f5329400700 1 -- 172.21.15.99:0/1780685680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c10d250 msgr2=0x7f532c1a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.269+0000 7f5329400700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c10d250 0x7f532c1a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.269+0000 7f5329400700 1 -- 172.21.15.99:0/1780685680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c073810 msgr2=0x7f532c19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.269+0000 7f5329400700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c073810 0x7f532c19d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.269+0000 7f5329400700 1 -- 172.21.15.99:0/1780685680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5324009d00 con 0x7f532c074df0 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.269+0000 7f5329400700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f532c074df0 0x7f532c19d750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f531800e970 tx=0x7f531800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.269+0000 7f5322a00700 1 -- 172.21.15.99:0/1780685680 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318009790 con 0x7f532c074df0 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.269+0000 7f5330c00700 1 -- 172.21.15.99:0/1780685680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c009520 con 0x7f532c074df0 2024-03-30T00:32:46.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.269+0000 7f5322a00700 1 -- 172.21.15.99:0/1780685680 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f531800c7e0 con 0x7f532c074df0 2024-03-30T00:32:46.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.270+0000 7f5322a00700 1 -- 172.21.15.99:0/1780685680 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318003bd0 con 0x7f532c074df0 2024-03-30T00:32:46.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.270+0000 7f5330c00700 1 -- 172.21.15.99:0/1780685680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c1a5670 con 0x7f532c074df0 2024-03-30T00:32:46.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.271+0000 7f5330c00700 1 -- 172.21.15.99:0/1780685680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c1034b0 con 0x7f532c074df0 2024-03-30T00:32:46.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.273+0000 7f5322a00700 1 -- 172.21.15.99:0/1780685680 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f531800c950 con 0x7f532c074df0 2024-03-30T00:32:46.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.274+0000 7f5322a00700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53100a1220 0x7f53100a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:46.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.274+0000 7f5322a00700 1 -- 172.21.15.99:0/1780685680 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5318014070 con 0x7f532c074df0 2024-03-30T00:32:46.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.274+0000 7f5329e00700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53100a1220 0x7f53100a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:46.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.275+0000 7f5329e00700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53100a1220 0x7f53100a36e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f531c00f9d0 tx=0x7f531c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:46.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.276+0000 7f5322a00700 1 -- 172.21.15.99:0/1780685680 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f53180968c0 con 0x7f532c074df0 2024-03-30T00:32:46.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.557+0000 7f5330c00700 1 -- 172.21.15.99:0/1780685680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f532c067a10 con 0x7f532c074df0 2024-03-30T00:32:46.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.558+0000 7f5322a00700 1 -- 172.21.15.99:0/1780685680 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7f5318099ee0 con 0x7f532c074df0 2024-03-30T00:32:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:46.565 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-30T00:32:46.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 -- 172.21.15.99:0/1780685680 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53100a1220 msgr2=0x7f53100a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:46.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53100a1220 0x7f53100a36e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f531c00f9d0 tx=0x7f531c019040 comp rx=0 tx=0).stop 2024-03-30T00:32:46.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 -- 172.21.15.99:0/1780685680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f532c074df0 msgr2=0x7f532c19d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f532c074df0 0x7f532c19d750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f531800e970 tx=0x7f531800ec80 comp rx=0 tx=0).stop 2024-03-30T00:32:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 -- 172.21.15.99:0/1780685680 shutdown_connections 2024-03-30T00:32:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53100a1220 0x7f53100a36e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c10d250 0x7f532c1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f532c073810 0x7f532c19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 --2- 172.21.15.99:0/1780685680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f532c074df0 0x7f532c19d750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 -- 172.21.15.99:0/1780685680 >> 172.21.15.99:0/1780685680 conn(0x7f532c0fc1f0 msgr2=0x7f532c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 -- 172.21.15.99:0/1780685680 shutdown_connections 2024-03-30T00:32:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:46.560+0000 7f5330c00700 1 -- 172.21.15.99:0/1780685680 wait complete. 2024-03-30T00:32:46.568 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 3 2024-03-30T00:32:46.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:46 smithi161 ceph-mon[42221]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:46.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:46 smithi099 ceph-mon[38079]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:46.738 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 30 00:32:46 smithi099 bash[86707]: 519a94f9796fa258814daa2712118270216e9f15324337a98f6dd9bf2800a223 2024-03-30T00:32:46.738 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 30 00:32:46 smithi099 systemd[1]: Started Ceph mds.j for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:32:46.738 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 30 00:32:46 smithi099 ceph-mds[86765]: -- 172.21.15.99:0/2802249801 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c4428e1a0 con 0x555c44266c00 2024-03-30T00:32:46.738 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 30 00:32:46 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-j[86741]: starting mds.j at 2024-03-30T00:32:46.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:46 smithi142 ceph-mon[41779]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:47.629 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:47 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1780685680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:47.629 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:47.629 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:47.629 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:47.629 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:47.629 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:47.629 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:47 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:47.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:47 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1780685680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:47.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:47.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:47.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:47.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:47.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:47.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:47 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:47.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:47 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1780685680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:47.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:47.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:47.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:47.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:47.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:47.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:47 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:48.488 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 30 00:32:48 smithi099 systemd[1]: Starting Ceph mds.d for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:32:48.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:48 smithi099 ceph-mon[38079]: Deploying daemon mds.d on smithi099 2024-03-30T00:32:48.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:48 smithi099 ceph-mon[38079]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:48.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:48 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.99:6834/380315573,v1:172.21.15.99:6835/380315573] up:boot 2024-03-30T00:32:48.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:48 smithi099 ceph-mon[38079]: fsmap 3 up:standby 2024-03-30T00:32:48.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:48 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-30T00:32:48.589 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:32:48.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:48 smithi161 ceph-mon[42221]: Deploying daemon mds.d on smithi099 2024-03-30T00:32:48.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:48 smithi161 ceph-mon[42221]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:48.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:48 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.99:6834/380315573,v1:172.21.15.99:6835/380315573] up:boot 2024-03-30T00:32:48.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:48 smithi161 ceph-mon[42221]: fsmap 3 up:standby 2024-03-30T00:32:48.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:48 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-30T00:32:48.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.790+0000 7fa881c00700 1 -- 172.21.15.99:0/1751261821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c10d250 msgr2=0x7fa87c10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:48.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.790+0000 7fa881c00700 1 --2- 172.21.15.99:0/1751261821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c10d250 0x7fa87c10f640 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa87400b0a0 tx=0x7fa87400b3b0 comp rx=0 tx=0).stop 2024-03-30T00:32:48.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.790+0000 7fa881c00700 1 -- 172.21.15.99:0/1751261821 shutdown_connections 2024-03-30T00:32:48.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.790+0000 7fa881c00700 1 --2- 172.21.15.99:0/1751261821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c10d250 0x7fa87c10f640 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:48.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.790+0000 7fa881c00700 1 --2- 172.21.15.99:0/1751261821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa87c074df0 0x7fa87c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:48.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.790+0000 7fa881c00700 1 --2- 172.21.15.99:0/1751261821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa87c073810 0x7fa87c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:48.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.790+0000 7fa881c00700 1 -- 172.21.15.99:0/1751261821 >> 172.21.15.99:0/1751261821 conn(0x7fa87c0fc1f0 msgr2=0x7fa87c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:48.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.791+0000 7fa881c00700 1 -- 172.21.15.99:0/1751261821 shutdown_connections 2024-03-30T00:32:48.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.791+0000 7fa881c00700 1 -- 172.21.15.99:0/1751261821 wait complete. 2024-03-30T00:32:48.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.792+0000 7fa881c00700 1 Processor -- start 2024-03-30T00:32:48.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.792+0000 7fa881c00700 1 -- start start 2024-03-30T00:32:48.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.792+0000 7fa881c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c073810 0x7fa87c19d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.792+0000 7fa881c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa87c074df0 0x7fa87c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.792+0000 7fa881c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa87c10d250 0x7fa87c1a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.792+0000 7fa881c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c1a4d10 con 0x7fa87c073810 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.792+0000 7fa881c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c1a4e50 con 0x7fa87c074df0 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.792+0000 7fa881c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c1a4fe0 con 0x7fa87c10d250 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.792+0000 7fa87aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c073810 0x7fa87c19d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.792+0000 7fa87aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c073810 0x7fa87c19d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36712/0 (socket says 172.21.15.99:36712) 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.792+0000 7fa87aa00700 1 -- 172.21.15.99:0/4184398493 learned_addr learned my addr 172.21.15.99:0/4184398493 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa87a000700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa87c074df0 0x7fa87c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa87b400700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa87c10d250 0x7fa87c1a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa87aa00700 1 -- 172.21.15.99:0/4184398493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa87c10d250 msgr2=0x7fa87c1a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa87aa00700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa87c10d250 0x7fa87c1a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa87aa00700 1 -- 172.21.15.99:0/4184398493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa87c074df0 msgr2=0x7fa87c19d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:48.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa87aa00700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa87c074df0 0x7fa87c19d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:48.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa87aa00700 1 -- 172.21.15.99:0/4184398493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa874009d00 con 0x7fa87c073810 2024-03-30T00:32:48.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa87aa00700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c073810 0x7fa87c19d1e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00e9a0 tx=0x7fa86c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:48.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa873400700 1 -- 172.21.15.99:0/4184398493 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c009790 con 0x7fa87c073810 2024-03-30T00:32:48.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa873400700 1 -- 172.21.15.99:0/4184398493 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa86c010430 con 0x7fa87c073810 2024-03-30T00:32:48.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa881c00700 1 -- 172.21.15.99:0/4184398493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c1a52c0 con 0x7fa87c073810 2024-03-30T00:32:48.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.793+0000 7fa873400700 1 -- 172.21.15.99:0/4184398493 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c00cea0 con 0x7fa87c073810 2024-03-30T00:32:48.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.794+0000 7fa881c00700 1 -- 172.21.15.99:0/4184398493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87c1a57d0 con 0x7fa87c073810 2024-03-30T00:32:48.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.795+0000 7fa881c00700 1 -- 172.21.15.99:0/4184398493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87c1034b0 con 0x7fa87c073810 2024-03-30T00:32:48.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.796+0000 7fa873400700 1 -- 172.21.15.99:0/4184398493 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa86c0105a0 con 0x7fa87c073810 2024-03-30T00:32:48.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.798+0000 7fa873400700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa8600a1120 0x7fa8600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:48.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.798+0000 7fa873400700 1 -- 172.21.15.99:0/4184398493 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa86c014070 con 0x7fa87c073810 2024-03-30T00:32:48.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.798+0000 7fa87a000700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa8600a1120 0x7fa8600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:48.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.799+0000 7fa87a000700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa8600a1120 0x7fa8600a35e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa87c074b10 tx=0x7fa868009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:48.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:48.800+0000 7fa873400700 1 -- 172.21.15.99:0/4184398493 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa86c096430 con 0x7fa87c073810 2024-03-30T00:32:48.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:48 smithi142 ceph-mon[41779]: Deploying daemon mds.d on smithi099 2024-03-30T00:32:48.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:48 smithi142 ceph-mon[41779]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:48.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:48 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.99:6834/380315573,v1:172.21.15.99:6835/380315573] up:boot 2024-03-30T00:32:48.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:48 smithi142 ceph-mon[41779]: fsmap 3 up:standby 2024-03-30T00:32:48.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:48 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-30T00:32:49.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.100+0000 7fa881c00700 1 -- 172.21.15.99:0/4184398493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa87c067a10 con 0x7fa87c073810 2024-03-30T00:32:49.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.101+0000 7fa873400700 1 -- 172.21.15.99:0/4184398493 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2771 (secure 0 0 0) 0x7fa86c026af0 con 0x7fa87c073810 2024-03-30T00:32:49.107 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:49.108 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-30T00:32:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 -- 172.21.15.99:0/4184398493 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa8600a1120 msgr2=0x7fa8600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa8600a1120 0x7fa8600a35e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa87c074b10 tx=0x7fa868009260 comp rx=0 tx=0).stop 2024-03-30T00:32:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 -- 172.21.15.99:0/4184398493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c073810 msgr2=0x7fa87c19d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c073810 0x7fa87c19d1e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00e9a0 tx=0x7fa86c00ecb0 comp rx=0 tx=0).stop 2024-03-30T00:32:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 -- 172.21.15.99:0/4184398493 shutdown_connections 2024-03-30T00:32:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa87c10d250 0x7fa87c1a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:49.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa87c073810 0x7fa87c19d1e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa8600a1120 0x7fa8600a35e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 --2- 172.21.15.99:0/4184398493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa87c074df0 0x7fa87c19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 -- 172.21.15.99:0/4184398493 >> 172.21.15.99:0/4184398493 conn(0x7fa87c0fc1f0 msgr2=0x7fa87c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 -- 172.21.15.99:0/4184398493 shutdown_connections 2024-03-30T00:32:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:49.103+0000 7fa881c00700 1 -- 172.21.15.99:0/4184398493 wait complete. 2024-03-30T00:32:49.111 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 4 2024-03-30T00:32:49.237 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 30 00:32:48 smithi099 ceph-mds[87151]: -- 172.21.15.99:0/2387800217 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562cf1c3a1a0 con 0x562cf1c12c00 2024-03-30T00:32:49.237 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 30 00:32:48 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-d[87108]: starting mds.d at 2024-03-30T00:32:49.237 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 30 00:32:48 smithi099 bash[87070]: 43ecc6bbd0545d6dacbb2dd2088fc285681374500807f9be484ed34341016c14 2024-03-30T00:32:49.238 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 30 00:32:48 smithi099 systemd[1]: Started Ceph mds.d for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:32:50.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:50.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:50.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:50.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:50.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:50.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:50.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:50.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: Deploying daemon mds.e on smithi142 2024-03-30T00:32:50.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4184398493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:50.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.99:6836/4277780379,v1:172.21.15.99:6837/4277780379] up:boot 2024-03-30T00:32:50.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: fsmap 4 up:standby 2024-03-30T00:32:50.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:50 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-30T00:32:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:50.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:50.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: Deploying daemon mds.e on smithi142 2024-03-30T00:32:50.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4184398493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:50.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.99:6836/4277780379,v1:172.21.15.99:6837/4277780379] up:boot 2024-03-30T00:32:50.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: fsmap 4 up:standby 2024-03-30T00:32:50.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:50 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-30T00:32:50.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:50.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:50.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:50.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:50.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:50.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:50.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:50.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: Deploying daemon mds.e on smithi142 2024-03-30T00:32:50.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4184398493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:50.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.99:6836/4277780379,v1:172.21.15.99:6837/4277780379] up:boot 2024-03-30T00:32:50.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: fsmap 4 up:standby 2024-03-30T00:32:50.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:50 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-30T00:32:50.875 INFO:journalctl@ceph.mds.e.smithi142.stdout:Mar 30 00:32:50 smithi142 systemd[1]: Starting Ceph mds.e for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:32:51.130 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:32:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.326+0000 7fa775800700 1 -- 172.21.15.99:0/738905172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa770073810 msgr2=0x7fa770073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.326+0000 7fa775800700 1 --2- 172.21.15.99:0/738905172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa770073810 0x7fa770073c90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009990 tx=0x7fa75c009ca0 comp rx=0 tx=0).stop 2024-03-30T00:32:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.326+0000 7fa775800700 1 -- 172.21.15.99:0/738905172 shutdown_connections 2024-03-30T00:32:51.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.326+0000 7fa775800700 1 --2- 172.21.15.99:0/738905172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa770108b40 0x7fa77010afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:51.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.326+0000 7fa775800700 1 --2- 172.21.15.99:0/738905172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa770074df0 0x7fa770073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:51.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.326+0000 7fa775800700 1 --2- 172.21.15.99:0/738905172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa770073810 0x7fa770073c90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:51.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.326+0000 7fa775800700 1 -- 172.21.15.99:0/738905172 >> 172.21.15.99:0/738905172 conn(0x7fa7700fc1f0 msgr2=0x7fa7700fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:51.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.326+0000 7fa775800700 1 -- 172.21.15.99:0/738905172 shutdown_connections 2024-03-30T00:32:51.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.326+0000 7fa775800700 1 -- 172.21.15.99:0/738905172 wait complete. 2024-03-30T00:32:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.327+0000 7fa775800700 1 Processor -- start 2024-03-30T00:32:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.327+0000 7fa775800700 1 -- start start 2024-03-30T00:32:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.327+0000 7fa775800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa770073810 0x7fa770198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.327+0000 7fa775800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa770074df0 0x7fa770199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.327+0000 7fa775800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa770108b40 0x7fa7701a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.327+0000 7fa775800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7701a0810 con 0x7fa770074df0 2024-03-30T00:32:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.327+0000 7fa775800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7701a0980 con 0x7fa770073810 2024-03-30T00:32:51.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.327+0000 7fa775800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7701a0b40 con 0x7fa770108b40 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa770074df0 0x7fa770199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa770074df0 0x7fa770199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36766/0 (socket says 172.21.15.99:36766) 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76e000700 1 -- 172.21.15.99:0/2205475400 learned_addr learned my addr 172.21.15.99:0/2205475400 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76ea00700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa770073810 0x7fa770198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76f400700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa770108b40 0x7fa7701a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76e000700 1 -- 172.21.15.99:0/2205475400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa770108b40 msgr2=0x7fa7701a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76e000700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa770108b40 0x7fa7701a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76e000700 1 -- 172.21.15.99:0/2205475400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa770073810 msgr2=0x7fa770198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76e000700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa770073810 0x7fa770198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76e000700 1 -- 172.21.15.99:0/2205475400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c0095f0 con 0x7fa770074df0 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76f400700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa770108b40 0x7fa7701a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.328+0000 7fa76e000700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa770074df0 0x7fa770199230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa75c000c00 tx=0x7fa75c003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.329+0000 7fa767400700 1 -- 172.21.15.99:0/2205475400 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c01d070 con 0x7fa770074df0 2024-03-30T00:32:51.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.329+0000 7fa775800700 1 -- 172.21.15.99:0/2205475400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7701a0dc0 con 0x7fa770074df0 2024-03-30T00:32:51.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.329+0000 7fa767400700 1 -- 172.21.15.99:0/2205475400 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa75c00bd70 con 0x7fa770074df0 2024-03-30T00:32:51.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.329+0000 7fa767400700 1 -- 172.21.15.99:0/2205475400 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c022bf0 con 0x7fa770074df0 2024-03-30T00:32:51.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.329+0000 7fa775800700 1 -- 172.21.15.99:0/2205475400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7701a1190 con 0x7fa770074df0 2024-03-30T00:32:51.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.330+0000 7fa775800700 1 -- 172.21.15.99:0/2205475400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7701071f0 con 0x7fa770074df0 2024-03-30T00:32:51.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.332+0000 7fa767400700 1 -- 172.21.15.99:0/2205475400 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa75c022d50 con 0x7fa770074df0 2024-03-30T00:32:51.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.333+0000 7fa767400700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa7540a1120 0x7fa7540a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:51.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.333+0000 7fa767400700 1 -- 172.21.15.99:0/2205475400 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa75c0cd8b0 con 0x7fa770074df0 2024-03-30T00:32:51.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.333+0000 7fa76ea00700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa7540a1120 0x7fa7540a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:51.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.334+0000 7fa76ea00700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa7540a1120 0x7fa7540a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa7700ffd40 tx=0x7fa7600073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:51.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.337+0000 7fa767400700 1 -- 172.21.15.99:0/2205475400 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa75c0975f0 con 0x7fa770074df0 2024-03-30T00:32:51.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.595+0000 7fa775800700 1 -- 172.21.15.99:0/2205475400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa77010e7d0 con 0x7fa770074df0 2024-03-30T00:32:51.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.595+0000 7fa767400700 1 -- 172.21.15.99:0/2205475400 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3528 (secure 0 0 0) 0x7fa75c027de0 con 0x7fa770074df0 2024-03-30T00:32:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:51.604 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-30T00:32:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.597+0000 7fa775800700 1 -- 172.21.15.99:0/2205475400 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa7540a1120 msgr2=0x7fa7540a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.597+0000 7fa775800700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa7540a1120 0x7fa7540a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa7700ffd40 tx=0x7fa7600073d0 comp rx=0 tx=0).stop 2024-03-30T00:32:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.597+0000 7fa775800700 1 -- 172.21.15.99:0/2205475400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa770074df0 msgr2=0x7fa770199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.597+0000 7fa775800700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa770074df0 0x7fa770199230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa75c000c00 tx=0x7fa75c003a50 comp rx=0 tx=0).stop 2024-03-30T00:32:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.598+0000 7fa775800700 1 -- 172.21.15.99:0/2205475400 shutdown_connections 2024-03-30T00:32:51.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.598+0000 7fa775800700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa770108b40 0x7fa7701a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:51.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.598+0000 7fa775800700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa7540a1120 0x7fa7540a35e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:51.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.598+0000 7fa775800700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa770073810 0x7fa770198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:51.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.598+0000 7fa775800700 1 --2- 172.21.15.99:0/2205475400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa770074df0 0x7fa770199230 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:51.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.598+0000 7fa775800700 1 -- 172.21.15.99:0/2205475400 >> 172.21.15.99:0/2205475400 conn(0x7fa7700fc1f0 msgr2=0x7fa7700fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:51.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.598+0000 7fa775800700 1 -- 172.21.15.99:0/2205475400 shutdown_connections 2024-03-30T00:32:51.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:51.598+0000 7fa775800700 1 -- 172.21.15.99:0/2205475400 wait complete. 2024-03-30T00:32:51.609 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 5 2024-03-30T00:32:51.817 INFO:journalctl@ceph.mds.e.smithi142.stdout:Mar 30 00:32:51 smithi142 bash[77195]: 9f784a44d663783a267ef3acb0c8027bcec747f4ad7c86eda693740a027fdcd3 2024-03-30T00:32:51.817 INFO:journalctl@ceph.mds.e.smithi142.stdout:Mar 30 00:32:51 smithi142 systemd[1]: Started Ceph mds.e for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:32:51.817 INFO:journalctl@ceph.mds.e.smithi142.stdout:Mar 30 00:32:51 smithi142 ceph-mds[77233]: -- 172.21.15.142:0/935026817 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564e816ea1a0 con 0x564e816c2800 2024-03-30T00:32:51.817 INFO:journalctl@ceph.mds.e.smithi142.stdout:Mar 30 00:32:51 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-e[77210]: starting mds.e at 2024-03-30T00:32:52.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:52.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:52.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:52.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:52.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:52.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:52.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:52.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: Deploying daemon mds.f on smithi161 2024-03-30T00:32:52.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2205475400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:52.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.142:6834/1641167461,v1:172.21.15.142:6835/1641167461] up:boot 2024-03-30T00:32:52.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: fsmap 5 up:standby 2024-03-30T00:32:52.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:52 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T00:32:52.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:52.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:52.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:52.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:52.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:52.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:52.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:52.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: Deploying daemon mds.f on smithi161 2024-03-30T00:32:52.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2205475400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:52.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.142:6834/1641167461,v1:172.21.15.142:6835/1641167461] up:boot 2024-03-30T00:32:52.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: fsmap 5 up:standby 2024-03-30T00:32:52.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:52 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: Deploying daemon mds.f on smithi161 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2205475400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.142:6834/1641167461,v1:172.21.15.142:6835/1641167461] up:boot 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: fsmap 5 up:standby 2024-03-30T00:32:52.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:52 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T00:32:53.325 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 30 00:32:53 smithi161 systemd[1]: Starting Ceph mds.f for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:32:53.626 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:32:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.821+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1649935696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e0108b40 msgr2=0x7fe1e010afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.821+0000 7fe1e6e00700 1 --2- 172.21.15.99:0/1649935696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e0108b40 0x7fe1e010afd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800b0a0 tx=0x7fe1d800b3b0 comp rx=0 tx=0).stop 2024-03-30T00:32:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.821+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1649935696 shutdown_connections 2024-03-30T00:32:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.821+0000 7fe1e6e00700 1 --2- 172.21.15.99:0/1649935696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e0108b40 0x7fe1e010afd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.821+0000 7fe1e6e00700 1 --2- 172.21.15.99:0/1649935696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0074df0 0x7fe1e0073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.821+0000 7fe1e6e00700 1 --2- 172.21.15.99:0/1649935696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0073810 0x7fe1e0073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.821+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1649935696 >> 172.21.15.99:0/1649935696 conn(0x7fe1e00fc1f0 msgr2=0x7fe1e00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.822+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1649935696 shutdown_connections 2024-03-30T00:32:53.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.822+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1649935696 wait complete. 2024-03-30T00:32:53.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.823+0000 7fe1e6e00700 1 Processor -- start 2024-03-30T00:32:53.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.823+0000 7fe1e6e00700 1 -- start start 2024-03-30T00:32:53.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1e6e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e0073810 0x7fe1e0198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:53.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1e6e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0074df0 0x7fe1e0199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:53.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1e6e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0108b40 0x7fe1e01a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1e6e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e01a0930 con 0x7fe1e0073810 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1e6e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e01a0a70 con 0x7fe1e0074df0 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1e6e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e01a0c00 con 0x7fe1e0108b40 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1dfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e0073810 0x7fe1e0198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1dfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e0073810 0x7fe1e0198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36786/0 (socket says 172.21.15.99:36786) 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1dfe00700 1 -- 172.21.15.99:0/1906813310 learned_addr learned my addr 172.21.15.99:0/1906813310 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1e4e00700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0108b40 0x7fe1e01a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1df400700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0074df0 0x7fe1e0199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1dfe00700 1 -- 172.21.15.99:0/1906813310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0108b40 msgr2=0x7fe1e01a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1dfe00700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0108b40 0x7fe1e01a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1dfe00700 1 -- 172.21.15.99:0/1906813310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0074df0 msgr2=0x7fe1e0199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1dfe00700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0074df0 0x7fe1e0199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:53.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1dfe00700 1 -- 172.21.15.99:0/1906813310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8009d00 con 0x7fe1e0073810 2024-03-30T00:32:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.824+0000 7fe1df400700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0074df0 0x7fe1e0199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:32:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.825+0000 7fe1dfe00700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e0073810 0x7fe1e0198e00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000d950 tx=0x7fe1d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.825+0000 7fe1dcc00700 1 -- 172.21.15.99:0/1906813310 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0009940 con 0x7fe1e0073810 2024-03-30T00:32:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.825+0000 7fe1dcc00700 1 -- 172.21.15.99:0/1906813310 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1d0010460 con 0x7fe1e0073810 2024-03-30T00:32:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.825+0000 7fe1dcc00700 1 -- 172.21.15.99:0/1906813310 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d000f620 con 0x7fe1e0073810 2024-03-30T00:32:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.825+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1906813310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e01a0ee0 con 0x7fe1e0073810 2024-03-30T00:32:53.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.825+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1906813310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e01a12a0 con 0x7fe1e0073810 2024-03-30T00:32:53.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.826+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1906813310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e0107900 con 0x7fe1e0073810 2024-03-30T00:32:53.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.828+0000 7fe1dcc00700 1 -- 172.21.15.99:0/1906813310 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fe1d00105d0 con 0x7fe1e0073810 2024-03-30T00:32:53.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.829+0000 7fe1dcc00700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1c40a1120 0x7fe1c40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:53.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.829+0000 7fe1dcc00700 1 -- 172.21.15.99:0/1906813310 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe1d00cc310 con 0x7fe1e0073810 2024-03-30T00:32:53.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.829+0000 7fe1df400700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1c40a1120 0x7fe1c40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:53.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.830+0000 7fe1df400700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1c40a1120 0x7fe1c40a35e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0074b10 tx=0x7fe1cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:53.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:53.833+0000 7fe1dcc00700 1 -- 172.21.15.99:0/1906813310 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe1d0095ea0 con 0x7fe1e0073810 2024-03-30T00:32:54.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.097+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1906813310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1e0067a60 con 0x7fe1e0073810 2024-03-30T00:32:54.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.098+0000 7fe1dcc00700 1 -- 172.21.15.99:0/1906813310 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4288 (secure 0 0 0) 0x7fe1d00994c0 con 0x7fe1e0073810 2024-03-30T00:32:54.106 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:54.106 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-03-30T00:32:54.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.100+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1906813310 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1c40a1120 msgr2=0x7fe1c40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:54.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.100+0000 7fe1e6e00700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1c40a1120 0x7fe1c40a35e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0074b10 tx=0x7fe1cc009260 comp rx=0 tx=0).stop 2024-03-30T00:32:54.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.100+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1906813310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e0073810 msgr2=0x7fe1e0198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:54.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.100+0000 7fe1e6e00700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e0073810 0x7fe1e0198e00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000d950 tx=0x7fe1d000dc60 comp rx=0 tx=0).stop 2024-03-30T00:32:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.100+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1906813310 shutdown_connections 2024-03-30T00:32:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.100+0000 7fe1e6e00700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1e0108b40 0x7fe1e01a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.100+0000 7fe1e6e00700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1e0073810 0x7fe1e0198e00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.100+0000 7fe1e6e00700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1c40a1120 0x7fe1c40a35e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.100+0000 7fe1e6e00700 1 --2- 172.21.15.99:0/1906813310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1e0074df0 0x7fe1e0199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.100+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1906813310 >> 172.21.15.99:0/1906813310 conn(0x7fe1e00fc1f0 msgr2=0x7fe1e0104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.101+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1906813310 shutdown_connections 2024-03-30T00:32:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:54.101+0000 7fe1e6e00700 1 -- 172.21.15.99:0/1906813310 wait complete. 2024-03-30T00:32:54.109 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-03-30T00:32:54.219 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 30 00:32:53 smithi161 bash[79223]: 24b249e2726f930db2899b5eca6124ae2e83b7493d49c1a50f48481c95cc8551 2024-03-30T00:32:54.219 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 30 00:32:53 smithi161 systemd[1]: Started Ceph mds.f for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:32:54.219 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 30 00:32:53 smithi161 ceph-mds[79261]: -- 172.21.15.161:0/1747893205 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653e56a01a0 con 0x5653e5678c00 2024-03-30T00:32:54.219 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 30 00:32:53 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-f[79238]: starting mds.f at 2024-03-30T00:32:54.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:54 smithi161 ceph-mon[42221]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:54.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:54.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:54.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:54.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:54.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:54.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:54 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:54.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:54 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1906813310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:54.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:54 smithi099 ceph-mon[38079]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:54.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:54 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:54.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:54 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1906813310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:54.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:54 smithi142 ceph-mon[41779]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:54.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:54.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:54.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:54.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:54.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:54.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:54 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:54.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:54 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1906813310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:55.582 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:55 smithi161 ceph-mon[42221]: Deploying daemon mds.c on smithi161 2024-03-30T00:32:55.583 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:55 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] up:boot 2024-03-30T00:32:55.583 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:55 smithi161 ceph-mon[42221]: fsmap 6 up:standby 2024-03-30T00:32:55.583 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:55 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T00:32:55.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:55 smithi099 ceph-mon[38079]: Deploying daemon mds.c on smithi161 2024-03-30T00:32:55.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:55 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] up:boot 2024-03-30T00:32:55.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:55 smithi099 ceph-mon[38079]: fsmap 6 up:standby 2024-03-30T00:32:55.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:55 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T00:32:55.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:55 smithi142 ceph-mon[41779]: Deploying daemon mds.c on smithi161 2024-03-30T00:32:55.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:55 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] up:boot 2024-03-30T00:32:55.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:55 smithi142 ceph-mon[41779]: fsmap 6 up:standby 2024-03-30T00:32:55.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:55 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T00:32:55.970 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 30 00:32:55 smithi161 systemd[1]: Starting Ceph mds.c for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:32:56.128 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:32:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.322+0000 7fa34a800700 1 -- 172.21.15.99:0/1061130920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa344106a30 msgr2=0x7fa344108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.322+0000 7fa34a800700 1 --2- 172.21.15.99:0/1061130920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa344106a30 0x7fa344108e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00b0a0 tx=0x7fa33c00b3b0 comp rx=0 tx=0).stop 2024-03-30T00:32:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.322+0000 7fa34a800700 1 -- 172.21.15.99:0/1061130920 shutdown_connections 2024-03-30T00:32:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.322+0000 7fa34a800700 1 --2- 172.21.15.99:0/1061130920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa344106a30 0x7fa344108e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.322+0000 7fa34a800700 1 --2- 172.21.15.99:0/1061130920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3441017d0 0x7fa344103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:56.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.322+0000 7fa34a800700 1 --2- 172.21.15.99:0/1061130920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa344104100 0x7fa3441064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:56.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.322+0000 7fa34a800700 1 -- 172.21.15.99:0/1061130920 >> 172.21.15.99:0/1061130920 conn(0x7fa3440fb100 msgr2=0x7fa3440fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:56.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.322+0000 7fa34a800700 1 -- 172.21.15.99:0/1061130920 shutdown_connections 2024-03-30T00:32:56.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.322+0000 7fa34a800700 1 -- 172.21.15.99:0/1061130920 wait complete. 2024-03-30T00:32:56.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.323+0000 7fa34a800700 1 Processor -- start 2024-03-30T00:32:56.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.323+0000 7fa34a800700 1 -- start start 2024-03-30T00:32:56.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.323+0000 7fa34a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3441017d0 0x7fa344196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.323+0000 7fa34a800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa344104100 0x7fa344197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.323+0000 7fa34a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa344106a30 0x7fa34419e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.323+0000 7fa34a800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34419e680 con 0x7fa3441017d0 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.323+0000 7fa34a800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34419e7c0 con 0x7fa344106a30 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.323+0000 7fa34a800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34419e950 con 0x7fa344104100 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa343400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3441017d0 0x7fa344196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa343400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3441017d0 0x7fa344196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37210/0 (socket says 172.21.15.99:37210) 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa343400700 1 -- 172.21.15.99:0/829043556 learned_addr learned my addr 172.21.15.99:0/829043556 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa342a00700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa344104100 0x7fa344197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa343e00700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa344106a30 0x7fa34419e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa343400700 1 -- 172.21.15.99:0/829043556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa344104100 msgr2=0x7fa344197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa343400700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa344104100 0x7fa344197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa343400700 1 -- 172.21.15.99:0/829043556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa344106a30 msgr2=0x7fa34419e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:56.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa343400700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa344106a30 0x7fa34419e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:56.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa343400700 1 -- 172.21.15.99:0/829043556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c009d00 con 0x7fa3441017d0 2024-03-30T00:32:56.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa343400700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3441017d0 0x7fa344196b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa33400c9b0 tx=0x7fa33400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:56.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa33be00700 1 -- 172.21.15.99:0/829043556 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334007b00 con 0x7fa3441017d0 2024-03-30T00:32:56.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa33be00700 1 -- 172.21.15.99:0/829043556 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa334007c60 con 0x7fa3441017d0 2024-03-30T00:32:56.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa34a800700 1 -- 172.21.15.99:0/829043556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34419ec30 con 0x7fa3441017d0 2024-03-30T00:32:56.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.324+0000 7fa33be00700 1 -- 172.21.15.99:0/829043556 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334018660 con 0x7fa3441017d0 2024-03-30T00:32:56.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.325+0000 7fa34a800700 1 -- 172.21.15.99:0/829043556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34419f140 con 0x7fa3441017d0 2024-03-30T00:32:56.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.325+0000 7fa34a800700 1 -- 172.21.15.99:0/829043556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3440fcc80 con 0x7fa3441017d0 2024-03-30T00:32:56.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.328+0000 7fa33be00700 1 -- 172.21.15.99:0/829043556 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa33400f450 con 0x7fa3441017d0 2024-03-30T00:32:56.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.329+0000 7fa33be00700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3280a1120 0x7fa3280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:56.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.329+0000 7fa33be00700 1 -- 172.21.15.99:0/829043556 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa3340cc760 con 0x7fa3441017d0 2024-03-30T00:32:56.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.329+0000 7fa342a00700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3280a1120 0x7fa3280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:56.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.330+0000 7fa342a00700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3280a1120 0x7fa3280a35e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa330009c60 tx=0x7fa330009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:56.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.332+0000 7fa33be00700 1 -- 172.21.15.99:0/829043556 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa3340963c0 con 0x7fa3441017d0 2024-03-30T00:32:56.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.586+0000 7fa34a800700 1 -- 172.21.15.99:0/829043556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa34410c320 con 0x7fa3441017d0 2024-03-30T00:32:56.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.587+0000 7fa33be00700 1 -- 172.21.15.99:0/829043556 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5042 (secure 0 0 0) 0x7fa33400fe60 con 0x7fa3441017d0 2024-03-30T00:32:56.595 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:56.595 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-03-30T00:32:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 -- 172.21.15.99:0/829043556 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3280a1120 msgr2=0x7fa3280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3280a1120 0x7fa3280a35e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa330009c60 tx=0x7fa330009260 comp rx=0 tx=0).stop 2024-03-30T00:32:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 -- 172.21.15.99:0/829043556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3441017d0 msgr2=0x7fa344196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3441017d0 0x7fa344196b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa33400c9b0 tx=0x7fa33400ccc0 comp rx=0 tx=0).stop 2024-03-30T00:32:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 -- 172.21.15.99:0/829043556 shutdown_connections 2024-03-30T00:32:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa3280a1120 0x7fa3280a35e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa344106a30 0x7fa34419e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3441017d0 0x7fa344196b50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 --2- 172.21.15.99:0/829043556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa344104100 0x7fa344197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 -- 172.21.15.99:0/829043556 >> 172.21.15.99:0/829043556 conn(0x7fa3440fb100 msgr2=0x7fa344102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 -- 172.21.15.99:0/829043556 shutdown_connections 2024-03-30T00:32:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:56.589+0000 7fa34a800700 1 -- 172.21.15.99:0/829043556 wait complete. 2024-03-30T00:32:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 7 2024-03-30T00:32:56.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:56 smithi161 ceph-mon[42221]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:56.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:56 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:56.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:56 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:56.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:56 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:56.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:56 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:56.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:56 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:56.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:56 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:56.720 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 30 00:32:56 smithi161 bash[79565]: 5850d9e78d22b054ba9385daebc9e3316b3c52c2e5f4ee55c6eba6bf60bcb490 2024-03-30T00:32:56.720 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 30 00:32:56 smithi161 systemd[1]: Started Ceph mds.c for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:32:56.720 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 30 00:32:56 smithi161 ceph-mds[79603]: -- 172.21.15.161:0/702491965 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560dd5dba1a0 con 0x560dd5d92800 2024-03-30T00:32:56.720 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 30 00:32:56 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-c[79580]: starting mds.c at 2024-03-30T00:32:56.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:56 smithi099 ceph-mon[38079]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:56.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:56 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:56.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:56 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:56.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:56 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:56.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:56 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:56.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:56 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:56.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:56 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:56.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:56 smithi142 ceph-mon[41779]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:56.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:56 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:56.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:56 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:56.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:56 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:56.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:56 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:56.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:56 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:56.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:56 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:57.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:57 smithi161 ceph-mon[42221]: Deploying daemon mds.h on smithi142 2024-03-30T00:32:57.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:57 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/829043556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:57.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:57 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.161:6836/2189606703,v1:172.21.15.161:6837/2189606703] up:boot 2024-03-30T00:32:57.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:57 smithi161 ceph-mon[42221]: fsmap 7 up:standby 2024-03-30T00:32:57.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:57 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T00:32:57.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:57 smithi099 ceph-mon[38079]: Deploying daemon mds.h on smithi142 2024-03-30T00:32:57.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:57 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/829043556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:57.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:57 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.161:6836/2189606703,v1:172.21.15.161:6837/2189606703] up:boot 2024-03-30T00:32:57.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:57 smithi099 ceph-mon[38079]: fsmap 7 up:standby 2024-03-30T00:32:57.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:57 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T00:32:57.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:57 smithi142 ceph-mon[41779]: Deploying daemon mds.h on smithi142 2024-03-30T00:32:57.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:57 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/829043556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:57.784 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:57 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.161:6836/2189606703,v1:172.21.15.161:6837/2189606703] up:boot 2024-03-30T00:32:57.784 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:57 smithi142 ceph-mon[41779]: fsmap 7 up:standby 2024-03-30T00:32:57.784 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:57 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T00:32:58.470 INFO:journalctl@ceph.mds.h.smithi142.stdout:Mar 30 00:32:58 smithi142 systemd[1]: Starting Ceph mds.h for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:32:58.618 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:32:58.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:58 smithi099 ceph-mon[38079]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:58.807 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:58 smithi142 ceph-mon[41779]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.814+0000 7fcef1c00700 1 -- 172.21.15.99:0/2303229873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fceec108b40 msgr2=0x7fceec10afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.814+0000 7fcef1c00700 1 --2- 172.21.15.99:0/2303229873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fceec108b40 0x7fceec10afd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcee400b0a0 tx=0x7fcee400b3b0 comp rx=0 tx=0).stop 2024-03-30T00:32:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.814+0000 7fcef1c00700 1 -- 172.21.15.99:0/2303229873 shutdown_connections 2024-03-30T00:32:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.814+0000 7fcef1c00700 1 --2- 172.21.15.99:0/2303229873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fceec108b40 0x7fceec10afd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.814+0000 7fcef1c00700 1 --2- 172.21.15.99:0/2303229873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec074df0 0x7fceec073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.814+0000 7fcef1c00700 1 --2- 172.21.15.99:0/2303229873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fceec073810 0x7fceec073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.814+0000 7fcef1c00700 1 -- 172.21.15.99:0/2303229873 >> 172.21.15.99:0/2303229873 conn(0x7fceec0fc1f0 msgr2=0x7fceec0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.814+0000 7fcef1c00700 1 -- 172.21.15.99:0/2303229873 shutdown_connections 2024-03-30T00:32:58.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.814+0000 7fcef1c00700 1 -- 172.21.15.99:0/2303229873 wait complete. 2024-03-30T00:32:58.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.815+0000 7fcef1c00700 1 Processor -- start 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.815+0000 7fcef1c00700 1 -- start start 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.815+0000 7fcef1c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec073810 0x7fceec198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.815+0000 7fcef1c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fceec074df0 0x7fceec199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.815+0000 7fcef1c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fceec108b40 0x7fceec1a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fcef1c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec1a0930 con 0x7fceec074df0 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fcef1c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec1a0a70 con 0x7fceec073810 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fcef1c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec1a0c00 con 0x7fceec108b40 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fceea000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fceec074df0 0x7fceec199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fceeaa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec073810 0x7fceec198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fceea000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fceec074df0 0x7fceec199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37238/0 (socket says 172.21.15.99:37238) 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fceea000700 1 -- 172.21.15.99:0/1597209286 learned_addr learned my addr 172.21.15.99:0/1597209286 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fceeb400700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fceec108b40 0x7fceec1a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:58.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fceea000700 1 -- 172.21.15.99:0/1597209286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fceec108b40 msgr2=0x7fceec1a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:58.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fceea000700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fceec108b40 0x7fceec1a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:58.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fceea000700 1 -- 172.21.15.99:0/1597209286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec073810 msgr2=0x7fceec198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:58.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fceea000700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec073810 0x7fceec198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:58.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fceea000700 1 -- 172.21.15.99:0/1597209286 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc009520 con 0x7fceec074df0 2024-03-30T00:32:58.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.816+0000 7fceea000700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fceec074df0 0x7fceec199340 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fced800e970 tx=0x7fced800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:58.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.817+0000 7fcee3400700 1 -- 172.21.15.99:0/1597209286 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8009790 con 0x7fceec074df0 2024-03-30T00:32:58.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.817+0000 7fcee3400700 1 -- 172.21.15.99:0/1597209286 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fced800c7e0 con 0x7fceec074df0 2024-03-30T00:32:58.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.817+0000 7fcef1c00700 1 -- 172.21.15.99:0/1597209286 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee4009d00 con 0x7fceec074df0 2024-03-30T00:32:58.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.817+0000 7fcee3400700 1 -- 172.21.15.99:0/1597209286 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8003bd0 con 0x7fceec074df0 2024-03-30T00:32:58.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.817+0000 7fcef1c00700 1 -- 172.21.15.99:0/1597209286 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceec1a1200 con 0x7fceec074df0 2024-03-30T00:32:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.818+0000 7fcef1c00700 1 -- 172.21.15.99:0/1597209286 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceec107900 con 0x7fceec074df0 2024-03-30T00:32:58.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.820+0000 7fcee3400700 1 -- 172.21.15.99:0/1597209286 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fced800c950 con 0x7fceec074df0 2024-03-30T00:32:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.821+0000 7fcee3400700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fced00a1120 0x7fced00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:32:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.821+0000 7fcee3400700 1 -- 172.21.15.99:0/1597209286 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fced8014070 con 0x7fceec074df0 2024-03-30T00:32:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.821+0000 7fceeaa00700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fced00a1120 0x7fced00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:32:58.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.822+0000 7fceeaa00700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fced00a1120 0x7fced00a35e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009df0 tx=0x7fcedc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:32:58.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:58.825+0000 7fcee3400700 1 -- 172.21.15.99:0/1597209286 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fced8096200 con 0x7fceec074df0 2024-03-30T00:32:58.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:58 smithi161 ceph-mon[42221]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:59.066 INFO:journalctl@ceph.mds.h.smithi142.stdout:Mar 30 00:32:58 smithi142 bash[77539]: 9bc08616c821b37049ee68e76323ca6d385ada7892e62539004a1c3ae99ba214 2024-03-30T00:32:59.067 INFO:journalctl@ceph.mds.h.smithi142.stdout:Mar 30 00:32:58 smithi142 systemd[1]: Started Ceph mds.h for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:32:59.067 INFO:journalctl@ceph.mds.h.smithi142.stdout:Mar 30 00:32:58 smithi142 ceph-mds[77577]: -- 172.21.15.142:0/1040158045 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e0041c1a0 con 0x560e003f4800 2024-03-30T00:32:59.067 INFO:journalctl@ceph.mds.h.smithi142.stdout:Mar 30 00:32:58 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-h[77554]: starting mds.h at 2024-03-30T00:32:59.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.081+0000 7fcef1c00700 1 -- 172.21.15.99:0/1597209286 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fceec067a10 con 0x7fceec074df0 2024-03-30T00:32:59.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.082+0000 7fcee3400700 1 -- 172.21.15.99:0/1597209286 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5802 (secure 0 0 0) 0x7fced8099820 con 0x7fceec074df0 2024-03-30T00:32:59.090 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:32:59.090 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-03-30T00:32:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.084+0000 7fcef1c00700 1 -- 172.21.15.99:0/1597209286 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fced00a1120 msgr2=0x7fced00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.084+0000 7fcef1c00700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fced00a1120 0x7fced00a35e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009df0 tx=0x7fcedc009260 comp rx=0 tx=0).stop 2024-03-30T00:32:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.084+0000 7fcef1c00700 1 -- 172.21.15.99:0/1597209286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fceec074df0 msgr2=0x7fceec199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:32:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.084+0000 7fcef1c00700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fceec074df0 0x7fceec199340 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fced800e970 tx=0x7fced800ec80 comp rx=0 tx=0).stop 2024-03-30T00:32:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.085+0000 7fcef1c00700 1 -- 172.21.15.99:0/1597209286 shutdown_connections 2024-03-30T00:32:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.085+0000 7fcef1c00700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fceec108b40 0x7fceec1a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.085+0000 7fcef1c00700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fced00a1120 0x7fced00a35e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.085+0000 7fcef1c00700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fceec073810 0x7fceec198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.085+0000 7fcef1c00700 1 --2- 172.21.15.99:0/1597209286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fceec074df0 0x7fceec199340 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:32:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.085+0000 7fcef1c00700 1 -- 172.21.15.99:0/1597209286 >> 172.21.15.99:0/1597209286 conn(0x7fceec0fc1f0 msgr2=0x7fceec104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:32:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.085+0000 7fcef1c00700 1 -- 172.21.15.99:0/1597209286 shutdown_connections 2024-03-30T00:32:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:32:59.085+0000 7fcef1c00700 1 -- 172.21.15.99:0/1597209286 wait complete. 2024-03-30T00:32:59.093 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2024-03-30T00:32:59.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:59.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:59.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:32:59.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:32:59.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:32:59.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:32:59.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: Deploying daemon mds.i on smithi161 2024-03-30T00:32:59.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:32:59.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1597209286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:32:59.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.142:6836/364916855,v1:172.21.15.142:6837/364916855] up:boot 2024-03-30T00:32:59.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: fsmap 8 up:standby 2024-03-30T00:32:59.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:32:59 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T00:33:00.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:00.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:00.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:00.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:33:00.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:33:00.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:00.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: Deploying daemon mds.i on smithi161 2024-03-30T00:33:00.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:00.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1597209286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:00.238 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.142:6836/364916855,v1:172.21.15.142:6837/364916855] up:boot 2024-03-30T00:33:00.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: fsmap 8 up:standby 2024-03-30T00:33:00.239 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:32:59 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T00:33:00.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:00.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:00.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:00.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:33:00.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:33:00.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:00.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: Deploying daemon mds.i on smithi161 2024-03-30T00:33:00.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:00.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1597209286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:00.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.142:6836/364916855,v1:172.21.15.142:6837/364916855] up:boot 2024-03-30T00:33:00.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: fsmap 8 up:standby 2024-03-30T00:33:00.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:32:59 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T00:33:00.914 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 30 00:33:00 smithi161 systemd[1]: Starting Ceph mds.i for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:33:01.111 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:33:01.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.306+0000 7f375c800700 1 -- 172.21.15.99:0/510115727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3754100c80 msgr2=0x7f37541010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:01.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.306+0000 7f375c800700 1 --2- 172.21.15.99:0/510115727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3754100c80 0x7f37541010a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f37480099c0 tx=0x7f3748009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:01.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.306+0000 7f375c800700 1 -- 172.21.15.99:0/510115727 shutdown_connections 2024-03-30T00:33:01.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.306+0000 7f375c800700 1 --2- 172.21.15.99:0/510115727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3754102810 0x7f37541090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:01.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.306+0000 7f375c800700 1 --2- 172.21.15.99:0/510115727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3754100c80 0x7f37541010a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:01.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.306+0000 7f375c800700 1 --2- 172.21.15.99:0/510115727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3754101e70 0x7f37541022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:01.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.306+0000 7f375c800700 1 -- 172.21.15.99:0/510115727 >> 172.21.15.99:0/510115727 conn(0x7f37540fc1f0 msgr2=0x7f37540fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:01.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.306+0000 7f375c800700 1 -- 172.21.15.99:0/510115727 shutdown_connections 2024-03-30T00:33:01.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.306+0000 7f375c800700 1 -- 172.21.15.99:0/510115727 wait complete. 2024-03-30T00:33:01.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.307+0000 7f375c800700 1 Processor -- start 2024-03-30T00:33:01.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.307+0000 7f375c800700 1 -- start start 2024-03-30T00:33:01.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.307+0000 7f375c800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3754100c80 0x7f3754198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:01.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f375c800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3754101e70 0x7f3754199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:01.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f375c800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3754102810 0x7f37541a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:01.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f375c800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37541a08f0 con 0x7f3754100c80 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f375c800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37541a0a30 con 0x7f3754101e70 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f375c800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37541a0bc0 con 0x7f3754102810 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f3759e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3754100c80 0x7f3754198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f3759e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3754100c80 0x7f3754198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37254/0 (socket says 172.21.15.99:37254) 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f3759e00700 1 -- 172.21.15.99:0/3651871953 learned_addr learned my addr 172.21.15.99:0/3651871953 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f375a800700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3754102810 0x7f37541a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f3759400700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3754101e70 0x7f3754199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f3759e00700 1 -- 172.21.15.99:0/3651871953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3754102810 msgr2=0x7f37541a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f3759e00700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3754102810 0x7f37541a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f3759e00700 1 -- 172.21.15.99:0/3651871953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3754101e70 msgr2=0x7f3754199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f3759e00700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3754101e70 0x7f3754199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f3759e00700 1 -- 172.21.15.99:0/3651871953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37480095f0 con 0x7f3754100c80 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f375a800700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3754102810 0x7f37541a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:33:01.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.308+0000 7f3759e00700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3754100c80 0x7f3754198dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3748000c00 tx=0x7f374800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:01.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.309+0000 7f374ea00700 1 -- 172.21.15.99:0/3651871953 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374801d070 con 0x7f3754100c80 2024-03-30T00:33:01.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.309+0000 7f374ea00700 1 -- 172.21.15.99:0/3651871953 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3748005050 con 0x7f3754100c80 2024-03-30T00:33:01.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.309+0000 7f375c800700 1 -- 172.21.15.99:0/3651871953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37541a0e40 con 0x7f3754100c80 2024-03-30T00:33:01.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.309+0000 7f374ea00700 1 -- 172.21.15.99:0/3651871953 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748004a70 con 0x7f3754100c80 2024-03-30T00:33:01.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.309+0000 7f375c800700 1 -- 172.21.15.99:0/3651871953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37541a1200 con 0x7f3754100c80 2024-03-30T00:33:01.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.310+0000 7f375c800700 1 -- 172.21.15.99:0/3651871953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3754067a10 con 0x7f3754100c80 2024-03-30T00:33:01.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.312+0000 7f374ea00700 1 -- 172.21.15.99:0/3651871953 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f37480051c0 con 0x7f3754100c80 2024-03-30T00:33:01.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.313+0000 7f374ea00700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f373c0a11a0 0x7f373c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:01.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.313+0000 7f374ea00700 1 -- 172.21.15.99:0/3651871953 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f37480cd120 con 0x7f3754100c80 2024-03-30T00:33:01.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.313+0000 7f3759400700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f373c0a11a0 0x7f373c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:01.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.314+0000 7f3759400700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f373c0a11a0 0x7f373c0a3660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3744009c60 tx=0x7f3744009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:01.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.317+0000 7f374ea00700 1 -- 172.21.15.99:0/3651871953 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3748096d40 con 0x7f3754100c80 2024-03-30T00:33:01.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.581+0000 7f375c800700 1 -- 172.21.15.99:0/3651871953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f375410c390 con 0x7f3754100c80 2024-03-30T00:33:01.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.582+0000 7f374ea00700 1 -- 172.21.15.99:0/3651871953 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6559 (secure 0 0 0) 0x7f374809a360 con 0x7f3754100c80 2024-03-30T00:33:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:01.590 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-03-30T00:33:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.584+0000 7f375c800700 1 -- 172.21.15.99:0/3651871953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f373c0a11a0 msgr2=0x7f373c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.584+0000 7f375c800700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f373c0a11a0 0x7f373c0a3660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3744009c60 tx=0x7f3744009260 comp rx=0 tx=0).stop 2024-03-30T00:33:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.584+0000 7f375c800700 1 -- 172.21.15.99:0/3651871953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3754100c80 msgr2=0x7f3754198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.584+0000 7f375c800700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3754100c80 0x7f3754198dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3748000c00 tx=0x7f374800f690 comp rx=0 tx=0).stop 2024-03-30T00:33:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.584+0000 7f375c800700 1 -- 172.21.15.99:0/3651871953 shutdown_connections 2024-03-30T00:33:01.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.584+0000 7f375c800700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3754102810 0x7f37541a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:01.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.584+0000 7f375c800700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3754100c80 0x7f3754198dc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:01.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.584+0000 7f375c800700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f373c0a11a0 0x7f373c0a3660 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:01.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.584+0000 7f375c800700 1 --2- 172.21.15.99:0/3651871953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3754101e70 0x7f3754199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:01.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.584+0000 7f375c800700 1 -- 172.21.15.99:0/3651871953 >> 172.21.15.99:0/3651871953 conn(0x7f37540fc1f0 msgr2=0x7f37541076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:01.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.585+0000 7f375c800700 1 -- 172.21.15.99:0/3651871953 shutdown_connections 2024-03-30T00:33:01.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:01.585+0000 7f375c800700 1 -- 172.21.15.99:0/3651871953 wait complete. 2024-03-30T00:33:01.593 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 9 2024-03-30T00:33:01.719 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 30 00:33:01 smithi161 ceph-mds[79946]: -- 172.21.15.161:0/2724317263 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b7c61f21a0 con 0x55b7c61ca800 2024-03-30T00:33:01.719 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 30 00:33:01 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-i[79923]: starting mds.i at 2024-03-30T00:33:01.719 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 30 00:33:01 smithi161 bash[79908]: 6bf8e61fcde892e2e609a93053b1b8c4ab1671a78146b339f2fb4cc62ddb0cf4 2024-03-30T00:33:01.719 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 30 00:33:01 smithi161 systemd[1]: Started Ceph mds.i for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:33:02.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:02 smithi099 ceph-mon[38079]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:02.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:02.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:02.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:02.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:33:02.815 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:33:02.816 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:02.816 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:02 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3651871953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:02.816 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:02 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.161:6838/1486341215,v1:172.21.15.161:6839/1486341215] up:boot 2024-03-30T00:33:02.816 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:02 smithi099 ceph-mon[38079]: fsmap 9 up:standby 2024-03-30T00:33:02.816 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:02 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T00:33:02.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:02 smithi142 ceph-mon[41779]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:02.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:02.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:02.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:02.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:33:02.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:33:02.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:02.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:02 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3651871953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:02.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:02 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.161:6838/1486341215,v1:172.21.15.161:6839/1486341215] up:boot 2024-03-30T00:33:02.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:02 smithi142 ceph-mon[41779]: fsmap 9 up:standby 2024-03-30T00:33:02.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:02 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T00:33:02.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:02 smithi161 ceph-mon[42221]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:02.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:02.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:02.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:02.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:33:02.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:33:02.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:02.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:02 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3651871953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:02.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:02 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.161:6838/1486341215,v1:172.21.15.161:6839/1486341215] up:boot 2024-03-30T00:33:02.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:02 smithi161 ceph-mon[42221]: fsmap 9 up:standby 2024-03-30T00:33:02.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:02 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T00:33:03.488 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 30 00:33:03 smithi099 systemd[1]: Starting Ceph mds.g for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:33:03.613 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:33:03.806 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:03 smithi099 ceph-mon[38079]: Deploying daemon mds.g on smithi099 2024-03-30T00:33:03.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:03 smithi142 ceph-mon[41779]: Deploying daemon mds.g on smithi099 2024-03-30T00:33:03.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.814+0000 7f7523a00700 1 -- 172.21.15.99:0/749944508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f751c100c80 msgr2=0x7f751c1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:03.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.814+0000 7f7523a00700 1 --2- 172.21.15.99:0/749944508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f751c100c80 0x7f751c1010a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f75100099c0 tx=0x7f7510009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:03.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.814+0000 7f7523a00700 1 -- 172.21.15.99:0/749944508 shutdown_connections 2024-03-30T00:33:03.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.814+0000 7f7523a00700 1 --2- 172.21.15.99:0/749944508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f751c102810 0x7f751c1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:03.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.814+0000 7f7523a00700 1 --2- 172.21.15.99:0/749944508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f751c100c80 0x7f751c1010a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:03.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.814+0000 7f7523a00700 1 --2- 172.21.15.99:0/749944508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f751c101e70 0x7f751c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:03.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.814+0000 7f7523a00700 1 -- 172.21.15.99:0/749944508 >> 172.21.15.99:0/749944508 conn(0x7f751c0fc1f0 msgr2=0x7f751c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:03.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.814+0000 7f7523a00700 1 -- 172.21.15.99:0/749944508 shutdown_connections 2024-03-30T00:33:03.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.814+0000 7f7523a00700 1 -- 172.21.15.99:0/749944508 wait complete. 2024-03-30T00:33:03.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.815+0000 7f7523a00700 1 Processor -- start 2024-03-30T00:33:03.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.815+0000 7f7523a00700 1 -- start start 2024-03-30T00:33:03.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.816+0000 7f7523a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f751c100c80 0x7f751c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:03.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.816+0000 7f7523a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f751c101e70 0x7f751c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:03.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.816+0000 7f7523a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f751c102810 0x7f751c1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:03.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.816+0000 7f7523a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c1a08c0 con 0x7f751c101e70 2024-03-30T00:33:03.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.816+0000 7f7523a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c1a0a00 con 0x7f751c102810 2024-03-30T00:33:03.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.816+0000 7f7523a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c1a0b90 con 0x7f751c100c80 2024-03-30T00:33:03.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.816+0000 7f751be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f751c101e70 0x7f751c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:03.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.816+0000 7f751be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f751c101e70 0x7f751c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37288/0 (socket says 172.21.15.99:37288) 2024-03-30T00:33:03.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.816+0000 7f751be00700 1 -- 172.21.15.99:0/3855351142 learned_addr learned my addr 172.21.15.99:0/3855351142 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:03.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.816+0000 7f7521a00700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f751c102810 0x7f751c1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.816+0000 7f7521000700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f751c100c80 0x7f751c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.817+0000 7f751be00700 1 -- 172.21.15.99:0/3855351142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f751c100c80 msgr2=0x7f751c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.817+0000 7f751be00700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f751c100c80 0x7f751c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.817+0000 7f751be00700 1 -- 172.21.15.99:0/3855351142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f751c102810 msgr2=0x7f751c1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.817+0000 7f751be00700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f751c102810 0x7f751c1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.817+0000 7f751be00700 1 -- 172.21.15.99:0/3855351142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75100095f0 con 0x7f751c101e70 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.817+0000 7f751be00700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f751c101e70 0x7f751c1992d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f750800d950 tx=0x7f750800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.817+0000 7f7519600700 1 -- 172.21.15.99:0/3855351142 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508009b40 con 0x7f751c101e70 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.817+0000 7f7519600700 1 -- 172.21.15.99:0/3855351142 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7508009ca0 con 0x7f751c101e70 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.817+0000 7f7519600700 1 -- 172.21.15.99:0/3855351142 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750800f620 con 0x7f751c101e70 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.817+0000 7f7523a00700 1 -- 172.21.15.99:0/3855351142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751c1a0e70 con 0x7f751c101e70 2024-03-30T00:33:03.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.818+0000 7f7523a00700 1 -- 172.21.15.99:0/3855351142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751c1a12a0 con 0x7f751c101e70 2024-03-30T00:33:03.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.819+0000 7f7523a00700 1 -- 172.21.15.99:0/3855351142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751c067a10 con 0x7f751c101e70 2024-03-30T00:33:03.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.821+0000 7f7519600700 1 -- 172.21.15.99:0/3855351142 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7508010460 con 0x7f751c101e70 2024-03-30T00:33:03.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.823+0000 7f7519600700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75040a11a0 0x7f75040a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:03.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.823+0000 7f7519600700 1 -- 172.21.15.99:0/3855351142 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f75080cc180 con 0x7f751c101e70 2024-03-30T00:33:03.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.824+0000 7f7521000700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75040a11a0 0x7f75040a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:03.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.824+0000 7f7521000700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75040a11a0 0x7f75040a3660 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7510000c00 tx=0x7f7510005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:03.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:03.828+0000 7f7519600700 1 -- 172.21.15.99:0/3855351142 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7508095d10 con 0x7f751c101e70 2024-03-30T00:33:03.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:03 smithi161 ceph-mon[42221]: Deploying daemon mds.g on smithi099 2024-03-30T00:33:04.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.119+0000 7f7523a00700 1 -- 172.21.15.99:0/3855351142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f751c1a1760 con 0x7f751c101e70 2024-03-30T00:33:04.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.120+0000 7f7519600700 1 -- 172.21.15.99:0/3855351142 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7321 (secure 0 0 0) 0x7f7508099330 con 0x7f751c101e70 2024-03-30T00:33:04.128 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:04.128 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-03-30T00:33:04.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 -- 172.21.15.99:0/3855351142 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75040a11a0 msgr2=0x7f75040a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:04.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75040a11a0 0x7f75040a3660 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7510000c00 tx=0x7f7510005fb0 comp rx=0 tx=0).stop 2024-03-30T00:33:04.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 -- 172.21.15.99:0/3855351142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f751c101e70 msgr2=0x7f751c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:04.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f751c101e70 0x7f751c1992d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f750800d950 tx=0x7f750800dd10 comp rx=0 tx=0).stop 2024-03-30T00:33:04.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 -- 172.21.15.99:0/3855351142 shutdown_connections 2024-03-30T00:33:04.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75040a11a0 0x7f75040a3660 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:04.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f751c102810 0x7f751c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:04.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f751c100c80 0x7f751c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:04.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 --2- 172.21.15.99:0/3855351142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f751c101e70 0x7f751c1992d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:04.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 -- 172.21.15.99:0/3855351142 >> 172.21.15.99:0/3855351142 conn(0x7f751c0fc1f0 msgr2=0x7f751c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:04.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 -- 172.21.15.99:0/3855351142 shutdown_connections 2024-03-30T00:33:04.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:04.123+0000 7f7523a00700 1 -- 172.21.15.99:0/3855351142 wait complete. 2024-03-30T00:33:04.132 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 10 2024-03-30T00:33:04.147 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 30 00:33:03 smithi099 ceph-mds[87742]: -- 172.21.15.99:0/253095864 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5602b8dfe1a0 con 0x5602b8dd7000 2024-03-30T00:33:04.147 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 30 00:33:03 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-g[87715]: starting mds.g at 2024-03-30T00:33:04.147 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 30 00:33:03 smithi099 bash[87681]: 58255d9043ef81bf97ab6bf5dc3bbca7563da2083771e3b6e8b50f81f6200f3c 2024-03-30T00:33:04.147 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 30 00:33:03 smithi099 systemd[1]: Started Ceph mds.g for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:33:04.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:04 smithi142 ceph-mon[41779]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:04.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:04.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:04.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:04.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:33:04.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:33:04.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:04 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:04.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:04 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3855351142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:04.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:04 smithi099 ceph-mon[38079]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:04.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:04.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:04.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:04.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:33:04.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:33:04.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:04 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:04.884 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:04 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3855351142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:04.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:04 smithi161 ceph-mon[42221]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:04.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:04.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:04.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:04.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:33:04.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:33:04.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:04 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:04.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:04 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3855351142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:05.697 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:33:05 smithi099 systemd[1]: Starting Ceph mds.a for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:33:05.697 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:05 smithi099 ceph-mon[38079]: Deploying daemon mds.a on smithi099 2024-03-30T00:33:05.697 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:05 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.99:6838/757502672,v1:172.21.15.99:6839/757502672] up:boot 2024-03-30T00:33:05.697 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:05 smithi099 ceph-mon[38079]: fsmap 10 up:standby 2024-03-30T00:33:05.697 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:05 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T00:33:05.697 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:05 smithi099 ceph-mon[38079]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:05.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:05 smithi142 ceph-mon[41779]: Deploying daemon mds.a on smithi099 2024-03-30T00:33:05.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:05 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.99:6838/757502672,v1:172.21.15.99:6839/757502672] up:boot 2024-03-30T00:33:05.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:05 smithi142 ceph-mon[41779]: fsmap 10 up:standby 2024-03-30T00:33:05.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:05 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T00:33:05.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:05 smithi142 ceph-mon[41779]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:05 smithi161 ceph-mon[42221]: Deploying daemon mds.a on smithi099 2024-03-30T00:33:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:05 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.99:6838/757502672,v1:172.21.15.99:6839/757502672] up:boot 2024-03-30T00:33:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:05 smithi161 ceph-mon[42221]: fsmap 10 up:standby 2024-03-30T00:33:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:05 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T00:33:05.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:05 smithi161 ceph-mon[42221]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:06.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:33:06.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.352+0000 7f957c400700 1 -- 172.21.15.99:0/135909927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95741016e0 msgr2=0x7f9574103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:06.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.352+0000 7f957c400700 1 --2- 172.21.15.99:0/135909927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95741016e0 0x7f9574103ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9568009990 tx=0x7f9568009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:06.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.352+0000 7f957c400700 1 -- 172.21.15.99:0/135909927 shutdown_connections 2024-03-30T00:33:06.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.352+0000 7f957c400700 1 --2- 172.21.15.99:0/135909927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9574106940 0x7f9574108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:06.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.352+0000 7f957c400700 1 --2- 172.21.15.99:0/135909927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95741016e0 0x7f9574103ad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:06.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.352+0000 7f957c400700 1 --2- 172.21.15.99:0/135909927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9574104010 0x7f9574106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:06.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.352+0000 7f957c400700 1 -- 172.21.15.99:0/135909927 >> 172.21.15.99:0/135909927 conn(0x7f95740fb0d0 msgr2=0x7f95740fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:06.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.353+0000 7f957c400700 1 -- 172.21.15.99:0/135909927 shutdown_connections 2024-03-30T00:33:06.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.353+0000 7f957c400700 1 -- 172.21.15.99:0/135909927 wait complete. 2024-03-30T00:33:06.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.353+0000 7f957c400700 1 Processor -- start 2024-03-30T00:33:06.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f957c400700 1 -- start start 2024-03-30T00:33:06.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f957c400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95741016e0 0x7f9574198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:06.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f957c400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9574104010 0x7f9574199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:06.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f957c400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9574106940 0x7f95741a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f957c400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95741a0820 con 0x7f9574106940 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f957c400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95741a0990 con 0x7f9574104010 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f957c400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95741a0b50 con 0x7f95741016e0 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f957a400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9574106940 0x7f95741a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f957a400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9574106940 0x7f95741a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41750/0 (socket says 172.21.15.99:41750) 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f957a400700 1 -- 172.21.15.99:0/4048023139 learned_addr learned my addr 172.21.15.99:0/4048023139 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f9579000700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9574104010 0x7f9574199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.354+0000 7f9579a00700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95741016e0 0x7f9574198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.355+0000 7f957a400700 1 -- 172.21.15.99:0/4048023139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95741016e0 msgr2=0x7f9574198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.355+0000 7f957a400700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95741016e0 0x7f9574198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.355+0000 7f957a400700 1 -- 172.21.15.99:0/4048023139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9574104010 msgr2=0x7f9574199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.355+0000 7f957a400700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9574104010 0x7f9574199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.355+0000 7f957a400700 1 -- 172.21.15.99:0/4048023139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95680095f0 con 0x7f9574106940 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.355+0000 7f957a400700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9574106940 0x7f95741a0250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f956c00ef30 tx=0x7f956c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:06.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.355+0000 7f9572000700 1 -- 172.21.15.99:0/4048023139 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c018070 con 0x7f9574106940 2024-03-30T00:33:06.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.355+0000 7f9572000700 1 -- 172.21.15.99:0/4048023139 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f956c00f040 con 0x7f9574106940 2024-03-30T00:33:06.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.355+0000 7f957c400700 1 -- 172.21.15.99:0/4048023139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95741a0dd0 con 0x7f9574106940 2024-03-30T00:33:06.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.355+0000 7f9572000700 1 -- 172.21.15.99:0/4048023139 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c00cbc0 con 0x7f9574106940 2024-03-30T00:33:06.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.355+0000 7f957c400700 1 -- 172.21.15.99:0/4048023139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95741a1280 con 0x7f9574106940 2024-03-30T00:33:06.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.356+0000 7f957c400700 1 -- 172.21.15.99:0/4048023139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9574108510 con 0x7f9574106940 2024-03-30T00:33:06.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.358+0000 7f9572000700 1 -- 172.21.15.99:0/4048023139 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f956c00cd80 con 0x7f9574106940 2024-03-30T00:33:06.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.360+0000 7f9572000700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f955c0a1120 0x7f955c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:06.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.360+0000 7f9572000700 1 -- 172.21.15.99:0/4048023139 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f956c0cbef0 con 0x7f9574106940 2024-03-30T00:33:06.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.360+0000 7f9579a00700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f955c0a1120 0x7f955c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:06.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.361+0000 7f9579a00700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f955c0a1120 0x7f955c0a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9574100970 tx=0x7f956800b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:06.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.365+0000 7f9572000700 1 -- 172.21.15.99:0/4048023139 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f956c095b90 con 0x7f9574106940 2024-03-30T00:33:06.487 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:33:06 smithi099 bash[88070]: 92bf90815d975c40ebec02f0bfb46908eb240ff0edb399b86f3d38610475a363 2024-03-30T00:33:06.487 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:33:06 smithi099 ceph-mds[88109]: -- 172.21.15.99:0/217593857 <== mon.0 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55852820a1a0 con 0x5585281e2800 2024-03-30T00:33:06.487 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:33:06 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-a[88085]: starting mds.a at 2024-03-30T00:33:06.487 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:33:06 smithi099 systemd[1]: Started Ceph mds.a for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:33:06.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.616+0000 7f957c400700 1 -- 172.21.15.99:0/4048023139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9574067a10 con 0x7f9574106940 2024-03-30T00:33:06.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.616+0000 7f9572000700 1 -- 172.21.15.99:0/4048023139 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8076 (secure 0 0 0) 0x7f956c0991b0 con 0x7f9574106940 2024-03-30T00:33:06.624 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:06.624 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 -- 172.21.15.99:0/4048023139 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f955c0a1120 msgr2=0x7f955c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f955c0a1120 0x7f955c0a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9574100970 tx=0x7f956800b610 comp rx=0 tx=0).stop 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 -- 172.21.15.99:0/4048023139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9574106940 msgr2=0x7f95741a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9574106940 0x7f95741a0250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f956c00ef30 tx=0x7f956c00c3a0 comp rx=0 tx=0).stop 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 -- 172.21.15.99:0/4048023139 shutdown_connections 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9574106940 0x7f95741a0250 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95741016e0 0x7f9574198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f955c0a1120 0x7f955c0a35e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 --2- 172.21.15.99:0/4048023139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9574104010 0x7f9574199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 -- 172.21.15.99:0/4048023139 >> 172.21.15.99:0/4048023139 conn(0x7f95740fb0d0 msgr2=0x7f9574100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 -- 172.21.15.99:0/4048023139 shutdown_connections 2024-03-30T00:33:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:06.619+0000 7f957c400700 1 -- 172.21.15.99:0/4048023139 wait complete. 2024-03-30T00:33:06.628 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 11 2024-03-30T00:33:07.547 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:07 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:07.548 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:07 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:07.548 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:07 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:07.548 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:07 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:33:07.548 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:07 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:33:07.548 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:07 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:07.548 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:07 smithi142 ceph-mon[41779]: Deploying daemon mds.b on smithi142 2024-03-30T00:33:07.548 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:07 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4048023139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:07.548 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:07 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] up:boot 2024-03-30T00:33:07.548 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:07 smithi142 ceph-mon[41779]: fsmap 11 up:standby 2024-03-30T00:33:07.548 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:07 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T00:33:07.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:07 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:07.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:07 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:07.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:07 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:07.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:07 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:33:07.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:07 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:33:07.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:07 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:07.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:07 smithi161 ceph-mon[42221]: Deploying daemon mds.b on smithi142 2024-03-30T00:33:07.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:07 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4048023139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:07.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:07 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] up:boot 2024-03-30T00:33:07.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:07 smithi161 ceph-mon[42221]: fsmap 11 up:standby 2024-03-30T00:33:07.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:07 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T00:33:07.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:07 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:07.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:07 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:07.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:07 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:07.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:07 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T00:33:07.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:07 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T00:33:07.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:07 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:07.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:07 smithi099 ceph-mon[38079]: Deploying daemon mds.b on smithi142 2024-03-30T00:33:07.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:07 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4048023139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:07.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:07 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] up:boot 2024-03-30T00:33:07.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:07 smithi099 ceph-mon[38079]: fsmap 11 up:standby 2024-03-30T00:33:07.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:07 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T00:33:08.099 INFO:journalctl@ceph.mds.b.smithi142.stdout:Mar 30 00:33:08 smithi142 systemd[1]: Starting Ceph mds.b for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:33:08.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:08 smithi142 ceph-mon[41779]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:08.646 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:33:08.712 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:08 smithi099 ceph-mon[38079]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:08.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:08 smithi161 ceph-mon[42221]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:08.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.854+0000 7fc530a00700 1 -- 172.21.15.99:0/4197642537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c073790 msgr2=0x7fc52c073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:08.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.854+0000 7fc530a00700 1 --2- 172.21.15.99:0/4197642537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c073790 0x7fc52c073c10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc518009990 tx=0x7fc518009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:08.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.854+0000 7fc530a00700 1 -- 172.21.15.99:0/4197642537 shutdown_connections 2024-03-30T00:33:08.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.854+0000 7fc530a00700 1 --2- 172.21.15.99:0/4197642537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc52c108b70 0x7fc52c10b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:08.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.854+0000 7fc530a00700 1 --2- 172.21.15.99:0/4197642537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc52c074d70 0x7fc52c0731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:08.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.854+0000 7fc530a00700 1 --2- 172.21.15.99:0/4197642537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c073790 0x7fc52c073c10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:08.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.854+0000 7fc530a00700 1 -- 172.21.15.99:0/4197642537 >> 172.21.15.99:0/4197642537 conn(0x7fc52c0fc1f0 msgr2=0x7fc52c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:08.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.854+0000 7fc530a00700 1 -- 172.21.15.99:0/4197642537 shutdown_connections 2024-03-30T00:33:08.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.854+0000 7fc530a00700 1 -- 172.21.15.99:0/4197642537 wait complete. 2024-03-30T00:33:08.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.855+0000 7fc530a00700 1 Processor -- start 2024-03-30T00:33:08.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.855+0000 7fc530a00700 1 -- start start 2024-03-30T00:33:08.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.855+0000 7fc530a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc52c073790 0x7fc52c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.855+0000 7fc530a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c074d70 0x7fc52c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.855+0000 7fc530a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc52c108b70 0x7fc52c1a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.855+0000 7fc530a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1a0790 con 0x7fc52c074d70 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.855+0000 7fc530a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1a0900 con 0x7fc52c073790 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.855+0000 7fc530a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c1a0ac0 con 0x7fc52c108b70 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.855+0000 7fc529400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c074d70 0x7fc52c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc529400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c074d70 0x7fc52c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41772/0 (socket says 172.21.15.99:41772) 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc529400700 1 -- 172.21.15.99:0/3451092986 learned_addr learned my addr 172.21.15.99:0/3451092986 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc52a800700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc52c108b70 0x7fc52c1a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc529e00700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc52c073790 0x7fc52c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc529400700 1 -- 172.21.15.99:0/3451092986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc52c108b70 msgr2=0x7fc52c1a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc529400700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc52c108b70 0x7fc52c1a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:08.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc529400700 1 -- 172.21.15.99:0/3451092986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc52c073790 msgr2=0x7fc52c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:08.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc529400700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc52c073790 0x7fc52c198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:08.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc529400700 1 -- 172.21.15.99:0/3451092986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5180095f0 con 0x7fc52c074d70 2024-03-30T00:33:08.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc529e00700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc52c073790 0x7fc52c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:08.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc529400700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c074d70 0x7fc52c1991e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5180098e0 tx=0x7fc518005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:08.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc522a00700 1 -- 172.21.15.99:0/3451092986 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51801d070 con 0x7fc52c074d70 2024-03-30T00:33:08.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc522a00700 1 -- 172.21.15.99:0/3451092986 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc518004000 con 0x7fc52c074d70 2024-03-30T00:33:08.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc522a00700 1 -- 172.21.15.99:0/3451092986 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518022ca0 con 0x7fc52c074d70 2024-03-30T00:33:08.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.856+0000 7fc530a00700 1 -- 172.21.15.99:0/3451092986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c1a0d40 con 0x7fc52c074d70 2024-03-30T00:33:08.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.857+0000 7fc530a00700 1 -- 172.21.15.99:0/3451092986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c1a1110 con 0x7fc52c074d70 2024-03-30T00:33:08.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.858+0000 7fc530a00700 1 -- 172.21.15.99:0/3451092986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52c107980 con 0x7fc52c074d70 2024-03-30T00:33:08.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.859+0000 7fc522a00700 1 -- 172.21.15.99:0/3451092986 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc518022470 con 0x7fc52c074d70 2024-03-30T00:33:08.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.861+0000 7fc522a00700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc5100a1120 0x7fc5100a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:08.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.861+0000 7fc522a00700 1 -- 172.21.15.99:0/3451092986 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc5180cda10 con 0x7fc52c074d70 2024-03-30T00:33:08.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.861+0000 7fc529e00700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc5100a1120 0x7fc5100a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:08.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.861+0000 7fc529e00700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc5100a1120 0x7fc5100a35e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc52c19a120 tx=0x7fc51c00a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:08.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:08.865+0000 7fc522a00700 1 -- 172.21.15.99:0/3451092986 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc518097630 con 0x7fc52c074d70 2024-03-30T00:33:09.067 INFO:journalctl@ceph.mds.b.smithi142.stdout:Mar 30 00:33:08 smithi142 ceph-mds[77920]: -- 172.21.15.142:0/785553327 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b36dec1a0 con 0x559b36dc4800 2024-03-30T00:33:09.067 INFO:journalctl@ceph.mds.b.smithi142.stdout:Mar 30 00:33:08 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-b[77897]: starting mds.b at 2024-03-30T00:33:09.067 INFO:journalctl@ceph.mds.b.smithi142.stdout:Mar 30 00:33:08 smithi142 bash[77882]: 9c254cab379e81d4d851e2027cd61b11c394b9d6faf72440b939d5adc9b0f888 2024-03-30T00:33:09.067 INFO:journalctl@ceph.mds.b.smithi142.stdout:Mar 30 00:33:08 smithi142 systemd[1]: Started Ceph mds.b for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:33:09.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.126+0000 7fc530a00700 1 -- 172.21.15.99:0/3451092986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc52c067a10 con 0x7fc52c074d70 2024-03-30T00:33:09.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.126+0000 7fc522a00700 1 -- 172.21.15.99:0/3451092986 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8834 (secure 0 0 0) 0x7fc51809ac50 con 0x7fc52c074d70 2024-03-30T00:33:09.133 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:09.133 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[]} 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 -- 172.21.15.99:0/3451092986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc5100a1120 msgr2=0x7fc5100a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc5100a1120 0x7fc5100a35e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc52c19a120 tx=0x7fc51c00a680 comp rx=0 tx=0).stop 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 -- 172.21.15.99:0/3451092986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c074d70 msgr2=0x7fc52c1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c074d70 0x7fc52c1991e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5180098e0 tx=0x7fc518005f10 comp rx=0 tx=0).stop 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 -- 172.21.15.99:0/3451092986 shutdown_connections 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc52c108b70 0x7fc52c1a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc5100a1120 0x7fc5100a35e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc52c073790 0x7fc52c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 --2- 172.21.15.99:0/3451092986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc52c074d70 0x7fc52c1991e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 -- 172.21.15.99:0/3451092986 >> 172.21.15.99:0/3451092986 conn(0x7fc52c0fc1f0 msgr2=0x7fc52c104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 -- 172.21.15.99:0/3451092986 shutdown_connections 2024-03-30T00:33:09.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:09.129+0000 7fc530a00700 1 -- 172.21.15.99:0/3451092986 wait complete. 2024-03-30T00:33:09.137 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 12 2024-03-30T00:33:09.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:09.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:33:09.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3451092986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:09.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] up:boot 2024-03-30T00:33:09.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: fsmap 12 up:standby 2024-03-30T00:33:09.970 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3451092986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] up:boot 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: fsmap 12 up:standby 2024-03-30T00:33:09.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T00:33:10.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:10.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:10.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:10.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:10.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:10.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:10.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:10.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:33:10.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3451092986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:10.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] up:boot 2024-03-30T00:33:10.067 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: fsmap 12 up:standby 2024-03-30T00:33:10.068 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T00:33:11.157 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:33:11.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.348+0000 7f8dad800700 1 -- 172.21.15.99:0/1390861816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da8073810 msgr2=0x7f8da8073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:11.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.348+0000 7f8dad800700 1 --2- 172.21.15.99:0/1390861816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da8073810 0x7f8da8073c90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009990 tx=0x7f8d94009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.348+0000 7f8dad800700 1 -- 172.21.15.99:0/1390861816 shutdown_connections 2024-03-30T00:33:11.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.348+0000 7f8dad800700 1 --2- 172.21.15.99:0/1390861816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da810d1e0 0x7f8da810f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.348+0000 7f8dad800700 1 --2- 172.21.15.99:0/1390861816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8da8074df0 0x7f8da8073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.348+0000 7f8dad800700 1 --2- 172.21.15.99:0/1390861816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da8073810 0x7f8da8073c90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.348+0000 7f8dad800700 1 -- 172.21.15.99:0/1390861816 >> 172.21.15.99:0/1390861816 conn(0x7f8da80fc160 msgr2=0x7f8da80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:11.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.348+0000 7f8dad800700 1 -- 172.21.15.99:0/1390861816 shutdown_connections 2024-03-30T00:33:11.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.348+0000 7f8dad800700 1 -- 172.21.15.99:0/1390861816 wait complete. 2024-03-30T00:33:11.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.350+0000 7f8dad800700 1 Processor -- start 2024-03-30T00:33:11.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.350+0000 7f8dad800700 1 -- start start 2024-03-30T00:33:11.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.350+0000 7f8dad800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da8073810 0x7f8da819d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.350+0000 7f8dad800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8da8074df0 0x7f8da819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.350+0000 7f8dad800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da810d1e0 0x7f8da81a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.350+0000 7f8dad800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da81a4c10 con 0x7f8da8073810 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.350+0000 7f8dad800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da81a4d50 con 0x7f8da810d1e0 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.350+0000 7f8dad800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da81a4ee0 con 0x7f8da8074df0 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.351+0000 7f8da6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da8073810 0x7f8da819d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.351+0000 7f8da6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da8073810 0x7f8da819d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41786/0 (socket says 172.21.15.99:41786) 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.351+0000 7f8da6a00700 1 -- 172.21.15.99:0/110371651 learned_addr learned my addr 172.21.15.99:0/110371651 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.351+0000 7f8da6000700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8da8074df0 0x7f8da819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.351+0000 7f8da7400700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da810d1e0 0x7f8da81a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.351+0000 7f8da6a00700 1 -- 172.21.15.99:0/110371651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8da8074df0 msgr2=0x7f8da819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.351+0000 7f8da6a00700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8da8074df0 0x7f8da819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.351+0000 7f8da6a00700 1 -- 172.21.15.99:0/110371651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da810d1e0 msgr2=0x7f8da81a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:11.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.351+0000 7f8da6a00700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da810d1e0 0x7f8da81a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.351+0000 7f8da6a00700 1 -- 172.21.15.99:0/110371651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d940095f0 con 0x7f8da8073810 2024-03-30T00:33:11.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.351+0000 7f8da6a00700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da8073810 0x7f8da819d130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800e9a0 tx=0x7f8d9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:11.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.352+0000 7f8d9f400700 1 -- 172.21.15.99:0/110371651 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98009790 con 0x7f8da8073810 2024-03-30T00:33:11.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.352+0000 7f8d9f400700 1 -- 172.21.15.99:0/110371651 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d98010430 con 0x7f8da8073810 2024-03-30T00:33:11.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.352+0000 7f8dad800700 1 -- 172.21.15.99:0/110371651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da81a51c0 con 0x7f8da8073810 2024-03-30T00:33:11.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.352+0000 7f8d9f400700 1 -- 172.21.15.99:0/110371651 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9800cea0 con 0x7f8da8073810 2024-03-30T00:33:11.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.352+0000 7f8dad800700 1 -- 172.21.15.99:0/110371651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da81a5720 con 0x7f8da8073810 2024-03-30T00:33:11.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.353+0000 7f8dad800700 1 -- 172.21.15.99:0/110371651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8103440 con 0x7f8da8073810 2024-03-30T00:33:11.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.355+0000 7f8d9f400700 1 -- 172.21.15.99:0/110371651 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f8d980105a0 con 0x7f8da8073810 2024-03-30T00:33:11.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.357+0000 7f8d9f400700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d8c0a11a0 0x7f8d8c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:11.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.357+0000 7f8d9f400700 1 -- 172.21.15.99:0/110371651 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8d98014070 con 0x7f8da8073810 2024-03-30T00:33:11.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.357+0000 7f8da6000700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d8c0a11a0 0x7f8d8c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:11.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.358+0000 7f8da6000700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d8c0a11a0 0x7f8d8c0a3660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8d94005910 tx=0x7f8d9400b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:11.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.361+0000 7f8d9f400700 1 -- 172.21.15.99:0/110371651 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8d980961c0 con 0x7f8da8073810 2024-03-30T00:33:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.642+0000 7f8dad800700 1 -- 172.21.15.99:0/110371651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8da8112bb0 con 0x7f8da8073810 2024-03-30T00:33:11.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.643+0000 7f8d9f400700 1 -- 172.21.15.99:0/110371651 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9595 (secure 0 0 0) 0x7f8d980997e0 con 0x7f8da8073810 2024-03-30T00:33:11.650 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:11.650 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24473,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24503,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24572,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 -- 172.21.15.99:0/110371651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d8c0a11a0 msgr2=0x7f8d8c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d8c0a11a0 0x7f8d8c0a3660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8d94005910 tx=0x7f8d9400b610 comp rx=0 tx=0).stop 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 -- 172.21.15.99:0/110371651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da8073810 msgr2=0x7f8da819d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da8073810 0x7f8da819d130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800e9a0 tx=0x7f8d9800ecb0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 -- 172.21.15.99:0/110371651 shutdown_connections 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d8c0a11a0 0x7f8d8c0a3660 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8da810d1e0 0x7f8da81a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8da8073810 0x7f8da819d130 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 --2- 172.21.15.99:0/110371651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8da8074df0 0x7f8da819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 -- 172.21.15.99:0/110371651 >> 172.21.15.99:0/110371651 conn(0x7f8da80fc160 msgr2=0x7f8da8100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 -- 172.21.15.99:0/110371651 shutdown_connections 2024-03-30T00:33:11.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.646+0000 7f8dad800700 1 -- 172.21.15.99:0/110371651 wait complete. 2024-03-30T00:33:11.654 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 13 2024-03-30T00:33:11.673 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:33:11.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.861+0000 7f2117a00700 1 -- 172.21.15.99:0/1402638005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2110102810 msgr2=0x7f21101090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:11.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.861+0000 7f2117a00700 1 --2- 172.21.15.99:0/1402638005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2110102810 0x7f21101090c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f210800b0d0 tx=0x7f210800b3e0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.861+0000 7f2117a00700 1 -- 172.21.15.99:0/1402638005 shutdown_connections 2024-03-30T00:33:11.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.861+0000 7f2117a00700 1 --2- 172.21.15.99:0/1402638005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2110102810 0x7f21101090c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.861+0000 7f2117a00700 1 --2- 172.21.15.99:0/1402638005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2110100c80 0x7f21101010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.861+0000 7f2117a00700 1 --2- 172.21.15.99:0/1402638005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2110101e70 0x7f21101022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.861+0000 7f2117a00700 1 -- 172.21.15.99:0/1402638005 >> 172.21.15.99:0/1402638005 conn(0x7f21100fc1f0 msgr2=0x7f21100fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:11.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.861+0000 7f2117a00700 1 -- 172.21.15.99:0/1402638005 shutdown_connections 2024-03-30T00:33:11.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.861+0000 7f2117a00700 1 -- 172.21.15.99:0/1402638005 wait complete. 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.862+0000 7f2117a00700 1 Processor -- start 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.862+0000 7f2117a00700 1 -- start start 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2117a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2110100c80 0x7f2110198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2117a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2110101e70 0x7f2110199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2117a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2110102810 0x7f21101a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2117a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21101a0970 con 0x7f2110100c80 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2117a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21101a0ab0 con 0x7f2110101e70 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2117a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21101a0c40 con 0x7f2110102810 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2115000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2110100c80 0x7f2110198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2115000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2110100c80 0x7f2110198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41796/0 (socket says 172.21.15.99:41796) 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2115000700 1 -- 172.21.15.99:0/289846739 learned_addr learned my addr 172.21.15.99:0/289846739 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2115a00700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2110102810 0x7f21101a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:11.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f210fe00700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2110101e70 0x7f2110199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:11.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2115000700 1 -- 172.21.15.99:0/289846739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2110102810 msgr2=0x7f21101a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:11.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2115000700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2110102810 0x7f21101a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2115000700 1 -- 172.21.15.99:0/289846739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2110101e70 msgr2=0x7f2110199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:11.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2115000700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2110101e70 0x7f2110199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:11.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f2115000700 1 -- 172.21.15.99:0/289846739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2108009d00 con 0x7f2110100c80 2024-03-30T00:33:11.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.863+0000 7f210fe00700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2110101e70 0x7f2110199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:11.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.864+0000 7f2115000700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2110100c80 0x7f2110198e40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f210400e970 tx=0x7f210400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:11.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.864+0000 7f210d600700 1 -- 172.21.15.99:0/289846739 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104009790 con 0x7f2110100c80 2024-03-30T00:33:11.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.864+0000 7f210d600700 1 -- 172.21.15.99:0/289846739 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f210400eec0 con 0x7f2110100c80 2024-03-30T00:33:11.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.864+0000 7f2117a00700 1 -- 172.21.15.99:0/289846739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21101a0f20 con 0x7f2110100c80 2024-03-30T00:33:11.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.864+0000 7f210d600700 1 -- 172.21.15.99:0/289846739 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104003c10 con 0x7f2110100c80 2024-03-30T00:33:11.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.864+0000 7f2117a00700 1 -- 172.21.15.99:0/289846739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21101a1340 con 0x7f2110100c80 2024-03-30T00:33:11.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.865+0000 7f2117a00700 1 -- 172.21.15.99:0/289846739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2110067a80 con 0x7f2110100c80 2024-03-30T00:33:11.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.867+0000 7f210d600700 1 -- 172.21.15.99:0/289846739 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f2104003db0 con 0x7f2110100c80 2024-03-30T00:33:11.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.868+0000 7f210d600700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f20f80a1120 0x7f20f80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:11.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.868+0000 7f210d600700 1 -- 172.21.15.99:0/289846739 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2104014070 con 0x7f2110100c80 2024-03-30T00:33:11.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.868+0000 7f210fe00700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f20f80a1120 0x7f20f80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:11.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.869+0000 7f210fe00700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f20f80a1120 0x7f20f80a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009cb0 tx=0x7f20fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:11.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:11.872+0000 7f210d600700 1 -- 172.21.15.99:0/289846739 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2110067a80 con 0x7f2110100c80 2024-03-30T00:33:11.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:11.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:11.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:11.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:11.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:11.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:11.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:11 smithi099 ceph-mon[38079]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:11.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:11.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:33:11.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:11 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:11.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:11 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/110371651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:12.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.134+0000 7f2117a00700 1 -- 172.21.15.99:0/289846739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f211010c350 con 0x7f2110100c80 2024-03-30T00:33:12.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.135+0000 7f210d600700 1 -- 172.21.15.99:0/289846739 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18423 (secure 0 0 0) 0x7f2104096310 con 0x7f2110100c80 2024-03-30T00:33:12.142 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:12.142 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":79,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:32:33.039895+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:33:12.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.137+0000 7f2117a00700 1 -- 172.21.15.99:0/289846739 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f20f80a1120 msgr2=0x7f20f80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:12.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.137+0000 7f2117a00700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f20f80a1120 0x7f20f80a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009cb0 tx=0x7f20fc009260 comp rx=0 tx=0).stop 2024-03-30T00:33:12.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.137+0000 7f2117a00700 1 -- 172.21.15.99:0/289846739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2110100c80 msgr2=0x7f2110198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:12.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.137+0000 7f2117a00700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2110100c80 0x7f2110198e40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f210400e970 tx=0x7f210400ec80 comp rx=0 tx=0).stop 2024-03-30T00:33:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.137+0000 7f2117a00700 1 -- 172.21.15.99:0/289846739 shutdown_connections 2024-03-30T00:33:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.137+0000 7f2117a00700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2110102810 0x7f21101a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.137+0000 7f2117a00700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2110100c80 0x7f2110198e40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.137+0000 7f2117a00700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f20f80a1120 0x7f20f80a35e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.137+0000 7f2117a00700 1 --2- 172.21.15.99:0/289846739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2110101e70 0x7f2110199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.137+0000 7f2117a00700 1 -- 172.21.15.99:0/289846739 >> 172.21.15.99:0/289846739 conn(0x7f21100fc1f0 msgr2=0x7f2110072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.137+0000 7f2117a00700 1 -- 172.21.15.99:0/289846739 shutdown_connections 2024-03-30T00:33:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.138+0000 7f2117a00700 1 -- 172.21.15.99:0/289846739 wait complete. 2024-03-30T00:33:12.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:33:12.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:33:12.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:11 smithi161 ceph-mon[42221]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:12.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:12.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:33:12.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:11 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.220 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:11 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/110371651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:12.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:11 smithi142 ceph-mon[41779]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:12.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:12.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:33:12.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:11 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:12.318 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:11 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/110371651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:12.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.367+0000 7f5f0e400700 1 -- 172.21.15.99:0/529068579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f08108c50 msgr2=0x7f5f0810b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:12.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.367+0000 7f5f0e400700 1 --2- 172.21.15.99:0/529068579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f08108c50 0x7f5f0810b0c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009f90 tx=0x7f5f0000b330 comp rx=0 tx=0).stop 2024-03-30T00:33:12.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.367+0000 7f5f0e400700 1 -- 172.21.15.99:0/529068579 shutdown_connections 2024-03-30T00:33:12.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.367+0000 7f5f0e400700 1 --2- 172.21.15.99:0/529068579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f08108c50 0x7f5f0810b0c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.367+0000 7f5f0e400700 1 --2- 172.21.15.99:0/529068579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f08074d90 0x7f5f080731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.367+0000 7f5f0e400700 1 --2- 172.21.15.99:0/529068579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f080737b0 0x7f5f08073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.367+0000 7f5f0e400700 1 -- 172.21.15.99:0/529068579 >> 172.21.15.99:0/529068579 conn(0x7f5f080fc2f0 msgr2=0x7f5f080fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:12.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.368+0000 7f5f0e400700 1 -- 172.21.15.99:0/529068579 shutdown_connections 2024-03-30T00:33:12.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.368+0000 7f5f0e400700 1 -- 172.21.15.99:0/529068579 wait complete. 2024-03-30T00:33:12.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f0e400700 1 Processor -- start 2024-03-30T00:33:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f0e400700 1 -- start start 2024-03-30T00:33:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f0e400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f080737b0 0x7f5f08198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f0e400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f08074d90 0x7f5f081994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f0e400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f08108c50 0x7f5f081a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f0e400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081a0aa0 con 0x7f5f08108c50 2024-03-30T00:33:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f0e400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081a0be0 con 0x7f5f08074d90 2024-03-30T00:33:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f0e400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f081a0d70 con 0x7f5f080737b0 2024-03-30T00:33:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f07e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f08108c50 0x7f5f081a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f07400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f080737b0 0x7f5f08198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f07e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f08108c50 0x7f5f081a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41814/0 (socket says 172.21.15.99:41814) 2024-03-30T00:33:12.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f07e00700 1 -- 172.21.15.99:0/1508918166 learned_addr learned my addr 172.21.15.99:0/1508918166 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.369+0000 7f5f06a00700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f08074d90 0x7f5f081994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.370+0000 7f5f07e00700 1 -- 172.21.15.99:0/1508918166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f080737b0 msgr2=0x7f5f08198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.370+0000 7f5f07e00700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f080737b0 0x7f5f08198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.370+0000 7f5f07e00700 1 -- 172.21.15.99:0/1508918166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f08074d90 msgr2=0x7f5f081994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.370+0000 7f5f07e00700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f08074d90 0x7f5f081994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.370+0000 7f5f07e00700 1 -- 172.21.15.99:0/1508918166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f00009c40 con 0x7f5f08108c50 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.370+0000 7f5f07e00700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f08108c50 0x7f5f081a04d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009f60 tx=0x7f5f00008c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.370+0000 7f5effe00700 1 -- 172.21.15.99:0/1508918166 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000e040 con 0x7f5f08108c50 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.370+0000 7f5effe00700 1 -- 172.21.15.99:0/1508918166 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f0000bc00 con 0x7f5f08108c50 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.370+0000 7f5f0e400700 1 -- 172.21.15.99:0/1508918166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f081a0ff0 con 0x7f5f08108c50 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.370+0000 7f5effe00700 1 -- 172.21.15.99:0/1508918166 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00005040 con 0x7f5f08108c50 2024-03-30T00:33:12.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.370+0000 7f5f0e400700 1 -- 172.21.15.99:0/1508918166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f081a13c0 con 0x7f5f08108c50 2024-03-30T00:33:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.372+0000 7f5efd600700 1 -- 172.21.15.99:0/1508918166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee00052f0 con 0x7f5f08108c50 2024-03-30T00:33:12.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.373+0000 7f5effe00700 1 -- 172.21.15.99:0/1508918166 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f5f0001d470 con 0x7f5f08108c50 2024-03-30T00:33:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.374+0000 7f5effe00700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5eec0a1120 0x7f5eec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:12.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.375+0000 7f5effe00700 1 -- 172.21.15.99:0/1508918166 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5f000cd9e0 con 0x7f5f08108c50 2024-03-30T00:33:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.375+0000 7f5f07400700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5eec0a1120 0x7f5eec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:12.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.375+0000 7f5f07400700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5eec0a1120 0x7f5eec0a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8005fd0 tx=0x7f5ef8009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:12.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.378+0000 7f5effe00700 1 -- 172.21.15.99:0/1508918166 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5f00097600 con 0x7f5f08108c50 2024-03-30T00:33:12.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.602+0000 7f5efd600700 1 -- 172.21.15.99:0/1508918166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ee0005b20 con 0x7f5f08108c50 2024-03-30T00:33:12.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.603+0000 7f5effe00700 1 -- 172.21.15.99:0/1508918166 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f0009ac20 con 0x7f5f08108c50 2024-03-30T00:33:12.610 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:33:12.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.605+0000 7f5efd600700 1 -- 172.21.15.99:0/1508918166 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5eec0a1120 msgr2=0x7f5eec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:12.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.605+0000 7f5efd600700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5eec0a1120 0x7f5eec0a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8005fd0 tx=0x7f5ef8009310 comp rx=0 tx=0).stop 2024-03-30T00:33:12.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.605+0000 7f5efd600700 1 -- 172.21.15.99:0/1508918166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f08108c50 msgr2=0x7f5f081a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:12.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.605+0000 7f5efd600700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f08108c50 0x7f5f081a04d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009f60 tx=0x7f5f00008c80 comp rx=0 tx=0).stop 2024-03-30T00:33:12.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.605+0000 7f5efd600700 1 -- 172.21.15.99:0/1508918166 shutdown_connections 2024-03-30T00:33:12.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.605+0000 7f5efd600700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f08108c50 0x7f5f081a04d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.605+0000 7f5efd600700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f080737b0 0x7f5f08198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.605+0000 7f5efd600700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5eec0a1120 0x7f5eec0a35e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.605+0000 7f5efd600700 1 --2- 172.21.15.99:0/1508918166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f08074d90 0x7f5f081994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.605+0000 7f5efd600700 1 -- 172.21.15.99:0/1508918166 >> 172.21.15.99:0/1508918166 conn(0x7f5f080fc2f0 msgr2=0x7f5f08104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:12.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.606+0000 7f5efd600700 1 -- 172.21.15.99:0/1508918166 shutdown_connections 2024-03-30T00:33:12.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.606+0000 7f5efd600700 1 -- 172.21.15.99:0/1508918166 wait complete. 2024-03-30T00:33:12.631 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T00:33:12.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.828+0000 7f1d87200700 1 -- 172.21.15.99:0/109499185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d80074df0 msgr2=0x7f1d80073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:12.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.828+0000 7f1d87200700 1 --2- 172.21.15.99:0/109499185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d80074df0 0x7f1d80073240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009990 tx=0x7f1d70009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.828+0000 7f1d87200700 1 -- 172.21.15.99:0/109499185 shutdown_connections 2024-03-30T00:33:12.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.828+0000 7f1d87200700 1 --2- 172.21.15.99:0/109499185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d8010d250 0x7f1d8010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.828+0000 7f1d87200700 1 --2- 172.21.15.99:0/109499185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d80074df0 0x7f1d80073240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.828+0000 7f1d87200700 1 --2- 172.21.15.99:0/109499185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d80073810 0x7f1d80073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.828+0000 7f1d87200700 1 -- 172.21.15.99:0/109499185 >> 172.21.15.99:0/109499185 conn(0x7f1d800fc1f0 msgr2=0x7f1d800fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:12.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.829+0000 7f1d87200700 1 -- 172.21.15.99:0/109499185 shutdown_connections 2024-03-30T00:33:12.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.829+0000 7f1d87200700 1 -- 172.21.15.99:0/109499185 wait complete. 2024-03-30T00:33:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d87200700 1 Processor -- start 2024-03-30T00:33:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d87200700 1 -- start start 2024-03-30T00:33:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d87200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d80073810 0x7f1d8019d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d87200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d80074df0 0x7f1d8019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d87200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d8010d250 0x7f1d801a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d87200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d801a4c30 con 0x7f1d8010d250 2024-03-30T00:33:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d87200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d801a4d70 con 0x7f1d80073810 2024-03-30T00:33:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d87200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d801a4f00 con 0x7f1d80074df0 2024-03-30T00:33:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d7fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d80073810 0x7f1d8019d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d7fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d80073810 0x7f1d8019d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:55420/0 (socket says 172.21.15.99:55420) 2024-03-30T00:33:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d7fe00700 1 -- 172.21.15.99:0/823654561 learned_addr learned my addr 172.21.15.99:0/823654561 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:12.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.830+0000 7f1d7f400700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d80074df0 0x7f1d8019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:12.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.831+0000 7f1d85200700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d8010d250 0x7f1d801a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:12.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.831+0000 7f1d7fe00700 1 -- 172.21.15.99:0/823654561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d80074df0 msgr2=0x7f1d8019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:12.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.831+0000 7f1d7fe00700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d80074df0 0x7f1d8019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.831+0000 7f1d7fe00700 1 -- 172.21.15.99:0/823654561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d8010d250 msgr2=0x7f1d801a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:12.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.831+0000 7f1d7fe00700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d8010d250 0x7f1d801a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:12.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.831+0000 7f1d7fe00700 1 -- 172.21.15.99:0/823654561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c009520 con 0x7f1d80073810 2024-03-30T00:33:12.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.831+0000 7f1d7fe00700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d80073810 0x7f1d8019d100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d70005770 tx=0x7f1d700057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:12.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.831+0000 7f1d7cc00700 1 -- 172.21.15.99:0/823654561 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7001d070 con 0x7f1d80073810 2024-03-30T00:33:12.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.831+0000 7f1d87200700 1 -- 172.21.15.99:0/823654561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d700095f0 con 0x7f1d80073810 2024-03-30T00:33:12.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.831+0000 7f1d7cc00700 1 -- 172.21.15.99:0/823654561 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d7000f460 con 0x7f1d80073810 2024-03-30T00:33:12.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.831+0000 7f1d7cc00700 1 -- 172.21.15.99:0/823654561 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70021670 con 0x7f1d80073810 2024-03-30T00:33:12.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.832+0000 7f1d87200700 1 -- 172.21.15.99:0/823654561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d801a5470 con 0x7f1d80073810 2024-03-30T00:33:12.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.833+0000 7f1d87200700 1 -- 172.21.15.99:0/823654561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d801034b0 con 0x7f1d80073810 2024-03-30T00:33:12.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.834+0000 7f1d7cc00700 1 -- 172.21.15.99:0/823654561 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1d7000f7d0 con 0x7f1d80073810 2024-03-30T00:33:12.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.836+0000 7f1d7cc00700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1d640a1120 0x7f1d640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:12.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.836+0000 7f1d7cc00700 1 -- 172.21.15.99:0/823654561 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1d700cd7b0 con 0x7f1d80073810 2024-03-30T00:33:12.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.836+0000 7f1d7f400700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1d640a1120 0x7f1d640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:12.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.837+0000 7f1d7f400700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1d640a1120 0x7f1d640a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1d80074b10 tx=0x7f1d6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:12.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:12.839+0000 7f1d7cc00700 1 -- 172.21.15.99:0/823654561 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1d700974f0 con 0x7f1d80073810 2024-03-30T00:33:12.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:12 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:33:12.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:12 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/289846739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:12.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:12 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1508918166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:33:13.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.100+0000 7f1d87200700 1 -- 172.21.15.99:0/823654561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1d80112920 con 0x7f1d80073810 2024-03-30T00:33:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.100+0000 7f1d7cc00700 1 -- 172.21.15.99:0/823654561 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1d70026060 con 0x7f1d80073810 2024-03-30T00:33:13.107 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:13.107 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-03-30T00:33:13.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:33:13.108 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-03-30T00:33:13.108 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-03-30T00:33:13.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:33:13.108 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-03-30T00:33:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 -- 172.21.15.99:0/823654561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1d640a1120 msgr2=0x7f1d640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1d640a1120 0x7f1d640a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1d80074b10 tx=0x7f1d6c009260 comp rx=0 tx=0).stop 2024-03-30T00:33:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 -- 172.21.15.99:0/823654561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d80073810 msgr2=0x7f1d8019d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d80073810 0x7f1d8019d100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d70005770 tx=0x7f1d700057a0 comp rx=0 tx=0).stop 2024-03-30T00:33:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 -- 172.21.15.99:0/823654561 shutdown_connections 2024-03-30T00:33:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d8010d250 0x7f1d801a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1d640a1120 0x7f1d640a35e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d80073810 0x7f1d8019d100 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:13.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 --2- 172.21.15.99:0/823654561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d80074df0 0x7f1d8019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:13.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 -- 172.21.15.99:0/823654561 >> 172.21.15.99:0/823654561 conn(0x7f1d800fc1f0 msgr2=0x7f1d801005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:13.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 -- 172.21.15.99:0/823654561 shutdown_connections 2024-03-30T00:33:13.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.103+0000 7f1d87200700 1 -- 172.21.15.99:0/823654561 wait complete. 2024-03-30T00:33:13.130 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T00:33:13.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T00:33:13.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:12 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:33:13.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:12 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/289846739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:13.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:12 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1508918166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:33:13.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:12 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:33:13.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:12 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/289846739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:13.317 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:12 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1508918166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:33:13.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.344+0000 7f9fb5400700 1 -- 172.21.15.99:0/86111711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fb0074df0 msgr2=0x7f9fb0073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:13.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.344+0000 7f9fb5400700 1 --2- 172.21.15.99:0/86111711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fb0074df0 0x7f9fb0073240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009990 tx=0x7f9fa0009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:13.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.344+0000 7f9fb5400700 1 -- 172.21.15.99:0/86111711 shutdown_connections 2024-03-30T00:33:13.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.344+0000 7f9fb5400700 1 --2- 172.21.15.99:0/86111711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb0108c40 0x7f9fb010b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:13.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.344+0000 7f9fb5400700 1 --2- 172.21.15.99:0/86111711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fb0074df0 0x7f9fb0073240 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:13.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.344+0000 7f9fb5400700 1 --2- 172.21.15.99:0/86111711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fb0073810 0x7f9fb0073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:13.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.344+0000 7f9fb5400700 1 -- 172.21.15.99:0/86111711 >> 172.21.15.99:0/86111711 conn(0x7f9fb00fc2f0 msgr2=0x7f9fb00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:13.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.344+0000 7f9fb5400700 1 -- 172.21.15.99:0/86111711 shutdown_connections 2024-03-30T00:33:13.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.344+0000 7f9fb5400700 1 -- 172.21.15.99:0/86111711 wait complete. 2024-03-30T00:33:13.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fb5400700 1 Processor -- start 2024-03-30T00:33:13.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fb5400700 1 -- start start 2024-03-30T00:33:13.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fb5400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fb0073810 0x7f9fb0198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:13.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fb5400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fb0074df0 0x7f9fb0199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:13.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fb5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb0108c40 0x7f9fb01a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:13.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fb5400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb01a0930 con 0x7f9fb0074df0 2024-03-30T00:33:13.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fb5400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb01a0a70 con 0x7f9fb0108c40 2024-03-30T00:33:13.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fb5400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb01a0c00 con 0x7f9fb0073810 2024-03-30T00:33:13.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fade00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fb0074df0 0x7f9fb0199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:13.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fade00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fb0074df0 0x7f9fb0199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41840/0 (socket says 172.21.15.99:41840) 2024-03-30T00:33:13.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fade00700 1 -- 172.21.15.99:0/2962056990 learned_addr learned my addr 172.21.15.99:0/2962056990 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:13.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.346+0000 7f9fae800700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fb0073810 0x7f9fb0198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:13.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.347+0000 7f9faf200700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb0108c40 0x7f9fb01a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:13.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.347+0000 7f9fae800700 1 -- 172.21.15.99:0/2962056990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb0108c40 msgr2=0x7f9fb01a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:13.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.347+0000 7f9fae800700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb0108c40 0x7f9fb01a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:13.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.347+0000 7f9fae800700 1 -- 172.21.15.99:0/2962056990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fb0074df0 msgr2=0x7f9fb0199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:13.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.347+0000 7f9fae800700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fb0074df0 0x7f9fb0199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:13.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.347+0000 7f9fae800700 1 -- 172.21.15.99:0/2962056990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa00095f0 con 0x7f9fb0073810 2024-03-30T00:33:13.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.347+0000 7f9fade00700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fb0074df0 0x7f9fb0199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:33:13.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.347+0000 7f9fae800700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fb0073810 0x7f9fb0198e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00098e0 tx=0x7f9fa0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:13.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.348+0000 7f9fa7400700 1 -- 172.21.15.99:0/2962056990 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa001d070 con 0x7f9fb0073810 2024-03-30T00:33:13.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.348+0000 7f9fa7400700 1 -- 172.21.15.99:0/2962056990 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fa000bdb0 con 0x7f9fb0073810 2024-03-30T00:33:13.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.348+0000 7f9fa7400700 1 -- 172.21.15.99:0/2962056990 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0022cc0 con 0x7f9fb0073810 2024-03-30T00:33:13.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.348+0000 7f9fb5400700 1 -- 172.21.15.99:0/2962056990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb01a0e80 con 0x7f9fb0073810 2024-03-30T00:33:13.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.348+0000 7f9fb5400700 1 -- 172.21.15.99:0/2962056990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb01a11f0 con 0x7f9fb0073810 2024-03-30T00:33:13.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.349+0000 7f9fb5400700 1 -- 172.21.15.99:0/2962056990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb0107a00 con 0x7f9fb0073810 2024-03-30T00:33:13.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.351+0000 7f9fa7400700 1 -- 172.21.15.99:0/2962056990 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f9fa0022470 con 0x7f9fb0073810 2024-03-30T00:33:13.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.352+0000 7f9fa7400700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9f940a11a0 0x7f9f940a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:13.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.352+0000 7f9fa7400700 1 -- 172.21.15.99:0/2962056990 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9fa00cde60 con 0x7f9fb0073810 2024-03-30T00:33:13.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.352+0000 7f9fade00700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9f940a11a0 0x7f9f940a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:13.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.353+0000 7f9fade00700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9f940a11a0 0x7f9f940a3660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c005950 tx=0x7f9f9c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:13.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.357+0000 7f9fa7400700 1 -- 172.21.15.99:0/2962056990 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9fa0097a80 con 0x7f9fb0073810 2024-03-30T00:33:13.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:13.583+0000 7f9fb5400700 1 -- 172.21.15.99:0/2962056990 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9fb0067af0 con 0x7f9fb0073810 2024-03-30T00:33:14.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.140+0000 7f9fa7400700 1 -- 172.21.15.99:0/2962056990 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9fa009b0a0 con 0x7f9fb0073810 2024-03-30T00:33:14.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.144+0000 7f9fb5400700 1 -- 172.21.15.99:0/2962056990 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9f940a11a0 msgr2=0x7f9f940a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:14.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.144+0000 7f9fb5400700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9f940a11a0 0x7f9f940a3660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c005950 tx=0x7f9f9c009260 comp rx=0 tx=0).stop 2024-03-30T00:33:14.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.144+0000 7f9fb5400700 1 -- 172.21.15.99:0/2962056990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fb0073810 msgr2=0x7f9fb0198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:14.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.144+0000 7f9fb5400700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fb0073810 0x7f9fb0198e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00098e0 tx=0x7f9fa0004730 comp rx=0 tx=0).stop 2024-03-30T00:33:14.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.144+0000 7f9fb5400700 1 -- 172.21.15.99:0/2962056990 shutdown_connections 2024-03-30T00:33:14.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.144+0000 7f9fb5400700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9f940a11a0 0x7f9f940a3660 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:14.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.144+0000 7f9fb5400700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9fb0108c40 0x7f9fb01a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:14.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.144+0000 7f9fb5400700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fb0073810 0x7f9fb0198e00 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00098e0 tx=0x7f9fa0004730 comp rx=0 tx=0).stop 2024-03-30T00:33:14.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.144+0000 7f9fb5400700 1 --2- 172.21.15.99:0/2962056990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fb0074df0 0x7f9fb0199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:14.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.144+0000 7f9fb5400700 1 -- 172.21.15.99:0/2962056990 >> 172.21.15.99:0/2962056990 conn(0x7f9fb00fc2f0 msgr2=0x7f9fb0104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:14.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.145+0000 7f9fb5400700 1 -- 172.21.15.99:0/2962056990 shutdown_connections 2024-03-30T00:33:14.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.146+0000 7f9fb5400700 1 -- 172.21.15.99:0/2962056990 wait complete. 2024-03-30T00:33:14.155 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-03-30T00:33:14.174 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T00:33:14.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.371+0000 7f399fa00700 1 -- 172.21.15.99:0/1825098092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3998108c40 msgr2=0x7f399810b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:14.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.371+0000 7f399fa00700 1 --2- 172.21.15.99:0/1825098092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3998108c40 0x7f399810b0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f399000b0a0 tx=0x7f399000b3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:14.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.372+0000 7f399fa00700 1 -- 172.21.15.99:0/1825098092 shutdown_connections 2024-03-30T00:33:14.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.372+0000 7f399fa00700 1 --2- 172.21.15.99:0/1825098092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3998108c40 0x7f399810b0d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:14.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.372+0000 7f399fa00700 1 --2- 172.21.15.99:0/1825098092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3998074df0 0x7f3998073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:14.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.372+0000 7f399fa00700 1 --2- 172.21.15.99:0/1825098092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3998073810 0x7f3998073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:14.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.372+0000 7f399fa00700 1 -- 172.21.15.99:0/1825098092 >> 172.21.15.99:0/1825098092 conn(0x7f39980fc2f0 msgr2=0x7f39980fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:14.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.372+0000 7f399fa00700 1 -- 172.21.15.99:0/1825098092 shutdown_connections 2024-03-30T00:33:14.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.372+0000 7f399fa00700 1 -- 172.21.15.99:0/1825098092 wait complete. 2024-03-30T00:33:14.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.373+0000 7f399fa00700 1 Processor -- start 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.373+0000 7f399fa00700 1 -- start start 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.373+0000 7f399fa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3998073810 0x7f3998198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.373+0000 7f399fa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3998074df0 0x7f3998199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.373+0000 7f399fa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3998108c40 0x7f39981a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.373+0000 7f399fa00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39981a0a70 con 0x7f3998073810 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.373+0000 7f399fa00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39981a0bb0 con 0x7f3998074df0 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.373+0000 7f399fa00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39981a0d40 con 0x7f3998108c40 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.373+0000 7f399d000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3998073810 0x7f3998198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.373+0000 7f399d000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3998073810 0x7f3998198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41846/0 (socket says 172.21.15.99:41846) 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.373+0000 7f399d000700 1 -- 172.21.15.99:0/3337110441 learned_addr learned my addr 172.21.15.99:0/3337110441 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f3997e00700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3998074df0 0x7f3998199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f399da00700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3998108c40 0x7f39981a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:14.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f399d000700 1 -- 172.21.15.99:0/3337110441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3998108c40 msgr2=0x7f39981a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:14.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f399d000700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3998108c40 0x7f39981a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:14.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f399d000700 1 -- 172.21.15.99:0/3337110441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3998074df0 msgr2=0x7f3998199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:14.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f399d000700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3998074df0 0x7f3998199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:14.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f399d000700 1 -- 172.21.15.99:0/3337110441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3990009d00 con 0x7f3998073810 2024-03-30T00:33:14.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f399d000700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3998073810 0x7f3998198f40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f398c00e9a0 tx=0x7f398c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:14.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f3995600700 1 -- 172.21.15.99:0/3337110441 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c009790 con 0x7f3998073810 2024-03-30T00:33:14.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f3995600700 1 -- 172.21.15.99:0/3337110441 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f398c010430 con 0x7f3998073810 2024-03-30T00:33:14.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f3995600700 1 -- 172.21.15.99:0/3337110441 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c00cea0 con 0x7f3998073810 2024-03-30T00:33:14.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.374+0000 7f399fa00700 1 -- 172.21.15.99:0/3337110441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39981a0fc0 con 0x7f3998073810 2024-03-30T00:33:14.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.375+0000 7f399fa00700 1 -- 172.21.15.99:0/3337110441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39981a1390 con 0x7f3998073810 2024-03-30T00:33:14.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.375+0000 7f399fa00700 1 -- 172.21.15.99:0/3337110441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3998107a00 con 0x7f3998073810 2024-03-30T00:33:14.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.378+0000 7f3995600700 1 -- 172.21.15.99:0/3337110441 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f398c0105a0 con 0x7f3998073810 2024-03-30T00:33:14.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.379+0000 7f3995600700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f39800a1120 0x7f39800a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:14.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.379+0000 7f3995600700 1 -- 172.21.15.99:0/3337110441 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f398c014070 con 0x7f3998073810 2024-03-30T00:33:14.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.379+0000 7f3997e00700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f39800a1120 0x7f39800a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:14.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.380+0000 7f3997e00700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f39800a1120 0x7f39800a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3998074b10 tx=0x7f3984009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:14.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.382+0000 7f3995600700 1 -- 172.21.15.99:0/3337110441 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f398c0963e0 con 0x7f3998073810 2024-03-30T00:33:14.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:14 smithi161 ceph-mon[42221]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:14.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:14 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/823654561' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-30T00:33:14.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:14 smithi099 ceph-mon[38079]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:14.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:14 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/823654561' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-30T00:33:14.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:14 smithi142 ceph-mon[41779]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:14.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:14 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/823654561' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-30T00:33:14.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:14.592+0000 7f399fa00700 1 -- 172.21.15.99:0/3337110441 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3998067af0 con 0x7f3998073810 2024-03-30T00:33:15.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.149+0000 7f3995600700 1 -- 172.21.15.99:0/3337110441 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f398c099a00 con 0x7f3998073810 2024-03-30T00:33:15.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.156+0000 7f399fa00700 1 -- 172.21.15.99:0/3337110441 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f39800a1120 msgr2=0x7f39800a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:15.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.156+0000 7f399fa00700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f39800a1120 0x7f39800a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3998074b10 tx=0x7f3984009260 comp rx=0 tx=0).stop 2024-03-30T00:33:15.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.156+0000 7f399fa00700 1 -- 172.21.15.99:0/3337110441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3998073810 msgr2=0x7f3998198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:15.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.156+0000 7f399fa00700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3998073810 0x7f3998198f40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f398c00e9a0 tx=0x7f398c00ecb0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.157+0000 7f399fa00700 1 -- 172.21.15.99:0/3337110441 shutdown_connections 2024-03-30T00:33:15.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.157+0000 7f399fa00700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3998108c40 0x7f39981a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.157+0000 7f399fa00700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3998073810 0x7f3998198f40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.157+0000 7f399fa00700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f39800a1120 0x7f39800a35e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.157+0000 7f399fa00700 1 --2- 172.21.15.99:0/3337110441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3998074df0 0x7f3998199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.157+0000 7f399fa00700 1 -- 172.21.15.99:0/3337110441 >> 172.21.15.99:0/3337110441 conn(0x7f39980fc2f0 msgr2=0x7f3998104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.158+0000 7f399fa00700 1 -- 172.21.15.99:0/3337110441 shutdown_connections 2024-03-30T00:33:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.158+0000 7f399fa00700 1 -- 172.21.15.99:0/3337110441 wait complete. 2024-03-30T00:33:15.170 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-03-30T00:33:15.196 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T00:33:15.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.386+0000 7f8d48c00700 1 -- 172.21.15.99:0/1027718512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d44101f70 msgr2=0x7f8d441023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:15.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.386+0000 7f8d48c00700 1 --2- 172.21.15.99:0/1027718512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d44101f70 0x7f8d441023d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d300099c0 tx=0x7f8d30009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.386+0000 7f8d48c00700 1 -- 172.21.15.99:0/1027718512 shutdown_connections 2024-03-30T00:33:15.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.386+0000 7f8d48c00700 1 --2- 172.21.15.99:0/1027718512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d44102910 0x7f8d441091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.386+0000 7f8d48c00700 1 --2- 172.21.15.99:0/1027718512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d44100d80 0x7f8d441011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.386+0000 7f8d48c00700 1 --2- 172.21.15.99:0/1027718512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d44101f70 0x7f8d441023d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.386+0000 7f8d48c00700 1 -- 172.21.15.99:0/1027718512 >> 172.21.15.99:0/1027718512 conn(0x7f8d440fc2f0 msgr2=0x7f8d440fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:15.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.386+0000 7f8d48c00700 1 -- 172.21.15.99:0/1027718512 shutdown_connections 2024-03-30T00:33:15.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.387+0000 7f8d48c00700 1 -- 172.21.15.99:0/1027718512 wait complete. 2024-03-30T00:33:15.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.387+0000 7f8d48c00700 1 Processor -- start 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d48c00700 1 -- start start 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d48c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d44100d80 0x7f8d44196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d48c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d44101f70 0x7f8d44197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d48c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d44102910 0x7f8d4419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d48c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4419e770 con 0x7f8d44102910 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d48c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4419e8b0 con 0x7f8d44101f70 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d48c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4419ea40 con 0x7f8d44100d80 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d41e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d44100d80 0x7f8d44196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d42800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d44102910 0x7f8d4419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d41e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d44100d80 0x7f8d44196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:45760/0 (socket says 172.21.15.99:45760) 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d41e00700 1 -- 172.21.15.99:0/3404002592 learned_addr learned my addr 172.21.15.99:0/3404002592 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.388+0000 7f8d41400700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d44101f70 0x7f8d44197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d41e00700 1 -- 172.21.15.99:0/3404002592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d44101f70 msgr2=0x7f8d44197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d41e00700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d44101f70 0x7f8d44197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d41e00700 1 -- 172.21.15.99:0/3404002592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d44102910 msgr2=0x7f8d4419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d41e00700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d44102910 0x7f8d4419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d41e00700 1 -- 172.21.15.99:0/3404002592 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300095f0 con 0x7f8d44100d80 2024-03-30T00:33:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d42800700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d44102910 0x7f8d4419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:33:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d41e00700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d44100d80 0x7f8d44196c40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d980 tx=0x7f8d3400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d3aa00700 1 -- 172.21.15.99:0/3404002592 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34004020 con 0x7f8d44100d80 2024-03-30T00:33:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d48c00700 1 -- 172.21.15.99:0/3404002592 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4419ed20 con 0x7f8d44100d80 2024-03-30T00:33:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d3aa00700 1 -- 172.21.15.99:0/3404002592 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d3400f690 con 0x7f8d44100d80 2024-03-30T00:33:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d3aa00700 1 -- 172.21.15.99:0/3404002592 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d340039e0 con 0x7f8d44100d80 2024-03-30T00:33:15.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.389+0000 7f8d48c00700 1 -- 172.21.15.99:0/3404002592 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4419f180 con 0x7f8d44100d80 2024-03-30T00:33:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.390+0000 7f8d48c00700 1 -- 172.21.15.99:0/3404002592 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d44108250 con 0x7f8d44100d80 2024-03-30T00:33:15.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.392+0000 7f8d3aa00700 1 -- 172.21.15.99:0/3404002592 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f8d3400f800 con 0x7f8d44100d80 2024-03-30T00:33:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.393+0000 7f8d3aa00700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d280a1120 0x7f8d280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.393+0000 7f8d3aa00700 1 -- 172.21.15.99:0/3404002592 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f8d340cc380 con 0x7f8d44100d80 2024-03-30T00:33:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.394+0000 7f8d41400700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d280a1120 0x7f8d280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.394+0000 7f8d41400700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d280a1120 0x7f8d280a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000b580 tx=0x7f8d30005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:15.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.397+0000 7f8d3aa00700 1 -- 172.21.15.99:0/3404002592 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8d34095c20 con 0x7f8d44100d80 2024-03-30T00:33:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2962056990' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T00:33:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T00:33:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:33:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-30T00:33:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: osdmap e80: 12 total, 12 up, 12 in 2024-03-30T00:33:15.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:15 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3337110441' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-30T00:33:15.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2962056990' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T00:33:15.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T00:33:15.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:15.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:33:15.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-30T00:33:15.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: osdmap e80: 12 total, 12 up, 12 in 2024-03-30T00:33:15.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:15 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3337110441' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-30T00:33:15.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2962056990' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T00:33:15.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T00:33:15.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:33:15.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:33:15.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:33:15.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-30T00:33:15.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: osdmap e80: 12 total, 12 up, 12 in 2024-03-30T00:33:15.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:15 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3337110441' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-30T00:33:15.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.646+0000 7f8d48c00700 1 -- 172.21.15.99:0/3404002592 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8d44067af0 con 0x7f8d44100d80 2024-03-30T00:33:15.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.656+0000 7f8d3aa00700 1 -- 172.21.15.99:0/3404002592 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8d34099240 con 0x7f8d44100d80 2024-03-30T00:33:15.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.660+0000 7f8d48c00700 1 -- 172.21.15.99:0/3404002592 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d280a1120 msgr2=0x7f8d280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:15.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.660+0000 7f8d48c00700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d280a1120 0x7f8d280a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000b580 tx=0x7f8d30005e80 comp rx=0 tx=0).stop 2024-03-30T00:33:15.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.660+0000 7f8d48c00700 1 -- 172.21.15.99:0/3404002592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d44100d80 msgr2=0x7f8d44196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:15.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.660+0000 7f8d48c00700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d44100d80 0x7f8d44196c40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d980 tx=0x7f8d3400dc90 comp rx=0 tx=0).stop 2024-03-30T00:33:15.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.661+0000 7f8d48c00700 1 -- 172.21.15.99:0/3404002592 shutdown_connections 2024-03-30T00:33:15.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.661+0000 7f8d48c00700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d44102910 0x7f8d4419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.661+0000 7f8d48c00700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d44100d80 0x7f8d44196c40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.661+0000 7f8d48c00700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d280a1120 0x7f8d280a35e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.661+0000 7f8d48c00700 1 --2- 172.21.15.99:0/3404002592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d44101f70 0x7f8d44197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.661+0000 7f8d48c00700 1 -- 172.21.15.99:0/3404002592 >> 172.21.15.99:0/3404002592 conn(0x7f8d440fc2f0 msgr2=0x7f8d44105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:15.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.661+0000 7f8d48c00700 1 -- 172.21.15.99:0/3404002592 shutdown_connections 2024-03-30T00:33:15.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.661+0000 7f8d48c00700 1 -- 172.21.15.99:0/3404002592 wait complete. 2024-03-30T00:33:15.672 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 2 and data pool 3 2024-03-30T00:33:15.701 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:33:15.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.885+0000 7f1de7c00700 1 -- 172.21.15.99:0/4091200220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de0100c80 msgr2=0x7f1de01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:15.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.885+0000 7f1de7c00700 1 --2- 172.21.15.99:0/4091200220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de0100c80 0x7f1de01010a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40099c0 tx=0x7f1dd4009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.885+0000 7f1de7c00700 1 -- 172.21.15.99:0/4091200220 shutdown_connections 2024-03-30T00:33:15.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.885+0000 7f1de7c00700 1 --2- 172.21.15.99:0/4091200220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de0102810 0x7f1de01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.885+0000 7f1de7c00700 1 --2- 172.21.15.99:0/4091200220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de0100c80 0x7f1de01010a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.885+0000 7f1de7c00700 1 --2- 172.21.15.99:0/4091200220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1de0101e70 0x7f1de01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.885+0000 7f1de7c00700 1 -- 172.21.15.99:0/4091200220 >> 172.21.15.99:0/4091200220 conn(0x7f1de00fc1f0 msgr2=0x7f1de00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:15.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.885+0000 7f1de7c00700 1 -- 172.21.15.99:0/4091200220 shutdown_connections 2024-03-30T00:33:15.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.885+0000 7f1de7c00700 1 -- 172.21.15.99:0/4091200220 wait complete. 2024-03-30T00:33:15.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.886+0000 7f1de7c00700 1 Processor -- start 2024-03-30T00:33:15.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.886+0000 7f1de7c00700 1 -- start start 2024-03-30T00:33:15.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de7c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1de0100c80 0x7f1de0074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:15.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de7c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de0101e70 0x7f1de00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:15.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de7c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de0102810 0x7f1de00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:15.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de7c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0073de0 con 0x7f1de0101e70 2024-03-30T00:33:15.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de7c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0073f50 con 0x7f1de0102810 2024-03-30T00:33:15.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de7c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de01a0c50 con 0x7f1de0100c80 2024-03-30T00:33:15.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1ddfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de0101e70 0x7f1de00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:15.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de5200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1de0100c80 0x7f1de0074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:15.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1ddfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de0101e70 0x7f1de00731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35122/0 (socket says 172.21.15.99:35122) 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1ddfe00700 1 -- 172.21.15.99:0/709156996 learned_addr learned my addr 172.21.15.99:0/709156996 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de5c00700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de0102810 0x7f1de00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de5200700 1 -- 172.21.15.99:0/709156996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de0102810 msgr2=0x7f1de00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de5200700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de0102810 0x7f1de00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de5200700 1 -- 172.21.15.99:0/709156996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de0101e70 msgr2=0x7f1de00731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de5200700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de0101e70 0x7f1de00731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de5200700 1 -- 172.21.15.99:0/709156996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd40095f0 con 0x7f1de0100c80 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1ddfe00700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de0101e70 0x7f1de00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.887+0000 7f1de5c00700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de0102810 0x7f1de00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.888+0000 7f1de5200700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1de0100c80 0x7f1de0074ac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400b580 tx=0x7f1dd4003a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.888+0000 7f1ddd600700 1 -- 172.21.15.99:0/709156996 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd401d070 con 0x7f1de0100c80 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.888+0000 7f1ddd600700 1 -- 172.21.15.99:0/709156996 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dd400bda0 con 0x7f1de0100c80 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.888+0000 7f1de7c00700 1 -- 172.21.15.99:0/709156996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de01a0e50 con 0x7f1de0100c80 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.888+0000 7f1ddd600700 1 -- 172.21.15.99:0/709156996 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4022cb0 con 0x7f1de0100c80 2024-03-30T00:33:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.888+0000 7f1de7c00700 1 -- 172.21.15.99:0/709156996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de01a1210 con 0x7f1de0100c80 2024-03-30T00:33:15.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.889+0000 7f1de7c00700 1 -- 172.21.15.99:0/709156996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de01081f0 con 0x7f1de0100c80 2024-03-30T00:33:15.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.891+0000 7f1ddd600700 1 -- 172.21.15.99:0/709156996 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1dd4022470 con 0x7f1de0100c80 2024-03-30T00:33:15.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.892+0000 7f1ddd600700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1dc80a11a0 0x7f1dc80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:15.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.892+0000 7f1ddd600700 1 -- 172.21.15.99:0/709156996 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1dd40cdc70 con 0x7f1de0100c80 2024-03-30T00:33:15.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.892+0000 7f1ddfe00700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1dc80a11a0 0x7f1dc80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:15.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.893+0000 7f1ddfe00700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1dc80a11a0 0x7f1dc80a3660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc005950 tx=0x7f1dcc00b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:15.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:15.895+0000 7f1ddd600700 1 -- 172.21.15.99:0/709156996 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1dd4097540 con 0x7f1de0100c80 2024-03-30T00:33:16.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.132+0000 7f1de7c00700 1 -- 172.21.15.99:0/709156996 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1de0067a10 con 0x7f1de0100c80 2024-03-30T00:33:16.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.135+0000 7f1ddd600700 1 -- 172.21.15.99:0/709156996 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22133 (secure 0 0 0) 0x7f1dd409ab60 con 0x7f1de0100c80 2024-03-30T00:33:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:16.142 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":82,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:33:15.648503+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:33:16.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 -- 172.21.15.99:0/709156996 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1dc80a11a0 msgr2=0x7f1dc80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:16.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1dc80a11a0 0x7f1dc80a3660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc005950 tx=0x7f1dcc00b500 comp rx=0 tx=0).stop 2024-03-30T00:33:16.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 -- 172.21.15.99:0/709156996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1de0100c80 msgr2=0x7f1de0074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:16.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1de0100c80 0x7f1de0074ac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400b580 tx=0x7f1dd4003a50 comp rx=0 tx=0).stop 2024-03-30T00:33:16.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 -- 172.21.15.99:0/709156996 shutdown_connections 2024-03-30T00:33:16.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1dc80a11a0 0x7f1dc80a3660 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1de0102810 0x7f1de00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1de0100c80 0x7f1de0074ac0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 --2- 172.21.15.99:0/709156996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1de0101e70 0x7f1de00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 -- 172.21.15.99:0/709156996 >> 172.21.15.99:0/709156996 conn(0x7f1de00fc1f0 msgr2=0x7f1de0105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:16.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 -- 172.21.15.99:0/709156996 shutdown_connections 2024-03-30T00:33:16.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.137+0000 7f1de7c00700 1 -- 172.21.15.99:0/709156996 wait complete. 2024-03-30T00:33:16.163 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:33:16.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.348+0000 7fe97ae00700 1 -- 172.21.15.99:0/3699851847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe974101e70 msgr2=0x7fe9741022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:16.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.348+0000 7fe97ae00700 1 --2- 172.21.15.99:0/3699851847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe974101e70 0x7fe9741022d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe9600099c0 tx=0x7fe960009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.348+0000 7fe97ae00700 1 -- 172.21.15.99:0/3699851847 shutdown_connections 2024-03-30T00:33:16.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.348+0000 7fe97ae00700 1 --2- 172.21.15.99:0/3699851847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe974102810 0x7fe9741090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.348+0000 7fe97ae00700 1 --2- 172.21.15.99:0/3699851847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe974100c80 0x7fe9741010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.348+0000 7fe97ae00700 1 --2- 172.21.15.99:0/3699851847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe974101e70 0x7fe9741022d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.348+0000 7fe97ae00700 1 -- 172.21.15.99:0/3699851847 >> 172.21.15.99:0/3699851847 conn(0x7fe9740fc1f0 msgr2=0x7fe9740fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:16.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.348+0000 7fe97ae00700 1 -- 172.21.15.99:0/3699851847 shutdown_connections 2024-03-30T00:33:16.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.348+0000 7fe97ae00700 1 -- 172.21.15.99:0/3699851847 wait complete. 2024-03-30T00:33:16.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.349+0000 7fe97ae00700 1 Processor -- start 2024-03-30T00:33:16.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.349+0000 7fe97ae00700 1 -- start start 2024-03-30T00:33:16.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.349+0000 7fe97ae00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe974100c80 0x7fe974198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:16.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.349+0000 7fe97ae00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe974101e70 0x7fe974199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:16.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.349+0000 7fe97ae00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe974102810 0x7fe9741a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:16.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.349+0000 7fe97ae00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9741a0820 con 0x7fe974101e70 2024-03-30T00:33:16.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.349+0000 7fe97ae00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9741a0990 con 0x7fe974100c80 2024-03-30T00:33:16.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.349+0000 7fe97ae00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9741a0b50 con 0x7fe974102810 2024-03-30T00:33:16.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.349+0000 7fe973400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe974101e70 0x7fe974199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.349+0000 7fe973400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe974101e70 0x7fe974199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35134/0 (socket says 172.21.15.99:35134) 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe973400700 1 -- 172.21.15.99:0/4088599472 learned_addr learned my addr 172.21.15.99:0/4088599472 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe973e00700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe974100c80 0x7fe974198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe973400700 1 -- 172.21.15.99:0/4088599472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe974102810 msgr2=0x7fe9741a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe973400700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe974102810 0x7fe9741a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe973400700 1 -- 172.21.15.99:0/4088599472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe974100c80 msgr2=0x7fe974198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe973400700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe974100c80 0x7fe974198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe973400700 1 -- 172.21.15.99:0/4088599472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600095f0 con 0x7fe974101e70 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe973400700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe974101e70 0x7fe974199240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe960003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe970c00700 1 -- 172.21.15.99:0/4088599472 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96001d070 con 0x7fe974101e70 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe970c00700 1 -- 172.21.15.99:0/4088599472 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe96000bd30 con 0x7fe974101e70 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe97ae00700 1 -- 172.21.15.99:0/4088599472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9741a0dd0 con 0x7fe974101e70 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.350+0000 7fe970c00700 1 -- 172.21.15.99:0/4088599472 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960022d20 con 0x7fe974101e70 2024-03-30T00:33:16.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.351+0000 7fe97ae00700 1 -- 172.21.15.99:0/4088599472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9741a11a0 con 0x7fe974101e70 2024-03-30T00:33:16.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.351+0000 7fe97ae00700 1 -- 172.21.15.99:0/4088599472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe974067a10 con 0x7fe974101e70 2024-03-30T00:33:16.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.353+0000 7fe970c00700 1 -- 172.21.15.99:0/4088599472 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fe960003f40 con 0x7fe974101e70 2024-03-30T00:33:16.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.355+0000 7fe970c00700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9580a1120 0x7fe9580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:16.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.356+0000 7fe970c00700 1 -- 172.21.15.99:0/4088599472 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe9600ce4e0 con 0x7fe974101e70 2024-03-30T00:33:16.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.356+0000 7fe973e00700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9580a1120 0x7fe9580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:16.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.356+0000 7fe973e00700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9580a1120 0x7fe9580a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe9740fe5c0 tx=0x7fe96400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:16.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.358+0000 7fe970c00700 1 -- 172.21.15.99:0/4088599472 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe960097db0 con 0x7fe974101e70 2024-03-30T00:33:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:16 smithi161 ceph-mon[42221]: pgmap v253: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:16 smithi161 ceph-mon[42221]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-30T00:33:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:16 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3337110441' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-30T00:33:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:16 smithi161 ceph-mon[42221]: osdmap e81: 12 total, 12 up, 12 in 2024-03-30T00:33:16.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:16 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3404002592' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-30T00:33:16.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:16 smithi161 ceph-mon[42221]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-30T00:33:16.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:16 smithi161 ceph-mon[42221]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-30T00:33:16.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:16 smithi161 ceph-mon[42221]: osdmap e82: 12 total, 12 up, 12 in 2024-03-30T00:33:16.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:16 smithi161 ceph-mon[42221]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-30T00:33:16.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:16 smithi161 ceph-mon[42221]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-30T00:33:16.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:16 smithi161 ceph-mon[42221]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-30T00:33:16.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:16 smithi099 ceph-mon[38079]: pgmap v253: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:16.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:16 smithi099 ceph-mon[38079]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-30T00:33:16.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:16 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3337110441' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-30T00:33:16.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:16 smithi099 ceph-mon[38079]: osdmap e81: 12 total, 12 up, 12 in 2024-03-30T00:33:16.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:16 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3404002592' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-30T00:33:16.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:16 smithi099 ceph-mon[38079]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-30T00:33:16.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:16 smithi099 ceph-mon[38079]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-30T00:33:16.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:16 smithi099 ceph-mon[38079]: osdmap e82: 12 total, 12 up, 12 in 2024-03-30T00:33:16.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:16 smithi099 ceph-mon[38079]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-30T00:33:16.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:16 smithi099 ceph-mon[38079]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-30T00:33:16.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:16 smithi099 ceph-mon[38079]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-30T00:33:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:16 smithi142 ceph-mon[41779]: pgmap v253: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T00:33:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:16 smithi142 ceph-mon[41779]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-30T00:33:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:16 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3337110441' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-30T00:33:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:16 smithi142 ceph-mon[41779]: osdmap e81: 12 total, 12 up, 12 in 2024-03-30T00:33:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:16 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3404002592' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-30T00:33:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:16 smithi142 ceph-mon[41779]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-30T00:33:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:16 smithi142 ceph-mon[41779]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-30T00:33:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:16 smithi142 ceph-mon[41779]: osdmap e82: 12 total, 12 up, 12 in 2024-03-30T00:33:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:16 smithi142 ceph-mon[41779]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-30T00:33:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:16 smithi142 ceph-mon[41779]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-30T00:33:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:16 smithi142 ceph-mon[41779]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-30T00:33:16.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.601+0000 7fe97ae00700 1 -- 172.21.15.99:0/4088599472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe97410c390 con 0x7fe974101e70 2024-03-30T00:33:16.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.602+0000 7fe970c00700 1 -- 172.21.15.99:0/4088599472 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22133 (secure 0 0 0) 0x7fe96009b3d0 con 0x7fe974101e70 2024-03-30T00:33:16.609 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:16.610 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":82,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:33:15.648503+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 -- 172.21.15.99:0/4088599472 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9580a1120 msgr2=0x7fe9580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9580a1120 0x7fe9580a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe9740fe5c0 tx=0x7fe96400b410 comp rx=0 tx=0).stop 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 -- 172.21.15.99:0/4088599472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe974101e70 msgr2=0x7fe974199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe974101e70 0x7fe974199240 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe960003a50 comp rx=0 tx=0).stop 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 -- 172.21.15.99:0/4088599472 shutdown_connections 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe974102810 0x7fe9741a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9580a1120 0x7fe9580a35e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe974100c80 0x7fe974198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 --2- 172.21.15.99:0/4088599472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe974101e70 0x7fe974199240 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 -- 172.21.15.99:0/4088599472 >> 172.21.15.99:0/4088599472 conn(0x7fe9740fc1f0 msgr2=0x7fe974072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 -- 172.21.15.99:0/4088599472 shutdown_connections 2024-03-30T00:33:16.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.605+0000 7fe97ae00700 1 -- 172.21.15.99:0/4088599472 wait complete. 2024-03-30T00:33:16.630 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T00:33:16.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.818+0000 7f8d91200700 1 -- 172.21.15.99:0/4116256587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c100d80 msgr2=0x7f8d8c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:16.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.818+0000 7f8d91200700 1 --2- 172.21.15.99:0/4116256587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c100d80 0x7f8d8c1011a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0099c0 tx=0x7f8d7c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.818+0000 7f8d91200700 1 -- 172.21.15.99:0/4116256587 shutdown_connections 2024-03-30T00:33:16.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.818+0000 7f8d91200700 1 --2- 172.21.15.99:0/4116256587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d8c102910 0x7f8d8c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.818+0000 7f8d91200700 1 --2- 172.21.15.99:0/4116256587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c100d80 0x7f8d8c1011a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.818+0000 7f8d91200700 1 --2- 172.21.15.99:0/4116256587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d8c101f70 0x7f8d8c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.818+0000 7f8d91200700 1 -- 172.21.15.99:0/4116256587 >> 172.21.15.99:0/4116256587 conn(0x7f8d8c0fc2f0 msgr2=0x7f8d8c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:16.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.818+0000 7f8d91200700 1 -- 172.21.15.99:0/4116256587 shutdown_connections 2024-03-30T00:33:16.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.819+0000 7f8d91200700 1 -- 172.21.15.99:0/4116256587 wait complete. 2024-03-30T00:33:16.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.819+0000 7f8d91200700 1 Processor -- start 2024-03-30T00:33:16.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d91200700 1 -- start start 2024-03-30T00:33:16.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d91200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d8c100d80 0x7f8d8c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d91200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d8c101f70 0x7f8d8c1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d91200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c102910 0x7f8d8c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d91200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c1a09b0 con 0x7f8d8c102910 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d91200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c1a0af0 con 0x7f8d8c100d80 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d91200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c1a0c80 con 0x7f8d8c101f70 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d8a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c102910 0x7f8d8c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d8a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c102910 0x7f8d8c1a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35146/0 (socket says 172.21.15.99:35146) 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d8a800700 1 -- 172.21.15.99:0/4293697269 learned_addr learned my addr 172.21.15.99:0/4293697269 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d89400700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d8c101f70 0x7f8d8c1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.820+0000 7f8d89e00700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d8c100d80 0x7f8d8c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.821+0000 7f8d8a800700 1 -- 172.21.15.99:0/4293697269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d8c101f70 msgr2=0x7f8d8c1993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.821+0000 7f8d8a800700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d8c101f70 0x7f8d8c1993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.821+0000 7f8d8a800700 1 -- 172.21.15.99:0/4293697269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d8c100d80 msgr2=0x7f8d8c198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:16.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.821+0000 7f8d8a800700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d8c100d80 0x7f8d8c198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.821+0000 7f8d8a800700 1 -- 172.21.15.99:0/4293697269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c0095f0 con 0x7f8d8c102910 2024-03-30T00:33:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.821+0000 7f8d8a800700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c102910 0x7f8d8c1a03e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8d84007ee0 tx=0x7f8d8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.821+0000 7f8d82a00700 1 -- 172.21.15.99:0/4293697269 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8400dd20 con 0x7f8d8c102910 2024-03-30T00:33:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.821+0000 7f8d82a00700 1 -- 172.21.15.99:0/4293697269 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d8400f040 con 0x7f8d8c102910 2024-03-30T00:33:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.821+0000 7f8d82a00700 1 -- 172.21.15.99:0/4293697269 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84004750 con 0x7f8d8c102910 2024-03-30T00:33:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.821+0000 7f8d91200700 1 -- 172.21.15.99:0/4293697269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c1a0f60 con 0x7f8d8c102910 2024-03-30T00:33:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.821+0000 7f8d91200700 1 -- 172.21.15.99:0/4293697269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c1a1330 con 0x7f8d8c102910 2024-03-30T00:33:16.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.822+0000 7f8d91200700 1 -- 172.21.15.99:0/4293697269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c193490 con 0x7f8d8c102910 2024-03-30T00:33:16.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.824+0000 7f8d82a00700 1 -- 172.21.15.99:0/4293697269 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f8d84003a00 con 0x7f8d8c102910 2024-03-30T00:33:16.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.826+0000 7f8d82a00700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d700a1120 0x7f8d700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:16.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.827+0000 7f8d82a00700 1 -- 172.21.15.99:0/4293697269 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8d840cc870 con 0x7f8d8c102910 2024-03-30T00:33:16.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.827+0000 7f8d89e00700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d700a1120 0x7f8d700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:16.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.827+0000 7f8d89e00700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d700a1120 0x7f8d700a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c006010 tx=0x7f8d7c003740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:16.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:16.829+0000 7f8d82a00700 1 -- 172.21.15.99:0/4293697269 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8d840960b0 con 0x7f8d8c102910 2024-03-30T00:33:17.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.085+0000 7f8d91200700 1 -- 172.21.15.99:0/4293697269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8d8c067af0 con 0x7f8d8c102910 2024-03-30T00:33:17.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:17 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/709156996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:17.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:17 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4088599472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:17.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:17 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] up:active 2024-03-30T00:33:17.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:17 smithi161 ceph-mon[42221]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:17.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:17 smithi161 ceph-mon[42221]: osdmap e83: 12 total, 12 up, 12 in 2024-03-30T00:33:17.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:17 smithi161 ceph-mon[42221]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:17.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:17 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4293697269' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-30T00:33:17.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:17 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/709156996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:17.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:17 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4088599472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:17.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:17 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] up:active 2024-03-30T00:33:17.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:17 smithi099 ceph-mon[38079]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:17.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:17 smithi099 ceph-mon[38079]: osdmap e83: 12 total, 12 up, 12 in 2024-03-30T00:33:17.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:17 smithi099 ceph-mon[38079]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:17.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:17 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4293697269' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-30T00:33:17.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:17 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/709156996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:17.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:17 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4088599472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:17.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:17 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] up:active 2024-03-30T00:33:17.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:17 smithi142 ceph-mon[41779]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:17.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:17 smithi142 ceph-mon[41779]: osdmap e83: 12 total, 12 up, 12 in 2024-03-30T00:33:17.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:17 smithi142 ceph-mon[41779]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:17.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:17 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4293697269' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-30T00:33:17.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.665+0000 7f8d82a00700 1 -- 172.21.15.99:0/4293697269 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8d840230b0 con 0x7f8d8c102910 2024-03-30T00:33:17.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.668+0000 7f8d91200700 1 -- 172.21.15.99:0/4293697269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d700a1120 msgr2=0x7f8d700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:17.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.668+0000 7f8d91200700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d700a1120 0x7f8d700a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c006010 tx=0x7f8d7c003740 comp rx=0 tx=0).stop 2024-03-30T00:33:17.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.668+0000 7f8d91200700 1 -- 172.21.15.99:0/4293697269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c102910 msgr2=0x7f8d8c1a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:17.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.668+0000 7f8d91200700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c102910 0x7f8d8c1a03e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8d84007ee0 tx=0x7f8d8400d3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:17.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.668+0000 7f8d91200700 1 -- 172.21.15.99:0/4293697269 shutdown_connections 2024-03-30T00:33:17.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.668+0000 7f8d91200700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c102910 0x7f8d8c1a03e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:17.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.668+0000 7f8d91200700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d700a1120 0x7f8d700a35e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:17.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.668+0000 7f8d91200700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d8c100d80 0x7f8d8c198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:17.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.668+0000 7f8d91200700 1 --2- 172.21.15.99:0/4293697269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d8c101f70 0x7f8d8c1993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:17.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.668+0000 7f8d91200700 1 -- 172.21.15.99:0/4293697269 >> 172.21.15.99:0/4293697269 conn(0x7f8d8c0fc2f0 msgr2=0x7f8d8c107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:17.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.668+0000 7f8d91200700 1 -- 172.21.15.99:0/4293697269 shutdown_connections 2024-03-30T00:33:17.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.669+0000 7f8d91200700 1 -- 172.21.15.99:0/4293697269 wait complete. 2024-03-30T00:33:17.696 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-30T00:33:17.696 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-30T00:33:17.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.885+0000 7f1a4a600700 1 -- 172.21.15.99:0/3986388653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44101f70 msgr2=0x7f1a441023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:17.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.885+0000 7f1a4a600700 1 --2- 172.21.15.99:0/3986388653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44101f70 0x7f1a441023d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a300099c0 tx=0x7f1a30009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:17.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.885+0000 7f1a4a600700 1 -- 172.21.15.99:0/3986388653 shutdown_connections 2024-03-30T00:33:17.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.885+0000 7f1a4a600700 1 --2- 172.21.15.99:0/3986388653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a44102910 0x7f1a441091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:17.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.885+0000 7f1a4a600700 1 --2- 172.21.15.99:0/3986388653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a44100d80 0x7f1a441011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:17.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.885+0000 7f1a4a600700 1 --2- 172.21.15.99:0/3986388653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44101f70 0x7f1a441023d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:17.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.885+0000 7f1a4a600700 1 -- 172.21.15.99:0/3986388653 >> 172.21.15.99:0/3986388653 conn(0x7f1a440fc2f0 msgr2=0x7f1a440fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:17.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.885+0000 7f1a4a600700 1 -- 172.21.15.99:0/3986388653 shutdown_connections 2024-03-30T00:33:17.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.885+0000 7f1a4a600700 1 -- 172.21.15.99:0/3986388653 wait complete. 2024-03-30T00:33:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.886+0000 7f1a4a600700 1 Processor -- start 2024-03-30T00:33:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.886+0000 7f1a4a600700 1 -- start start 2024-03-30T00:33:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.886+0000 7f1a4a600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a44100d80 0x7f1a44074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.886+0000 7f1a4a600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a44101f70 0x7f1a440731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a4a600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44102910 0x7f1a440736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a4a600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44073e80 con 0x7f1a44102910 2024-03-30T00:33:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a4a600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44073ff0 con 0x7f1a44100d80 2024-03-30T00:33:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a4a600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441a0c10 con 0x7f1a44101f70 2024-03-30T00:33:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a43e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44102910 0x7f1a440736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:17.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a43e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44102910 0x7f1a440736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35166/0 (socket says 172.21.15.99:35166) 2024-03-30T00:33:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a43e00700 1 -- 172.21.15.99:0/1025263765 learned_addr learned my addr 172.21.15.99:0/1025263765 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a43400700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a44100d80 0x7f1a44074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a42a00700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a44101f70 0x7f1a440731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a43e00700 1 -- 172.21.15.99:0/1025263765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a44101f70 msgr2=0x7f1a440731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a43e00700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a44101f70 0x7f1a440731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a43e00700 1 -- 172.21.15.99:0/1025263765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a44100d80 msgr2=0x7f1a44074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a43e00700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a44100d80 0x7f1a44074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a43e00700 1 -- 172.21.15.99:0/1025263765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a300095f0 con 0x7f1a44102910 2024-03-30T00:33:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a42a00700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a44101f70 0x7f1a440731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:17.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.887+0000 7f1a43e00700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44102910 0x7f1a440736e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c007ee0 tx=0x7f1a3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:17.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.888+0000 7f1a3be00700 1 -- 172.21.15.99:0/1025263765 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c017070 con 0x7f1a44102910 2024-03-30T00:33:17.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.888+0000 7f1a3be00700 1 -- 172.21.15.99:0/1025263765 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a3c00f040 con 0x7f1a44102910 2024-03-30T00:33:17.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.888+0000 7f1a3be00700 1 -- 172.21.15.99:0/1025263765 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c005570 con 0x7f1a44102910 2024-03-30T00:33:17.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.888+0000 7f1a4a600700 1 -- 172.21.15.99:0/1025263765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a441a0ea0 con 0x7f1a44102910 2024-03-30T00:33:17.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.888+0000 7f1a4a600700 1 -- 172.21.15.99:0/1025263765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a441a1270 con 0x7f1a44102910 2024-03-30T00:33:17.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.889+0000 7f1a4a600700 1 -- 172.21.15.99:0/1025263765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a441081d0 con 0x7f1a44102910 2024-03-30T00:33:17.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.891+0000 7f1a3be00700 1 -- 172.21.15.99:0/1025263765 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1a3c00db70 con 0x7f1a44102910 2024-03-30T00:33:17.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.893+0000 7f1a3be00700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1a280a1120 0x7f1a280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:17.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.893+0000 7f1a3be00700 1 -- 172.21.15.99:0/1025263765 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1a3c0cbff0 con 0x7f1a44102910 2024-03-30T00:33:17.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.893+0000 7f1a43400700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1a280a1120 0x7f1a280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:17.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.894+0000 7f1a43400700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1a280a1120 0x7f1a280a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1a44101dd0 tx=0x7f1a3400a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:17.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:17.897+0000 7f1a3be00700 1 -- 172.21.15.99:0/1025263765 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1a3c095830 con 0x7f1a44102910 2024-03-30T00:33:18.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:18.152+0000 7f1a4a600700 1 -- 172.21.15.99:0/1025263765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f1a4410c490 con 0x7f1a44102910 2024-03-30T00:33:18.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:18 smithi161 ceph-mon[42221]: pgmap v257: 97 pgs: 17 creating+peering, 28 active+clean, 52 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2024-03-30T00:33:18.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:18 smithi161 ceph-mon[42221]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-30T00:33:18.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:18 smithi161 ceph-mon[42221]: Cluster is now healthy 2024-03-30T00:33:18.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:18 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4293697269' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-30T00:33:18.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:18 smithi161 ceph-mon[42221]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:18.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:18 smithi099 ceph-mon[38079]: pgmap v257: 97 pgs: 17 creating+peering, 28 active+clean, 52 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2024-03-30T00:33:18.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:18 smithi099 ceph-mon[38079]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-30T00:33:18.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:18 smithi099 ceph-mon[38079]: Cluster is now healthy 2024-03-30T00:33:18.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:18 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4293697269' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-30T00:33:18.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:18 smithi099 ceph-mon[38079]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:18.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:18 smithi142 ceph-mon[41779]: pgmap v257: 97 pgs: 17 creating+peering, 28 active+clean, 52 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2024-03-30T00:33:18.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:18 smithi142 ceph-mon[41779]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-30T00:33:18.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:18 smithi142 ceph-mon[41779]: Cluster is now healthy 2024-03-30T00:33:18.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:18 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4293697269' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-30T00:33:18.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:18 smithi142 ceph-mon[41779]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:19.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.116+0000 7f1a3be00700 1 -- 172.21.15.99:0/1025263765 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f1a3c098e50 con 0x7f1a44102910 2024-03-30T00:33:19.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.119+0000 7f1a4a600700 1 -- 172.21.15.99:0/1025263765 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1a280a1120 msgr2=0x7f1a280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:19.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.119+0000 7f1a4a600700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1a280a1120 0x7f1a280a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1a44101dd0 tx=0x7f1a3400a3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.119+0000 7f1a4a600700 1 -- 172.21.15.99:0/1025263765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44102910 msgr2=0x7f1a440736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:19.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.119+0000 7f1a4a600700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44102910 0x7f1a440736e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c007ee0 tx=0x7f1a3c00d3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.119+0000 7f1a4a600700 1 -- 172.21.15.99:0/1025263765 shutdown_connections 2024-03-30T00:33:19.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.120+0000 7f1a4a600700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a44102910 0x7f1a440736e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.120+0000 7f1a4a600700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1a280a1120 0x7f1a280a35e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.120+0000 7f1a4a600700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a44100d80 0x7f1a44074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.120+0000 7f1a4a600700 1 --2- 172.21.15.99:0/1025263765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a44101f70 0x7f1a440731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.120+0000 7f1a4a600700 1 -- 172.21.15.99:0/1025263765 >> 172.21.15.99:0/1025263765 conn(0x7f1a440fc2f0 msgr2=0x7f1a44105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:19.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.120+0000 7f1a4a600700 1 -- 172.21.15.99:0/1025263765 shutdown_connections 2024-03-30T00:33:19.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.120+0000 7f1a4a600700 1 -- 172.21.15.99:0/1025263765 wait complete. 2024-03-30T00:33:19.154 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-30T00:33:19.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.346+0000 7fd37de00700 1 -- 172.21.15.99:0/1509399776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd378101f70 msgr2=0x7fd3781023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:19.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.346+0000 7fd37de00700 1 --2- 172.21.15.99:0/1509399776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd378101f70 0x7fd3781023d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd3640099c0 tx=0x7fd364009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.346+0000 7fd37de00700 1 -- 172.21.15.99:0/1509399776 shutdown_connections 2024-03-30T00:33:19.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.346+0000 7fd37de00700 1 --2- 172.21.15.99:0/1509399776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378102910 0x7fd3781091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.346+0000 7fd37de00700 1 --2- 172.21.15.99:0/1509399776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd378100d80 0x7fd3781011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.346+0000 7fd37de00700 1 --2- 172.21.15.99:0/1509399776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd378101f70 0x7fd3781023d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.346+0000 7fd37de00700 1 -- 172.21.15.99:0/1509399776 >> 172.21.15.99:0/1509399776 conn(0x7fd3780fc2f0 msgr2=0x7fd3780fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:19.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.346+0000 7fd37de00700 1 -- 172.21.15.99:0/1509399776 shutdown_connections 2024-03-30T00:33:19.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.346+0000 7fd37de00700 1 -- 172.21.15.99:0/1509399776 wait complete. 2024-03-30T00:33:19.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.347+0000 7fd37de00700 1 Processor -- start 2024-03-30T00:33:19.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd37de00700 1 -- start start 2024-03-30T00:33:19.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd37de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd378100d80 0x7fd378198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:19.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd37de00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378101f70 0x7fd378199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd37de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd378102910 0x7fd3781a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd37de00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781a0a70 con 0x7fd378102910 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd37de00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781a0bb0 con 0x7fd378100d80 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd37de00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781a0d40 con 0x7fd378101f70 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd377400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd378102910 0x7fd3781a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd377400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd378102910 0x7fd3781a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35184/0 (socket says 172.21.15.99:35184) 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd377400700 1 -- 172.21.15.99:0/3193558317 learned_addr learned my addr 172.21.15.99:0/3193558317 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd376a00700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd378100d80 0x7fd378198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.348+0000 7fd376000700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378101f70 0x7fd378199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.349+0000 7fd377400700 1 -- 172.21.15.99:0/3193558317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378101f70 msgr2=0x7fd378199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.349+0000 7fd377400700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378101f70 0x7fd378199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.349+0000 7fd377400700 1 -- 172.21.15.99:0/3193558317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd378100d80 msgr2=0x7fd378198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:19.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.349+0000 7fd377400700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd378100d80 0x7fd378198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:19.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.349+0000 7fd377400700 1 -- 172.21.15.99:0/3193558317 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3640095f0 con 0x7fd378102910 2024-03-30T00:33:19.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.349+0000 7fd377400700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd378102910 0x7fd3781a04a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd37000b970 tx=0x7fd37000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:19.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.349+0000 7fd37ca00700 1 -- 172.21.15.99:0/3193558317 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37000d860 con 0x7fd378102910 2024-03-30T00:33:19.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.349+0000 7fd37de00700 1 -- 172.21.15.99:0/3193558317 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3781a0fc0 con 0x7fd378102910 2024-03-30T00:33:19.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.349+0000 7fd37ca00700 1 -- 172.21.15.99:0/3193558317 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd37000de50 con 0x7fd378102910 2024-03-30T00:33:19.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.349+0000 7fd37ca00700 1 -- 172.21.15.99:0/3193558317 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370017400 con 0x7fd378102910 2024-03-30T00:33:19.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.349+0000 7fd37de00700 1 -- 172.21.15.99:0/3193558317 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3781a14a0 con 0x7fd378102910 2024-03-30T00:33:19.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.350+0000 7fd37de00700 1 -- 172.21.15.99:0/3193558317 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd378067af0 con 0x7fd378102910 2024-03-30T00:33:19.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.352+0000 7fd37ca00700 1 -- 172.21.15.99:0/3193558317 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fd37000d9c0 con 0x7fd378102910 2024-03-30T00:33:19.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.354+0000 7fd37ca00700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd35c0a11a0 0x7fd35c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:19.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.354+0000 7fd37ca00700 1 -- 172.21.15.99:0/3193558317 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd370006e40 con 0x7fd378102910 2024-03-30T00:33:19.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.354+0000 7fd376a00700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd35c0a11a0 0x7fd35c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:19.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.355+0000 7fd376a00700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd35c0a11a0 0x7fd35c0a3660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd378101dd0 tx=0x7fd36800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:19.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.358+0000 7fd37ca00700 1 -- 172.21.15.99:0/3193558317 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd370095bf0 con 0x7fd378102910 2024-03-30T00:33:19.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:19 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1025263765' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-30T00:33:19.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:19 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1025263765' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-30T00:33:19.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:19 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1025263765' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-30T00:33:19.567 INFO:journalctl@ceph.mds.b.smithi142.stdout:Mar 30 00:33:19 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-b[77897]: 2024-03-30T00:33:19.278+0000 7f4a5b600700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T00:33:19.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:19.609+0000 7fd37de00700 1 -- 172.21.15.99:0/3193558317 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fd37810c530 con 0x7fd378102910 2024-03-30T00:33:20.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.098+0000 7fd37ca00700 1 -- 172.21.15.99:0/3193558317 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fd370099210 con 0x7fd378102910 2024-03-30T00:33:20.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 -- 172.21.15.99:0/3193558317 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd35c0a11a0 msgr2=0x7fd35c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:20.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd35c0a11a0 0x7fd35c0a3660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd378101dd0 tx=0x7fd36800b410 comp rx=0 tx=0).stop 2024-03-30T00:33:20.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 -- 172.21.15.99:0/3193558317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd378102910 msgr2=0x7fd3781a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:20.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd378102910 0x7fd3781a04a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd37000b970 tx=0x7fd37000bc80 comp rx=0 tx=0).stop 2024-03-30T00:33:20.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 -- 172.21.15.99:0/3193558317 shutdown_connections 2024-03-30T00:33:20.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd378102910 0x7fd3781a04a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:20.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd35c0a11a0 0x7fd35c0a3660 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:20.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd378100d80 0x7fd378198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:20.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 --2- 172.21.15.99:0/3193558317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd378101f70 0x7fd378199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:20.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 -- 172.21.15.99:0/3193558317 >> 172.21.15.99:0/3193558317 conn(0x7fd3780fc2f0 msgr2=0x7fd378072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:20.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 -- 172.21.15.99:0/3193558317 shutdown_connections 2024-03-30T00:33:20.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.101+0000 7fd37de00700 1 -- 172.21.15.99:0/3193558317 wait complete. 2024-03-30T00:33:20.133 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-30T00:33:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.322+0000 7fc86a800700 1 -- 172.21.15.99:0/4094907774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc864101f70 msgr2=0x7fc8641023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.322+0000 7fc86a800700 1 --2- 172.21.15.99:0/4094907774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc864101f70 0x7fc8641023d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc850009920 tx=0x7fc850009c30 comp rx=0 tx=0).stop 2024-03-30T00:33:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.322+0000 7fc86a800700 1 -- 172.21.15.99:0/4094907774 shutdown_connections 2024-03-30T00:33:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.322+0000 7fc86a800700 1 --2- 172.21.15.99:0/4094907774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864102910 0x7fc8641091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.322+0000 7fc86a800700 1 --2- 172.21.15.99:0/4094907774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc864100d80 0x7fc8641011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.322+0000 7fc86a800700 1 --2- 172.21.15.99:0/4094907774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc864101f70 0x7fc8641023d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.322+0000 7fc86a800700 1 -- 172.21.15.99:0/4094907774 >> 172.21.15.99:0/4094907774 conn(0x7fc8640fc2f0 msgr2=0x7fc8640fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.322+0000 7fc86a800700 1 -- 172.21.15.99:0/4094907774 shutdown_connections 2024-03-30T00:33:20.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.322+0000 7fc86a800700 1 -- 172.21.15.99:0/4094907774 wait complete. 2024-03-30T00:33:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.323+0000 7fc86a800700 1 Processor -- start 2024-03-30T00:33:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.323+0000 7fc86a800700 1 -- start start 2024-03-30T00:33:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.323+0000 7fc86a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc864100d80 0x7fc864074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.323+0000 7fc86a800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864101f70 0x7fc8640731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.323+0000 7fc86a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc864102910 0x7fc8640736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.323+0000 7fc86a800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc864073de0 con 0x7fc864102910 2024-03-30T00:33:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.323+0000 7fc86a800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc864073f50 con 0x7fc864100d80 2024-03-30T00:33:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.323+0000 7fc86a800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8641a0db0 con 0x7fc864101f70 2024-03-30T00:33:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc863e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc864102910 0x7fc8640736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc863e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc864102910 0x7fc8640736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35212/0 (socket says 172.21.15.99:35212) 2024-03-30T00:33:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc863e00700 1 -- 172.21.15.99:0/4004954882 learned_addr learned my addr 172.21.15.99:0/4004954882 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc862a00700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864101f70 0x7fc8640731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc863400700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc864100d80 0x7fc864074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc863e00700 1 -- 172.21.15.99:0/4004954882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864101f70 msgr2=0x7fc8640731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc863e00700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864101f70 0x7fc8640731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc863e00700 1 -- 172.21.15.99:0/4004954882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc864100d80 msgr2=0x7fc864074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc863e00700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc864100d80 0x7fc864074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc863e00700 1 -- 172.21.15.99:0/4004954882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850009520 con 0x7fc864102910 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc863e00700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc864102910 0x7fc8640736e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00ef30 tx=0x7fc85c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc85be00700 1 -- 172.21.15.99:0/4004954882 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c018070 con 0x7fc864102910 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc85be00700 1 -- 172.21.15.99:0/4004954882 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc85c00f040 con 0x7fc864102910 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc86a800700 1 -- 172.21.15.99:0/4004954882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8641a0fb0 con 0x7fc864102910 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.324+0000 7fc85be00700 1 -- 172.21.15.99:0/4004954882 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00cbb0 con 0x7fc864102910 2024-03-30T00:33:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.325+0000 7fc86a800700 1 -- 172.21.15.99:0/4004954882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8641a1480 con 0x7fc864102910 2024-03-30T00:33:20.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.325+0000 7fc86a800700 1 -- 172.21.15.99:0/4004954882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc864108290 con 0x7fc864102910 2024-03-30T00:33:20.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.327+0000 7fc85be00700 1 -- 172.21.15.99:0/4004954882 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc85c00cd70 con 0x7fc864102910 2024-03-30T00:33:20.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.329+0000 7fc85be00700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc8480a11a0 0x7fc8480a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:20.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.329+0000 7fc85be00700 1 -- 172.21.15.99:0/4004954882 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc85c0cc150 con 0x7fc864102910 2024-03-30T00:33:20.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.329+0000 7fc863400700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc8480a11a0 0x7fc8480a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:20.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.329+0000 7fc863400700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc8480a11a0 0x7fc8480a3660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc864101dd0 tx=0x7fc854009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:20.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.333+0000 7fc85be00700 1 -- 172.21.15.99:0/4004954882 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc85c095ac0 con 0x7fc864102910 2024-03-30T00:33:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: pgmap v258: 97 pgs: 17 creating+peering, 67 active+clean, 13 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-03-30T00:33:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-30T00:33:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1025263765' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-30T00:33:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-30T00:33:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-30T00:33:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-30T00:33:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3193558317' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-30T00:33:20.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] up:active 2024-03-30T00:33:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3193558317' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-30T00:33:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-30T00:33:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-30T00:33:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: Cluster is now healthy 2024-03-30T00:33:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 2 up:standby-replay 8 up:standby 2024-03-30T00:33:20.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:20 smithi161 ceph-mon[42221]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-03-30T00:33:20.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: pgmap v258: 97 pgs: 17 creating+peering, 67 active+clean, 13 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-03-30T00:33:20.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-30T00:33:20.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1025263765' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-30T00:33:20.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-30T00:33:20.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-30T00:33:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-30T00:33:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3193558317' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-30T00:33:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] up:active 2024-03-30T00:33:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3193558317' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-30T00:33:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-30T00:33:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-30T00:33:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: Cluster is now healthy 2024-03-30T00:33:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 2 up:standby-replay 8 up:standby 2024-03-30T00:33:20.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:20 smithi099 ceph-mon[38079]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-03-30T00:33:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: pgmap v258: 97 pgs: 17 creating+peering, 67 active+clean, 13 unknown; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-03-30T00:33:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-30T00:33:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1025263765' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-30T00:33:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-30T00:33:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T00:33:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-03-30T00:33:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-03-30T00:33:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3193558317' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-30T00:33:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] up:active 2024-03-30T00:33:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3193558317' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-30T00:33:20.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-30T00:33:20.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-30T00:33:20.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: Cluster is now healthy 2024-03-30T00:33:20.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 2 up:standby-replay 8 up:standby 2024-03-30T00:33:20.568 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:20 smithi142 ceph-mon[41779]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-03-30T00:33:20.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:20.587+0000 7fc86a800700 1 -- 172.21.15.99:0/4004954882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fc864067b50 con 0x7fc864102910 2024-03-30T00:33:21.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.118+0000 7fc85be00700 1 -- 172.21.15.99:0/4004954882 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc85c0990e0 con 0x7fc864102910 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 -- 172.21.15.99:0/4004954882 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc8480a11a0 msgr2=0x7fc8480a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc8480a11a0 0x7fc8480a3660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc864101dd0 tx=0x7fc854009310 comp rx=0 tx=0).stop 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 -- 172.21.15.99:0/4004954882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc864102910 msgr2=0x7fc8640736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc864102910 0x7fc8640736e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00ef30 tx=0x7fc85c00c3a0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 -- 172.21.15.99:0/4004954882 shutdown_connections 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc864102910 0x7fc8640736e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc8480a11a0 0x7fc8480a3660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc864100d80 0x7fc864074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 --2- 172.21.15.99:0/4004954882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc864101f70 0x7fc8640731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 -- 172.21.15.99:0/4004954882 >> 172.21.15.99:0/4004954882 conn(0x7fc8640fc2f0 msgr2=0x7fc8641053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 -- 172.21.15.99:0/4004954882 shutdown_connections 2024-03-30T00:33:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.122+0000 7fc86a800700 1 -- 172.21.15.99:0/4004954882 wait complete. 2024-03-30T00:33:21.150 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-03-30T00:33:21.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-30T00:33:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.340+0000 7f5321200700 1 -- 172.21.15.99:0/883285995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f531c074df0 msgr2=0x7f531c073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.340+0000 7f5321200700 1 --2- 172.21.15.99:0/883285995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f531c074df0 0x7f531c073240 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f530c009990 tx=0x7f530c009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.340+0000 7f5321200700 1 -- 172.21.15.99:0/883285995 shutdown_connections 2024-03-30T00:33:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.340+0000 7f5321200700 1 --2- 172.21.15.99:0/883285995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c108c40 0x7f531c10b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.340+0000 7f5321200700 1 --2- 172.21.15.99:0/883285995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f531c074df0 0x7f531c073240 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.340+0000 7f5321200700 1 --2- 172.21.15.99:0/883285995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531c073810 0x7f531c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.340+0000 7f5321200700 1 -- 172.21.15.99:0/883285995 >> 172.21.15.99:0/883285995 conn(0x7f531c0fc2f0 msgr2=0x7f531c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.340+0000 7f5321200700 1 -- 172.21.15.99:0/883285995 shutdown_connections 2024-03-30T00:33:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.340+0000 7f5321200700 1 -- 172.21.15.99:0/883285995 wait complete. 2024-03-30T00:33:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.341+0000 7f5321200700 1 Processor -- start 2024-03-30T00:33:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.341+0000 7f5321200700 1 -- start start 2024-03-30T00:33:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.341+0000 7f5321200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c073810 0x7f531c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.341+0000 7f5321200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f531c074df0 0x7f531c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.341+0000 7f5321200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531c108c40 0x7f531c1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.341+0000 7f5321200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c1a09a0 con 0x7f531c074df0 2024-03-30T00:33:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.341+0000 7f5321200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c1a0ae0 con 0x7f531c108c40 2024-03-30T00:33:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.341+0000 7f5321200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c1a0c70 con 0x7f531c073810 2024-03-30T00:33:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5319400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f531c074df0 0x7f531c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5319400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f531c074df0 0x7f531c1993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35222/0 (socket says 172.21.15.99:35222) 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5319400700 1 -- 172.21.15.99:0/3335253110 learned_addr learned my addr 172.21.15.99:0/3335253110 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5319400700 1 -- 172.21.15.99:0/3335253110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c073810 msgr2=0x7f531c198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5319400700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c073810 0x7f531c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5319400700 1 -- 172.21.15.99:0/3335253110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531c108c40 msgr2=0x7f531c1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5319400700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531c108c40 0x7f531c1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5319400700 1 -- 172.21.15.99:0/3335253110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0095f0 con 0x7f531c074df0 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5319400700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f531c074df0 0x7f531c1993b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f530800e970 tx=0x7f530800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5312a00700 1 -- 172.21.15.99:0/3335253110 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308009790 con 0x7f531c074df0 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5312a00700 1 -- 172.21.15.99:0/3335253110 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f530800c950 con 0x7f531c074df0 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.342+0000 7f5321200700 1 -- 172.21.15.99:0/3335253110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c1a0f50 con 0x7f531c074df0 2024-03-30T00:33:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.343+0000 7f5312a00700 1 -- 172.21.15.99:0/3335253110 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308003ce0 con 0x7f531c074df0 2024-03-30T00:33:21.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.343+0000 7f5321200700 1 -- 172.21.15.99:0/3335253110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c1a1420 con 0x7f531c074df0 2024-03-30T00:33:21.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.343+0000 7f5321200700 1 -- 172.21.15.99:0/3335253110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c107a00 con 0x7f531c074df0 2024-03-30T00:33:21.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.345+0000 7f5312a00700 1 -- 172.21.15.99:0/3335253110 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f530800cac0 con 0x7f531c074df0 2024-03-30T00:33:21.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.347+0000 7f5312a00700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53000a11a0 0x7f53000a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:21.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.347+0000 7f5312a00700 1 -- 172.21.15.99:0/3335253110 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5308014070 con 0x7f531c074df0 2024-03-30T00:33:21.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.347+0000 7f5319e00700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53000a11a0 0x7f53000a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:21.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.347+0000 7f5319e00700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53000a11a0 0x7f53000a3660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b710 tx=0x7f530c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:21.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.350+0000 7f5312a00700 1 -- 172.21.15.99:0/3335253110 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5308096220 con 0x7f531c074df0 2024-03-30T00:33:21.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:21 smithi161 ceph-mon[42221]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-03-30T00:33:21.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:21 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4004954882' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-30T00:33:21.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:21 smithi099 ceph-mon[38079]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-03-30T00:33:21.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:21 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4004954882' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-30T00:33:21.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:21 smithi142 ceph-mon[41779]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-03-30T00:33:21.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:21 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4004954882' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-30T00:33:21.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.590+0000 7f5321200700 1 -- 172.21.15.99:0/3335253110 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f531c02cda0 con 0x7f53000a11a0 2024-03-30T00:33:21.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.618+0000 7f5312a00700 1 -- 172.21.15.99:0/3335253110 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f531c02cda0 con 0x7f53000a11a0 2024-03-30T00:33:21.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 -- 172.21.15.99:0/3335253110 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53000a11a0 msgr2=0x7f53000a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:21.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53000a11a0 0x7f53000a3660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b710 tx=0x7f530c005fd0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 -- 172.21.15.99:0/3335253110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f531c074df0 msgr2=0x7f531c1993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:21.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f531c074df0 0x7f531c1993b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f530800e970 tx=0x7f530800ed30 comp rx=0 tx=0).stop 2024-03-30T00:33:21.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 -- 172.21.15.99:0/3335253110 shutdown_connections 2024-03-30T00:33:21.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f53000a11a0 0x7f53000a3660 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531c108c40 0x7f531c1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f531c073810 0x7f531c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 --2- 172.21.15.99:0/3335253110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f531c074df0 0x7f531c1993b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 -- 172.21.15.99:0/3335253110 >> 172.21.15.99:0/3335253110 conn(0x7f531c0fc2f0 msgr2=0x7f531c104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 -- 172.21.15.99:0/3335253110 shutdown_connections 2024-03-30T00:33:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.620+0000 7f5321200700 1 -- 172.21.15.99:0/3335253110 wait complete. 2024-03-30T00:33:21.645 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-30T00:33:21.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.835+0000 7feacbc00700 1 -- 172.21.15.99:0/1373985317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac4101f70 msgr2=0x7feac41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:21.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.835+0000 7feacbc00700 1 --2- 172.21.15.99:0/1373985317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac4101f70 0x7feac41023d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7feab00099c0 tx=0x7feab0009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.835+0000 7feacbc00700 1 -- 172.21.15.99:0/1373985317 shutdown_connections 2024-03-30T00:33:21.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.835+0000 7feacbc00700 1 --2- 172.21.15.99:0/1373985317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feac4102910 0x7feac41091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.835+0000 7feacbc00700 1 --2- 172.21.15.99:0/1373985317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feac4100d80 0x7feac41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.835+0000 7feacbc00700 1 --2- 172.21.15.99:0/1373985317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac4101f70 0x7feac41023d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.835+0000 7feacbc00700 1 -- 172.21.15.99:0/1373985317 >> 172.21.15.99:0/1373985317 conn(0x7feac40fc2f0 msgr2=0x7feac40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:21.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.835+0000 7feacbc00700 1 -- 172.21.15.99:0/1373985317 shutdown_connections 2024-03-30T00:33:21.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.835+0000 7feacbc00700 1 -- 172.21.15.99:0/1373985317 wait complete. 2024-03-30T00:33:21.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.836+0000 7feacbc00700 1 Processor -- start 2024-03-30T00:33:21.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.836+0000 7feacbc00700 1 -- start start 2024-03-30T00:33:21.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.836+0000 7feacbc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feac4100d80 0x7feac4198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:21.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.836+0000 7feacbc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feac4101f70 0x7feac41993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:21.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.836+0000 7feacbc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac4102910 0x7feac41a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:21.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.836+0000 7feacbc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac41a0960 con 0x7feac4102910 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feacbc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac41a0aa0 con 0x7feac4100d80 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feacbc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac41a0c30 con 0x7feac4101f70 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac9c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac4102910 0x7feac41a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac9c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac4102910 0x7feac41a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35264/0 (socket says 172.21.15.99:35264) 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac9c00700 1 -- 172.21.15.99:0/831987876 learned_addr learned my addr 172.21.15.99:0/831987876 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac3e00700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feac4101f70 0x7feac41993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac9200700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feac4100d80 0x7feac4198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac9c00700 1 -- 172.21.15.99:0/831987876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feac4101f70 msgr2=0x7feac41993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac9c00700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feac4101f70 0x7feac41993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac9c00700 1 -- 172.21.15.99:0/831987876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feac4100d80 msgr2=0x7feac4198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac9c00700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feac4100d80 0x7feac4198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:21.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac9c00700 1 -- 172.21.15.99:0/831987876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab00095f0 con 0x7feac4102910 2024-03-30T00:33:21.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac9200700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feac4100d80 0x7feac4198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:21.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.837+0000 7feac9c00700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac4102910 0x7feac41a0390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feabc007ee0 tx=0x7feabc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:21.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.838+0000 7feac1600700 1 -- 172.21.15.99:0/831987876 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc017070 con 0x7feac4102910 2024-03-30T00:33:21.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.838+0000 7feac1600700 1 -- 172.21.15.99:0/831987876 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feabc00f040 con 0x7feac4102910 2024-03-30T00:33:21.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.838+0000 7feac1600700 1 -- 172.21.15.99:0/831987876 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc005710 con 0x7feac4102910 2024-03-30T00:33:21.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.838+0000 7feacbc00700 1 -- 172.21.15.99:0/831987876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac41a0f10 con 0x7feac4102910 2024-03-30T00:33:21.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.838+0000 7feacbc00700 1 -- 172.21.15.99:0/831987876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac41a1330 con 0x7feac4102910 2024-03-30T00:33:21.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.839+0000 7feacbc00700 1 -- 172.21.15.99:0/831987876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac4067af0 con 0x7feac4102910 2024-03-30T00:33:21.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.841+0000 7feac1600700 1 -- 172.21.15.99:0/831987876 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7feabc005870 con 0x7feac4102910 2024-03-30T00:33:21.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.842+0000 7feac1600700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feaac0a1120 0x7feaac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:21.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.842+0000 7feac1600700 1 -- 172.21.15.99:0/831987876 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7feabc0cbf40 con 0x7feac4102910 2024-03-30T00:33:21.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.842+0000 7feac9200700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feaac0a1120 0x7feaac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:21.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.843+0000 7feac9200700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feaac0a1120 0x7feaac0a35e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feac4101dd0 tx=0x7feab8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:21.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:21.846+0000 7feac1600700 1 -- 172.21.15.99:0/831987876 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7feabc095820 con 0x7feac4102910 2024-03-30T00:33:21.987 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:33:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:33:21.596+0000 7f2e09000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T00:33:21.987 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:33:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:33:21.596+0000 7f2e09000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T00:33:21.987 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:33:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:33:21.596+0000 7f2e09000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T00:33:21.988 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:33:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:33:21.596+0000 7f2e09000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T00:33:21.988 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:33:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:33:21.596+0000 7f2e09000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T00:33:22.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.089+0000 7feacbc00700 1 -- 172.21.15.99:0/831987876 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7feac4108e60 con 0x7feaac0a1120 2024-03-30T00:33:22.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.118+0000 7feac1600700 1 -- 172.21.15.99:0/831987876 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7feac4108e60 con 0x7feaac0a1120 2024-03-30T00:33:22.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.120+0000 7feacbc00700 1 -- 172.21.15.99:0/831987876 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feaac0a1120 msgr2=0x7feaac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:22.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.120+0000 7feacbc00700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feaac0a1120 0x7feaac0a35e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7feac4101dd0 tx=0x7feab8009260 comp rx=0 tx=0).stop 2024-03-30T00:33:22.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.120+0000 7feacbc00700 1 -- 172.21.15.99:0/831987876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac4102910 msgr2=0x7feac41a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:22.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.120+0000 7feacbc00700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac4102910 0x7feac41a0390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feabc007ee0 tx=0x7feabc00d3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.120+0000 7feacbc00700 1 -- 172.21.15.99:0/831987876 shutdown_connections 2024-03-30T00:33:22.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.121+0000 7feacbc00700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac4102910 0x7feac41a0390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.121+0000 7feacbc00700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feaac0a1120 0x7feaac0a35e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.121+0000 7feacbc00700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feac4100d80 0x7feac4198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.121+0000 7feacbc00700 1 --2- 172.21.15.99:0/831987876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feac4101f70 0x7feac41993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.121+0000 7feacbc00700 1 -- 172.21.15.99:0/831987876 >> 172.21.15.99:0/831987876 conn(0x7feac40fc2f0 msgr2=0x7feac4107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:22.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.121+0000 7feacbc00700 1 -- 172.21.15.99:0/831987876 shutdown_connections 2024-03-30T00:33:22.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.121+0000 7feacbc00700 1 -- 172.21.15.99:0/831987876 wait complete. 2024-03-30T00:33:22.146 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-30T00:33:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.337+0000 7f2a94c00700 1 -- 172.21.15.99:0/2456597444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a90106b30 msgr2=0x7f2a90108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.337+0000 7f2a94c00700 1 --2- 172.21.15.99:0/2456597444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a90106b30 0x7f2a90108f20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800b0a0 tx=0x7f2a8800b3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.337+0000 7f2a94c00700 1 -- 172.21.15.99:0/2456597444 shutdown_connections 2024-03-30T00:33:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.337+0000 7f2a94c00700 1 --2- 172.21.15.99:0/2456597444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a90106b30 0x7f2a90108f20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.337+0000 7f2a94c00700 1 --2- 172.21.15.99:0/2456597444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a901018d0 0x7f2a90103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.337+0000 7f2a94c00700 1 --2- 172.21.15.99:0/2456597444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a90104200 0x7f2a901065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.337+0000 7f2a94c00700 1 -- 172.21.15.99:0/2456597444 >> 172.21.15.99:0/2456597444 conn(0x7f2a900fb200 msgr2=0x7f2a900fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:22.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.337+0000 7f2a94c00700 1 -- 172.21.15.99:0/2456597444 shutdown_connections 2024-03-30T00:33:22.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.337+0000 7f2a94c00700 1 -- 172.21.15.99:0/2456597444 wait complete. 2024-03-30T00:33:22.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.338+0000 7f2a94c00700 1 Processor -- start 2024-03-30T00:33:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.338+0000 7f2a94c00700 1 -- start start 2024-03-30T00:33:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.338+0000 7f2a94c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a901018d0 0x7f2a90198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.338+0000 7f2a94c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a90104200 0x7f2a90199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.338+0000 7f2a94c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a90106b30 0x7f2a901a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.338+0000 7f2a94c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a901a09f0 con 0x7f2a90106b30 2024-03-30T00:33:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.338+0000 7f2a94c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a901a0b30 con 0x7f2a90104200 2024-03-30T00:33:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.338+0000 7f2a94c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a901a0cc0 con 0x7f2a901018d0 2024-03-30T00:33:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.338+0000 7f2a8e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a90106b30 0x7f2a901a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.338+0000 7f2a8e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a90106b30 0x7f2a901a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35278/0 (socket says 172.21.15.99:35278) 2024-03-30T00:33:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.338+0000 7f2a8e800700 1 -- 172.21.15.99:0/3529353231 learned_addr learned my addr 172.21.15.99:0/3529353231 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a8de00700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a901018d0 0x7f2a90198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a8e800700 1 -- 172.21.15.99:0/3529353231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a901018d0 msgr2=0x7f2a90198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a8e800700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a901018d0 0x7f2a90198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a8e800700 1 -- 172.21.15.99:0/3529353231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a90104200 msgr2=0x7f2a90199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a8e800700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a90104200 0x7f2a90199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a8e800700 1 -- 172.21.15.99:0/3529353231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a88009d00 con 0x7f2a90106b30 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a8e800700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a90106b30 0x7f2a901a0420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800b780 tx=0x7f2a8800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a86a00700 1 -- 172.21.15.99:0/3529353231 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800e040 con 0x7f2a90106b30 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a86a00700 1 -- 172.21.15.99:0/3529353231 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a88003e10 con 0x7f2a90106b30 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a94c00700 1 -- 172.21.15.99:0/3529353231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a901a0f40 con 0x7f2a90106b30 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a86a00700 1 -- 172.21.15.99:0/3529353231 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88005420 con 0x7f2a90106b30 2024-03-30T00:33:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.339+0000 7f2a94c00700 1 -- 172.21.15.99:0/3529353231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a901a1300 con 0x7f2a90106b30 2024-03-30T00:33:22.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.341+0000 7f2a94c00700 1 -- 172.21.15.99:0/3529353231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a901086b0 con 0x7f2a90106b30 2024-03-30T00:33:22.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.342+0000 7f2a86a00700 1 -- 172.21.15.99:0/3529353231 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f2a8801db10 con 0x7f2a90106b30 2024-03-30T00:33:22.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.344+0000 7f2a86a00700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2a740a11a0 0x7f2a740a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:22.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.344+0000 7f2a86a00700 1 -- 172.21.15.99:0/3529353231 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2a880cdc50 con 0x7f2a90106b30 2024-03-30T00:33:22.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.344+0000 7f2a8de00700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2a740a11a0 0x7f2a740a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:22.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.345+0000 7f2a8de00700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2a740a11a0 0x7f2a740a3660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2a80005ea0 tx=0x7f2a80005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:22.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.348+0000 7f2a86a00700 1 -- 172.21.15.99:0/3529353231 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2a880d3020 con 0x7f2a90106b30 2024-03-30T00:33:22.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:22 smithi161 ceph-mon[42221]: pgmap v259: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-03-30T00:33:22.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:22 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] up:active 2024-03-30T00:33:22.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:22 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4004954882' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-30T00:33:22.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:22 smithi161 ceph-mon[42221]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-03-30T00:33:22.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:22 smithi161 ceph-mon[42221]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:33:22.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:22 smithi099 ceph-mon[38079]: pgmap v259: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-03-30T00:33:22.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:22 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] up:active 2024-03-30T00:33:22.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:22 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4004954882' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-30T00:33:22.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:22 smithi099 ceph-mon[38079]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-03-30T00:33:22.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:22 smithi099 ceph-mon[38079]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:33:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:22 smithi142 ceph-mon[41779]: pgmap v259: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-03-30T00:33:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:22 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] up:active 2024-03-30T00:33:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:22 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4004954882' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-30T00:33:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:22 smithi142 ceph-mon[41779]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-03-30T00:33:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:22 smithi142 ceph-mon[41779]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:33:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.590+0000 7f2a94c00700 1 -- 172.21.15.99:0/3529353231 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a9002cd70 con 0x7f2a740a11a0 2024-03-30T00:33:22.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.594+0000 7f2a86a00700 1 -- 172.21.15.99:0/3529353231 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f2a9002cd70 con 0x7f2a740a11a0 2024-03-30T00:33:22.601 INFO:teuthology.orchestra.run.smithi099.stdout:/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af 2024-03-30T00:33:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 -- 172.21.15.99:0/3529353231 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2a740a11a0 msgr2=0x7f2a740a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2a740a11a0 0x7f2a740a3660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2a80005ea0 tx=0x7f2a80005e30 comp rx=0 tx=0).stop 2024-03-30T00:33:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 -- 172.21.15.99:0/3529353231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a90106b30 msgr2=0x7f2a901a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a90106b30 0x7f2a901a0420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800b780 tx=0x7f2a8800bd70 comp rx=0 tx=0).stop 2024-03-30T00:33:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 -- 172.21.15.99:0/3529353231 shutdown_connections 2024-03-30T00:33:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a90106b30 0x7f2a901a0420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a901018d0 0x7f2a90198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f2a740a11a0 0x7f2a740a3660 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 --2- 172.21.15.99:0/3529353231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a90104200 0x7f2a90199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 -- 172.21.15.99:0/3529353231 >> 172.21.15.99:0/3529353231 conn(0x7f2a900fb200 msgr2=0x7f2a90101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:22.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 -- 172.21.15.99:0/3529353231 shutdown_connections 2024-03-30T00:33:22.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.597+0000 7f2a94c00700 1 -- 172.21.15.99:0/3529353231 wait complete. 2024-03-30T00:33:22.622 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-30T00:33:22.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.811+0000 7f8d26600700 1 -- 172.21.15.99:0/162801517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d20101f70 msgr2=0x7f8d201023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:22.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.811+0000 7f8d26600700 1 --2- 172.21.15.99:0/162801517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d20101f70 0x7f8d201023d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0099c0 tx=0x7f8d0c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.811+0000 7f8d26600700 1 -- 172.21.15.99:0/162801517 shutdown_connections 2024-03-30T00:33:22.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.811+0000 7f8d26600700 1 --2- 172.21.15.99:0/162801517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20102910 0x7f8d201091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.811+0000 7f8d26600700 1 --2- 172.21.15.99:0/162801517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d20100d80 0x7f8d201011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.811+0000 7f8d26600700 1 --2- 172.21.15.99:0/162801517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d20101f70 0x7f8d201023d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.811+0000 7f8d26600700 1 -- 172.21.15.99:0/162801517 >> 172.21.15.99:0/162801517 conn(0x7f8d200fc2f0 msgr2=0x7f8d200fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.811+0000 7f8d26600700 1 -- 172.21.15.99:0/162801517 shutdown_connections 2024-03-30T00:33:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.811+0000 7f8d26600700 1 -- 172.21.15.99:0/162801517 wait complete. 2024-03-30T00:33:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.812+0000 7f8d26600700 1 Processor -- start 2024-03-30T00:33:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.812+0000 7f8d26600700 1 -- start start 2024-03-30T00:33:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.812+0000 7f8d26600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d20100d80 0x7f8d20074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.812+0000 7f8d26600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d20101f70 0x7f8d200731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.812+0000 7f8d26600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20102910 0x7f8d200736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.812+0000 7f8d26600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20073e80 con 0x7f8d20101f70 2024-03-30T00:33:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.812+0000 7f8d26600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20073ff0 con 0x7f8d20100d80 2024-03-30T00:33:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.812+0000 7f8d26600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d201a0c00 con 0x7f8d20102910 2024-03-30T00:33:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d20101f70 0x7f8d200731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d20101f70 0x7f8d200731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35304/0 (socket says 172.21.15.99:35304) 2024-03-30T00:33:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1ea00700 1 -- 172.21.15.99:0/4189783241 learned_addr learned my addr 172.21.15.99:0/4189783241 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1fe00700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20102910 0x7f8d200736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1f400700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d20100d80 0x7f8d20074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1ea00700 1 -- 172.21.15.99:0/4189783241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20102910 msgr2=0x7f8d200736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1ea00700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20102910 0x7f8d200736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1ea00700 1 -- 172.21.15.99:0/4189783241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d20100d80 msgr2=0x7f8d20074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1ea00700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d20100d80 0x7f8d20074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1ea00700 1 -- 172.21.15.99:0/4189783241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0095f0 con 0x7f8d20101f70 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1f400700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d20100d80 0x7f8d20074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d1ea00700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d20101f70 0x7f8d200731a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c000c00 tx=0x7f8d0c003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d17e00700 1 -- 172.21.15.99:0/4189783241 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c01d070 con 0x7f8d20101f70 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.813+0000 7f8d17e00700 1 -- 172.21.15.99:0/4189783241 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d0c00bd30 con 0x7f8d20101f70 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.814+0000 7f8d17e00700 1 -- 172.21.15.99:0/4189783241 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c022dc0 con 0x7f8d20101f70 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.814+0000 7f8d26600700 1 -- 172.21.15.99:0/4189783241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d201a0e30 con 0x7f8d20101f70 2024-03-30T00:33:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.814+0000 7f8d26600700 1 -- 172.21.15.99:0/4189783241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d201a11a0 con 0x7f8d20101f70 2024-03-30T00:33:22.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.815+0000 7f8d26600700 1 -- 172.21.15.99:0/4189783241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d20108200 con 0x7f8d20101f70 2024-03-30T00:33:22.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.817+0000 7f8d17e00700 1 -- 172.21.15.99:0/4189783241 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f8d0c003ff0 con 0x7f8d20101f70 2024-03-30T00:33:22.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.818+0000 7f8d17e00700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d040a11a0 0x7f8d040a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:22.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.818+0000 7f8d17e00700 1 -- 172.21.15.99:0/4189783241 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8d0c0cdcf0 con 0x7f8d20101f70 2024-03-30T00:33:22.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.818+0000 7f8d1f400700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d040a11a0 0x7f8d040a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:22.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.819+0000 7f8d1f400700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d040a11a0 0x7f8d040a3660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005fd0 tx=0x7f8d10005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:22.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:22.822+0000 7f8d17e00700 1 -- 172.21.15.99:0/4189783241 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8d0c0975c0 con 0x7f8d20101f70 2024-03-30T00:33:23.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.068+0000 7f8d26600700 1 -- 172.21.15.99:0/4189783241 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d200611f0 con 0x7f8d040a11a0 2024-03-30T00:33:23.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.096+0000 7f8d17e00700 1 -- 172.21.15.99:0/4189783241 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8d200611f0 con 0x7f8d040a11a0 2024-03-30T00:33:23.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 -- 172.21.15.99:0/4189783241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d040a11a0 msgr2=0x7f8d040a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:23.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d040a11a0 0x7f8d040a3660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005fd0 tx=0x7f8d10005e40 comp rx=0 tx=0).stop 2024-03-30T00:33:23.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 -- 172.21.15.99:0/4189783241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d20101f70 msgr2=0x7f8d200731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:23.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d20101f70 0x7f8d200731a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c000c00 tx=0x7f8d0c003b00 comp rx=0 tx=0).stop 2024-03-30T00:33:23.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 -- 172.21.15.99:0/4189783241 shutdown_connections 2024-03-30T00:33:23.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20102910 0x7f8d200736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8d040a11a0 0x7f8d040a3660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d20100d80 0x7f8d20074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 --2- 172.21.15.99:0/4189783241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d20101f70 0x7f8d200731a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 -- 172.21.15.99:0/4189783241 >> 172.21.15.99:0/4189783241 conn(0x7f8d200fc2f0 msgr2=0x7f8d20105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:23.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 -- 172.21.15.99:0/4189783241 shutdown_connections 2024-03-30T00:33:23.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.099+0000 7f8d26600700 1 -- 172.21.15.99:0/4189783241 wait complete. 2024-03-30T00:33:23.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-30T00:33:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.320+0000 7ff98f600700 1 -- 172.21.15.99:0/3937090226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff988102910 msgr2=0x7ff9881091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.321+0000 7ff98f600700 1 --2- 172.21.15.99:0/3937090226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff988102910 0x7ff9881091c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff98000b0d0 tx=0x7ff98000b3e0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.321+0000 7ff98f600700 1 -- 172.21.15.99:0/3937090226 shutdown_connections 2024-03-30T00:33:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.321+0000 7ff98f600700 1 --2- 172.21.15.99:0/3937090226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff988102910 0x7ff9881091c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.321+0000 7ff98f600700 1 --2- 172.21.15.99:0/3937090226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff988100d80 0x7ff9881011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.321+0000 7ff98f600700 1 --2- 172.21.15.99:0/3937090226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff988101f70 0x7ff9881023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.321+0000 7ff98f600700 1 -- 172.21.15.99:0/3937090226 >> 172.21.15.99:0/3937090226 conn(0x7ff9880fc2f0 msgr2=0x7ff9880fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.321+0000 7ff98f600700 1 -- 172.21.15.99:0/3937090226 shutdown_connections 2024-03-30T00:33:23.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.321+0000 7ff98f600700 1 -- 172.21.15.99:0/3937090226 wait complete. 2024-03-30T00:33:23.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff98f600700 1 Processor -- start 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff98f600700 1 -- start start 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff98f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff988100d80 0x7ff988198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff98f600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff988101f70 0x7ff988199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff98f600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff988102910 0x7ff9881a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff98f600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9881a0a70 con 0x7ff988101f70 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff98f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9881a0bb0 con 0x7ff988100d80 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff98f600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9881a0d40 con 0x7ff988102910 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff987e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff988101f70 0x7ff988199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff987e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff988101f70 0x7ff988199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35324/0 (socket says 172.21.15.99:35324) 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff987e00700 1 -- 172.21.15.99:0/3137109015 learned_addr learned my addr 172.21.15.99:0/3137109015 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.322+0000 7ff98cc00700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff988100d80 0x7ff988198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:23.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff98d600700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff988102910 0x7ff9881a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff987e00700 1 -- 172.21.15.99:0/3137109015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff988102910 msgr2=0x7ff9881a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff987e00700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff988102910 0x7ff9881a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff987e00700 1 -- 172.21.15.99:0/3137109015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff988100d80 msgr2=0x7ff988198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff987e00700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff988100d80 0x7ff988198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff987e00700 1 -- 172.21.15.99:0/3137109015 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff980009d00 con 0x7ff988101f70 2024-03-30T00:33:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff987e00700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff988101f70 0x7ff988199480 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff97400d950 tx=0x7ff97400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff985600700 1 -- 172.21.15.99:0/3137109015 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974009940 con 0x7ff988101f70 2024-03-30T00:33:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff985600700 1 -- 172.21.15.99:0/3137109015 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff974010460 con 0x7ff988101f70 2024-03-30T00:33:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff985600700 1 -- 172.21.15.99:0/3137109015 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97400f620 con 0x7ff988101f70 2024-03-30T00:33:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff98f600700 1 -- 172.21.15.99:0/3137109015 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9881a0fc0 con 0x7ff988101f70 2024-03-30T00:33:23.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.323+0000 7ff98f600700 1 -- 172.21.15.99:0/3137109015 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9881a14a0 con 0x7ff988101f70 2024-03-30T00:33:23.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.324+0000 7ff98f600700 1 -- 172.21.15.99:0/3137109015 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff988067af0 con 0x7ff988101f70 2024-03-30T00:33:23.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.326+0000 7ff985600700 1 -- 172.21.15.99:0/3137109015 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff9740105d0 con 0x7ff988101f70 2024-03-30T00:33:23.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.327+0000 7ff985600700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff9700a1230 0x7ff9700a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:23.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.328+0000 7ff985600700 1 -- 172.21.15.99:0/3137109015 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff9740cc780 con 0x7ff988101f70 2024-03-30T00:33:23.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.328+0000 7ff98cc00700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff9700a1230 0x7ff9700a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:23.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.328+0000 7ff98cc00700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff9700a1230 0x7ff9700a36f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff988101dd0 tx=0x7ff97c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:23.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.332+0000 7ff985600700 1 -- 172.21.15.99:0/3137109015 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff974095e60 con 0x7ff988101f70 2024-03-30T00:33:23.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:23 smithi161 ceph-mon[42221]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:23.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:23 smithi161 ceph-mon[42221]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:23.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:23 smithi099 ceph-mon[38079]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:23.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:23 smithi099 ceph-mon[38079]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:23.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:23 smithi142 ceph-mon[41779]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:23.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:23 smithi142 ceph-mon[41779]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:23.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.576+0000 7ff98f600700 1 -- 172.21.15.99:0/3137109015 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff988074700 con 0x7ff9700a1230 2024-03-30T00:33:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.581+0000 7ff985600700 1 -- 172.21.15.99:0/3137109015 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7ff988074700 con 0x7ff9700a1230 2024-03-30T00:33:23.587 INFO:teuthology.orchestra.run.smithi099.stdout:/volumes/qa/sv_1/b2e51193-5841-4789-84a6-b82754af1fbb 2024-03-30T00:33:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 -- 172.21.15.99:0/3137109015 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff9700a1230 msgr2=0x7ff9700a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff9700a1230 0x7ff9700a36f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff988101dd0 tx=0x7ff97c009260 comp rx=0 tx=0).stop 2024-03-30T00:33:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 -- 172.21.15.99:0/3137109015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff988101f70 msgr2=0x7ff988199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff988101f70 0x7ff988199480 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff97400d950 tx=0x7ff97400dc60 comp rx=0 tx=0).stop 2024-03-30T00:33:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 -- 172.21.15.99:0/3137109015 shutdown_connections 2024-03-30T00:33:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff988102910 0x7ff9881a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff9700a1230 0x7ff9700a36f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff988100d80 0x7ff988198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 --2- 172.21.15.99:0/3137109015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff988101f70 0x7ff988199480 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 -- 172.21.15.99:0/3137109015 >> 172.21.15.99:0/3137109015 conn(0x7ff9880fc2f0 msgr2=0x7ff988072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:23.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 -- 172.21.15.99:0/3137109015 shutdown_connections 2024-03-30T00:33:23.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.583+0000 7ff98f600700 1 -- 172.21.15.99:0/3137109015 wait complete. 2024-03-30T00:33:23.609 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:33:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.798+0000 7f1e3a000700 1 -- 172.21.15.99:0/2513641504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34100c80 msgr2=0x7f1e341010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.798+0000 7f1e3a000700 1 --2- 172.21.15.99:0/2513641504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34100c80 0x7f1e341010a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e240099c0 tx=0x7f1e24009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.798+0000 7f1e3a000700 1 -- 172.21.15.99:0/2513641504 shutdown_connections 2024-03-30T00:33:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.798+0000 7f1e3a000700 1 --2- 172.21.15.99:0/2513641504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e34102810 0x7f1e341090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.798+0000 7f1e3a000700 1 --2- 172.21.15.99:0/2513641504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34100c80 0x7f1e341010a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.798+0000 7f1e3a000700 1 --2- 172.21.15.99:0/2513641504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e34101e70 0x7f1e341022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.798+0000 7f1e3a000700 1 -- 172.21.15.99:0/2513641504 >> 172.21.15.99:0/2513641504 conn(0x7f1e340fc1f0 msgr2=0x7f1e340fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:23.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.798+0000 7f1e3a000700 1 -- 172.21.15.99:0/2513641504 shutdown_connections 2024-03-30T00:33:23.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.798+0000 7f1e3a000700 1 -- 172.21.15.99:0/2513641504 wait complete. 2024-03-30T00:33:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.799+0000 7f1e3a000700 1 Processor -- start 2024-03-30T00:33:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.799+0000 7f1e3a000700 1 -- start start 2024-03-30T00:33:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.799+0000 7f1e3a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e34100c80 0x7f1e34198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.799+0000 7f1e3a000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e34101e70 0x7f1e341992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.799+0000 7f1e3a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34102810 0x7f1e341a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.799+0000 7f1e3a000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e341a0830 con 0x7f1e34102810 2024-03-30T00:33:23.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.799+0000 7f1e3a000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e341a09a0 con 0x7f1e34100c80 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.799+0000 7f1e3a000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e341a0b60 con 0x7f1e34101e70 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e33e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34102810 0x7f1e341a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e33e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34102810 0x7f1e341a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35352/0 (socket says 172.21.15.99:35352) 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e33e00700 1 -- 172.21.15.99:0/4201926887 learned_addr learned my addr 172.21.15.99:0/4201926887 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e33400700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e34100c80 0x7f1e34198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e32a00700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e34101e70 0x7f1e341992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e33e00700 1 -- 172.21.15.99:0/4201926887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e34101e70 msgr2=0x7f1e341992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e33e00700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e34101e70 0x7f1e341992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e33e00700 1 -- 172.21.15.99:0/4201926887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e34100c80 msgr2=0x7f1e34198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e33e00700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e34100c80 0x7f1e34198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e33e00700 1 -- 172.21.15.99:0/4201926887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e240095f0 con 0x7f1e34102810 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e33e00700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34102810 0x7f1e341a0260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c007d70 tx=0x7f1e2c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e2be00700 1 -- 172.21.15.99:0/4201926887 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00ad40 con 0x7f1e34102810 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e2be00700 1 -- 172.21.15.99:0/4201926887 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e2c0128b0 con 0x7f1e34102810 2024-03-30T00:33:23.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.800+0000 7f1e3a000700 1 -- 172.21.15.99:0/4201926887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e341a0e40 con 0x7f1e34102810 2024-03-30T00:33:23.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.801+0000 7f1e2be00700 1 -- 172.21.15.99:0/4201926887 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0115a0 con 0x7f1e34102810 2024-03-30T00:33:23.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.801+0000 7f1e3a000700 1 -- 172.21.15.99:0/4201926887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e341a1270 con 0x7f1e34102810 2024-03-30T00:33:23.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.801+0000 7f1e3a000700 1 -- 172.21.15.99:0/4201926887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e34067a10 con 0x7f1e34102810 2024-03-30T00:33:23.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.803+0000 7f1e2be00700 1 -- 172.21.15.99:0/4201926887 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1e2c012a20 con 0x7f1e34102810 2024-03-30T00:33:23.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.805+0000 7f1e2be00700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1e180a11b0 0x7f1e180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:23.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.805+0000 7f1e2be00700 1 -- 172.21.15.99:0/4201926887 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1e2c0ccb20 con 0x7f1e34102810 2024-03-30T00:33:23.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.805+0000 7f1e33400700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1e180a11b0 0x7f1e180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.805+0000 7f1e33400700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1e180a11b0 0x7f1e180a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1e24000c00 tx=0x7f1e24003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:23.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:23.809+0000 7f1e2be00700 1 -- 172.21.15.99:0/4201926887 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1e2c096310 con 0x7f1e34102810 2024-03-30T00:33:24.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.065+0000 7f1e3a000700 1 -- 172.21.15.99:0/4201926887 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e3410c590 con 0x7f1e34102810 2024-03-30T00:33:24.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.066+0000 7f1e2be00700 1 -- 172.21.15.99:0/4201926887 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10860 (secure 0 0 0) 0x7f1e2c016090 con 0x7f1e34102810 2024-03-30T00:33:24.073 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:24.073 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:33:21.119410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:33:24.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.068+0000 7f1e3a000700 1 -- 172.21.15.99:0/4201926887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1e180a11b0 msgr2=0x7f1e180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.068+0000 7f1e3a000700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1e180a11b0 0x7f1e180a3670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1e24000c00 tx=0x7f1e24003680 comp rx=0 tx=0).stop 2024-03-30T00:33:24.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.068+0000 7f1e3a000700 1 -- 172.21.15.99:0/4201926887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34102810 msgr2=0x7f1e341a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.069+0000 7f1e3a000700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34102810 0x7f1e341a0260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c007d70 tx=0x7f1e2c00a340 comp rx=0 tx=0).stop 2024-03-30T00:33:24.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.069+0000 7f1e3a000700 1 -- 172.21.15.99:0/4201926887 shutdown_connections 2024-03-30T00:33:24.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.069+0000 7f1e3a000700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e34102810 0x7f1e341a0260 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.069+0000 7f1e3a000700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1e180a11b0 0x7f1e180a3670 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.069+0000 7f1e3a000700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e34100c80 0x7f1e34198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.069+0000 7f1e3a000700 1 --2- 172.21.15.99:0/4201926887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e34101e70 0x7f1e341992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.069+0000 7f1e3a000700 1 -- 172.21.15.99:0/4201926887 >> 172.21.15.99:0/4201926887 conn(0x7f1e340fc1f0 msgr2=0x7f1e34072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:24.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.069+0000 7f1e3a000700 1 -- 172.21.15.99:0/4201926887 shutdown_connections 2024-03-30T00:33:24.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.069+0000 7f1e3a000700 1 -- 172.21.15.99:0/4201926887 wait complete. 2024-03-30T00:33:24.077 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2024-03-30T00:33:24.094 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:33:24.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.278+0000 7fa2d3400700 1 -- 172.21.15.99:0/1068189131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc074d90 msgr2=0x7fa2cc0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.278+0000 7fa2d3400700 1 --2- 172.21.15.99:0/1068189131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc074d90 0x7fa2cc0731e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009990 tx=0x7fa2c0009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.278+0000 7fa2d3400700 1 -- 172.21.15.99:0/1068189131 shutdown_connections 2024-03-30T00:33:24.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.278+0000 7fa2d3400700 1 --2- 172.21.15.99:0/1068189131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc108b60 0x7fa2cc10afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.278+0000 7fa2d3400700 1 --2- 172.21.15.99:0/1068189131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc074d90 0x7fa2cc0731e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.278+0000 7fa2d3400700 1 --2- 172.21.15.99:0/1068189131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc0737b0 0x7fa2cc073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.278+0000 7fa2d3400700 1 -- 172.21.15.99:0/1068189131 >> 172.21.15.99:0/1068189131 conn(0x7fa2cc0fc200 msgr2=0x7fa2cc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:24.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.279+0000 7fa2d3400700 1 -- 172.21.15.99:0/1068189131 shutdown_connections 2024-03-30T00:33:24.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.279+0000 7fa2d3400700 1 -- 172.21.15.99:0/1068189131 wait complete. 2024-03-30T00:33:24.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d3400700 1 Processor -- start 2024-03-30T00:33:24.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d3400700 1 -- start start 2024-03-30T00:33:24.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d3400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc0737b0 0x7fa2cc198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc074d90 0x7fa2cc199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc108b60 0x7fa2cc1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d3400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc1a0830 con 0x7fa2cc108b60 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d3400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc1a09a0 con 0x7fa2cc074d90 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d3400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc1a0b60 con 0x7fa2cc0737b0 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc108b60 0x7fa2cc1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc108b60 0x7fa2cc1a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35372/0 (socket says 172.21.15.99:35372) 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d1400700 1 -- 172.21.15.99:0/4114606688 learned_addr learned my addr 172.21.15.99:0/4114606688 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.280+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc0737b0 0x7fa2cc198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2cbe00700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc074d90 0x7fa2cc199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2d1400700 1 -- 172.21.15.99:0/4114606688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc0737b0 msgr2=0x7fa2cc198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2d1400700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc0737b0 0x7fa2cc198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2d1400700 1 -- 172.21.15.99:0/4114606688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc074d90 msgr2=0x7fa2cc199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2d1400700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc074d90 0x7fa2cc199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2d1400700 1 -- 172.21.15.99:0/4114606688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c00095f0 con 0x7fa2cc108b60 2024-03-30T00:33:24.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc0737b0 0x7fa2cc198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:33:24.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2d1400700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc108b60 0x7fa2cc1a0260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4007ee0 tx=0x7fa2c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:24.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2c9600700 1 -- 172.21.15.99:0/4114606688 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4017070 con 0x7fa2cc108b60 2024-03-30T00:33:24.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2c9600700 1 -- 172.21.15.99:0/4114606688 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2c400f040 con 0x7fa2cc108b60 2024-03-30T00:33:24.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2d3400700 1 -- 172.21.15.99:0/4114606688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc1a0e40 con 0x7fa2cc108b60 2024-03-30T00:33:24.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2c9600700 1 -- 172.21.15.99:0/4114606688 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4005710 con 0x7fa2cc108b60 2024-03-30T00:33:24.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.281+0000 7fa2d3400700 1 -- 172.21.15.99:0/4114606688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc1a1260 con 0x7fa2cc108b60 2024-03-30T00:33:24.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.282+0000 7fa2d3400700 1 -- 172.21.15.99:0/4114606688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2cc107970 con 0x7fa2cc108b60 2024-03-30T00:33:24.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.284+0000 7fa2c9600700 1 -- 172.21.15.99:0/4114606688 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa2c4005870 con 0x7fa2cc108b60 2024-03-30T00:33:24.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.286+0000 7fa2c9600700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a11b0 0x7fa2b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:24.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.286+0000 7fa2c9600700 1 -- 172.21.15.99:0/4114606688 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa2c40cc100 con 0x7fa2cc108b60 2024-03-30T00:33:24.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.286+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a11b0 0x7fa2b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:24.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.287+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a11b0 0x7fa2b40a3670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0000c00 tx=0x7fa2c0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:24.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.290+0000 7fa2c9600700 1 -- 172.21.15.99:0/4114606688 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa2c40957e0 con 0x7fa2cc108b60 2024-03-30T00:33:24.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:24 smithi161 ceph-mon[42221]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:24.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:24 smithi161 ceph-mon[42221]: pgmap v260: 97 pgs: 97 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.3 KiB/s wr, 15 op/s 2024-03-30T00:33:24.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:24 smithi161 ceph-mon[42221]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:24.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:24 smithi161 ceph-mon[42221]: mgrmap e17: x(active, since 6m), standbys: y, z 2024-03-30T00:33:24.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:24 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4201926887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:24.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:24 smithi099 ceph-mon[38079]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:24.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:24 smithi099 ceph-mon[38079]: pgmap v260: 97 pgs: 97 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.3 KiB/s wr, 15 op/s 2024-03-30T00:33:24.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:24 smithi099 ceph-mon[38079]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:24.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:24 smithi099 ceph-mon[38079]: mgrmap e17: x(active, since 6m), standbys: y, z 2024-03-30T00:33:24.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:24 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4201926887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:24.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.527+0000 7fa2d3400700 1 -- 172.21.15.99:0/4114606688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2cc10e570 con 0x7fa2cc108b60 2024-03-30T00:33:24.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.528+0000 7fa2c9600700 1 -- 172.21.15.99:0/4114606688 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22120 (secure 0 0 0) 0x7fa2c4098e00 con 0x7fa2cc108b60 2024-03-30T00:33:24.535 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:24.535 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":83,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:33:16.651168+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:33:24.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.530+0000 7fa2d3400700 1 -- 172.21.15.99:0/4114606688 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a11b0 msgr2=0x7fa2b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.530+0000 7fa2d3400700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a11b0 0x7fa2b40a3670 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0000c00 tx=0x7fa2c0005fb0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.530+0000 7fa2d3400700 1 -- 172.21.15.99:0/4114606688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc108b60 msgr2=0x7fa2cc1a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.530+0000 7fa2d3400700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc108b60 0x7fa2cc1a0260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4007ee0 tx=0x7fa2c400d3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.530+0000 7fa2d3400700 1 -- 172.21.15.99:0/4114606688 shutdown_connections 2024-03-30T00:33:24.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.530+0000 7fa2d3400700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc108b60 0x7fa2cc1a0260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.530+0000 7fa2d3400700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc0737b0 0x7fa2cc198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.530+0000 7fa2d3400700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a11b0 0x7fa2b40a3670 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.531+0000 7fa2d3400700 1 --2- 172.21.15.99:0/4114606688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc074d90 0x7fa2cc199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.531+0000 7fa2d3400700 1 -- 172.21.15.99:0/4114606688 >> 172.21.15.99:0/4114606688 conn(0x7fa2cc0fc200 msgr2=0x7fa2cc104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:24.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.531+0000 7fa2d3400700 1 -- 172.21.15.99:0/4114606688 shutdown_connections 2024-03-30T00:33:24.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.531+0000 7fa2d3400700 1 -- 172.21.15.99:0/4114606688 wait complete. 2024-03-30T00:33:24.558 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T00:33:24.559 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:33:24.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:24 smithi142 ceph-mon[41779]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:24.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:24 smithi142 ceph-mon[41779]: pgmap v260: 97 pgs: 97 active+clean; 581 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.3 KiB/s wr, 15 op/s 2024-03-30T00:33:24.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:24 smithi142 ceph-mon[41779]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:24.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:24 smithi142 ceph-mon[41779]: mgrmap e17: x(active, since 6m), standbys: y, z 2024-03-30T00:33:24.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:24 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4201926887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:33:24.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.742+0000 7f04c5000700 1 -- 172.21.15.99:0/576110169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04c0101e70 msgr2=0x7f04c01022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.742+0000 7f04c5000700 1 --2- 172.21.15.99:0/576110169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04c0101e70 0x7f04c01022d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0099c0 tx=0x7f04ac009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.742+0000 7f04c5000700 1 -- 172.21.15.99:0/576110169 shutdown_connections 2024-03-30T00:33:24.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.742+0000 7f04c5000700 1 --2- 172.21.15.99:0/576110169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c0102810 0x7f04c01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.742+0000 7f04c5000700 1 --2- 172.21.15.99:0/576110169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c0100c80 0x7f04c01010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.742+0000 7f04c5000700 1 --2- 172.21.15.99:0/576110169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04c0101e70 0x7f04c01022d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.742+0000 7f04c5000700 1 -- 172.21.15.99:0/576110169 >> 172.21.15.99:0/576110169 conn(0x7f04c00fc1f0 msgr2=0x7f04c00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:24.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.742+0000 7f04c5000700 1 -- 172.21.15.99:0/576110169 shutdown_connections 2024-03-30T00:33:24.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.742+0000 7f04c5000700 1 -- 172.21.15.99:0/576110169 wait complete. 2024-03-30T00:33:24.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.743+0000 7f04c5000700 1 Processor -- start 2024-03-30T00:33:24.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.743+0000 7f04c5000700 1 -- start start 2024-03-30T00:33:24.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04c5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04c0100c80 0x7f04c0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04c5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c0101e70 0x7f04c00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04c5000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c0102810 0x7f04c00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04c5000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c0073e80 con 0x7f04c0100c80 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04c5000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c0073ff0 con 0x7f04c0101e70 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04c5000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c01a0ba0 con 0x7f04c0102810 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04bde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04c0100c80 0x7f04c0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04bde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04c0100c80 0x7f04c0074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35386/0 (socket says 172.21.15.99:35386) 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04bde00700 1 -- 172.21.15.99:0/716074374 learned_addr learned my addr 172.21.15.99:0/716074374 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04be800700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c0102810 0x7f04c00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04bd400700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c0101e70 0x7f04c00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04bde00700 1 -- 172.21.15.99:0/716074374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c0102810 msgr2=0x7f04c00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04bde00700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c0102810 0x7f04c00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04bde00700 1 -- 172.21.15.99:0/716074374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c0101e70 msgr2=0x7f04c00731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04bde00700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c0101e70 0x7f04c00731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04bde00700 1 -- 172.21.15.99:0/716074374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0095f0 con 0x7f04c0100c80 2024-03-30T00:33:24.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04be800700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c0102810 0x7f04c00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:24.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.744+0000 7f04bde00700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04c0100c80 0x7f04c0074b10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04b000e970 tx=0x7f04b000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:24.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.745+0000 7f04b6a00700 1 -- 172.21.15.99:0/716074374 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0009790 con 0x7f04c0100c80 2024-03-30T00:33:24.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.745+0000 7f04b6a00700 1 -- 172.21.15.99:0/716074374 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04b0004020 con 0x7f04c0100c80 2024-03-30T00:33:24.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.745+0000 7f04c5000700 1 -- 172.21.15.99:0/716074374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c01a0e30 con 0x7f04c0100c80 2024-03-30T00:33:24.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.745+0000 7f04b6a00700 1 -- 172.21.15.99:0/716074374 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0003c50 con 0x7f04c0100c80 2024-03-30T00:33:24.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.745+0000 7f04c5000700 1 -- 172.21.15.99:0/716074374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c01a1250 con 0x7f04c0100c80 2024-03-30T00:33:24.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.746+0000 7f04c5000700 1 -- 172.21.15.99:0/716074374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c0067a80 con 0x7f04c0100c80 2024-03-30T00:33:24.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.748+0000 7f04b6a00700 1 -- 172.21.15.99:0/716074374 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f04b0004190 con 0x7f04c0100c80 2024-03-30T00:33:24.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.749+0000 7f04b6a00700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f04a40a11b0 0x7f04a40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:24.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.749+0000 7f04b6a00700 1 -- 172.21.15.99:0/716074374 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f04b0014070 con 0x7f04c0100c80 2024-03-30T00:33:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.749+0000 7f04bd400700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f04a40a11b0 0x7f04a40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.750+0000 7f04bd400700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f04a40a11b0 0x7f04a40a3670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b580 tx=0x7f04ac01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:24.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.753+0000 7f04b6a00700 1 -- 172.21.15.99:0/716074374 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f04c0067a80 con 0x7f04c0100c80 2024-03-30T00:33:24.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.956+0000 7f04c5000700 1 -- 172.21.15.99:0/716074374 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04c0067a10 con 0x7f04a40a11b0 2024-03-30T00:33:24.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.969+0000 7f04b6a00700 1 -- 172.21.15.99:0/716074374 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335157 (secure 0 0 0) 0x7f04c0067a10 con 0x7f04a40a11b0 2024-03-30T00:33:24.980 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:24.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.973+0000 7f04c5000700 1 -- 172.21.15.99:0/716074374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f04a40a11b0 msgr2=0x7f04a40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.973+0000 7f04c5000700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f04a40a11b0 0x7f04a40a3670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b580 tx=0x7f04ac01a040 comp rx=0 tx=0).stop 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.973+0000 7f04c5000700 1 -- 172.21.15.99:0/716074374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04c0100c80 msgr2=0x7f04c0074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.973+0000 7f04c5000700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04c0100c80 0x7f04c0074b10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04b000e970 tx=0x7f04b000ed30 comp rx=0 tx=0).stop 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.974+0000 7f04c5000700 1 -- 172.21.15.99:0/716074374 shutdown_connections 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.974+0000 7f04c5000700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04c0102810 0x7f04c00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.974+0000 7f04c5000700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04c0100c80 0x7f04c0074b10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.974+0000 7f04c5000700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f04a40a11b0 0x7f04a40a3670 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.974+0000 7f04c5000700 1 --2- 172.21.15.99:0/716074374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04c0101e70 0x7f04c00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.974+0000 7f04c5000700 1 -- 172.21.15.99:0/716074374 >> 172.21.15.99:0/716074374 conn(0x7f04c00fc1f0 msgr2=0x7f04c0105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.974+0000 7f04c5000700 1 -- 172.21.15.99:0/716074374 shutdown_connections 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:24.974+0000 7f04c5000700 1 -- 172.21.15.99:0/716074374 wait complete. 2024-03-30T00:33:24.984 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-30T00:33:24.999 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-03-30T00:33:23.008867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594692,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":144,"num_read_kb":124,"num_write":285,"num_write_kb":2972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":247,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377600,"kb_used_data":12412,"kb_used_omap":18,"kb_used_meta":365037,"kb_avail":1124564224,"statfs":{"total":1151940427776,"available":1151553765376,"internally_reserved":0,"allocated":12709888,"data_stored":6929418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":373798214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,8,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4324,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":104,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.861994"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290853+0000","last_change":"2024-03-30T00:33:15.686814+0000","last_active":"2024-03-30T00:33:17.290853+0000","last_peered":"2024-03-30T00:33:17.290853+0000","last_clean":"2024-03-30T00:33:17.290853+0000","last_became_active":"2024-03-30T00:33:15.686432+0000","last_became_peered":"2024-03-30T00:33:15.686432+0000","last_unstale":"2024-03-30T00:33:17.290853+0000","last_undegraded":"2024-03-30T00:33:17.290853+0000","last_fullsized":"2024-03-30T00:33:17.290853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:35:05.898973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002227+0000","last_change":"2024-03-30T00:33:15.678089+0000","last_active":"2024-03-30T00:33:17.002227+0000","last_peered":"2024-03-30T00:33:17.002227+0000","last_clean":"2024-03-30T00:33:17.002227+0000","last_became_active":"2024-03-30T00:33:15.677254+0000","last_became_peered":"2024-03-30T00:33:15.677254+0000","last_unstale":"2024-03-30T00:33:17.002227+0000","last_undegraded":"2024-03-30T00:33:17.002227+0000","last_fullsized":"2024-03-30T00:33:17.002227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:45.922652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002176+0000","last_change":"2024-03-30T00:33:15.686777+0000","last_active":"2024-03-30T00:33:17.002176+0000","last_peered":"2024-03-30T00:33:17.002176+0000","last_clean":"2024-03-30T00:33:17.002176+0000","last_became_active":"2024-03-30T00:33:15.686461+0000","last_became_peered":"2024-03-30T00:33:15.686461+0000","last_unstale":"2024-03-30T00:33:17.002176+0000","last_undegraded":"2024-03-30T00:33:17.002176+0000","last_fullsized":"2024-03-30T00:33:17.002176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:33:27.073089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002223+0000","last_change":"2024-03-30T00:33:15.681237+0000","last_active":"2024-03-30T00:33:17.002223+0000","last_peered":"2024-03-30T00:33:17.002223+0000","last_clean":"2024-03-30T00:33:17.002223+0000","last_became_active":"2024-03-30T00:33:15.680669+0000","last_became_peered":"2024-03-30T00:33:15.680669+0000","last_unstale":"2024-03-30T00:33:17.002223+0000","last_undegraded":"2024-03-30T00:33:17.002223+0000","last_fullsized":"2024-03-30T00:33:17.002223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:59.300262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.682286+0000","last_change":"2024-03-30T00:33:15.678825+0000","last_active":"2024-03-30T00:33:15.682286+0000","last_peered":"2024-03-30T00:33:15.682286+0000","last_clean":"2024-03-30T00:33:15.682286+0000","last_became_active":"2024-03-30T00:33:15.676864+0000","last_became_peered":"2024-03-30T00:33:15.676864+0000","last_unstale":"2024-03-30T00:33:15.682286+0000","last_undegraded":"2024-03-30T00:33:15.682286+0000","last_fullsized":"2024-03-30T00:33:15.682286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:12:09.724441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.688863+0000","last_change":"2024-03-30T00:33:15.681866+0000","last_active":"2024-03-30T00:33:15.688863+0000","last_peered":"2024-03-30T00:33:15.688863+0000","last_clean":"2024-03-30T00:33:15.688863+0000","last_became_active":"2024-03-30T00:33:15.678118+0000","last_became_peered":"2024-03-30T00:33:15.678118+0000","last_unstale":"2024-03-30T00:33:15.688863+0000","last_undegraded":"2024-03-30T00:33:15.688863+0000","last_fullsized":"2024-03-30T00:33:15.688863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:24.433027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001193+0000","last_change":"2024-03-30T00:33:15.675999+0000","last_active":"2024-03-30T00:33:17.001193+0000","last_peered":"2024-03-30T00:33:17.001193+0000","last_clean":"2024-03-30T00:33:17.001193+0000","last_became_active":"2024-03-30T00:33:15.675319+0000","last_became_peered":"2024-03-30T00:33:15.675319+0000","last_unstale":"2024-03-30T00:33:17.001193+0000","last_undegraded":"2024-03-30T00:33:17.001193+0000","last_fullsized":"2024-03-30T00:33:17.001193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:42.668747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000423+0000","last_change":"2024-03-30T00:33:15.679211+0000","last_active":"2024-03-30T00:33:17.000423+0000","last_peered":"2024-03-30T00:33:17.000423+0000","last_clean":"2024-03-30T00:33:17.000423+0000","last_became_active":"2024-03-30T00:33:15.675061+0000","last_became_peered":"2024-03-30T00:33:15.675061+0000","last_unstale":"2024-03-30T00:33:17.000423+0000","last_undegraded":"2024-03-30T00:33:17.000423+0000","last_fullsized":"2024-03-30T00:33:17.000423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:43.973542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.692417+0000","last_change":"2024-03-30T00:33:15.680842+0000","last_active":"2024-03-30T00:33:15.692417+0000","last_peered":"2024-03-30T00:33:15.692417+0000","last_clean":"2024-03-30T00:33:15.692417+0000","last_became_active":"2024-03-30T00:33:15.680307+0000","last_became_peered":"2024-03-30T00:33:15.680307+0000","last_unstale":"2024-03-30T00:33:15.692417+0000","last_undegraded":"2024-03-30T00:33:15.692417+0000","last_fullsized":"2024-03-30T00:33:15.692417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:59:40.331779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693556+0000","last_change":"2024-03-30T00:33:15.692397+0000","last_active":"2024-03-30T00:33:15.693556+0000","last_peered":"2024-03-30T00:33:15.693556+0000","last_clean":"2024-03-30T00:33:15.693556+0000","last_became_active":"2024-03-30T00:33:15.691226+0000","last_became_peered":"2024-03-30T00:33:15.691226+0000","last_unstale":"2024-03-30T00:33:15.693556+0000","last_undegraded":"2024-03-30T00:33:15.693556+0000","last_fullsized":"2024-03-30T00:33:15.693556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:48.816644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001314+0000","last_change":"2024-03-30T00:33:15.688417+0000","last_active":"2024-03-30T00:33:17.001314+0000","last_peered":"2024-03-30T00:33:17.001314+0000","last_clean":"2024-03-30T00:33:17.001314+0000","last_became_active":"2024-03-30T00:33:15.687755+0000","last_became_peered":"2024-03-30T00:33:15.687755+0000","last_unstale":"2024-03-30T00:33:17.001314+0000","last_undegraded":"2024-03-30T00:33:17.001314+0000","last_fullsized":"2024-03-30T00:33:17.001314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:30:19.782280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002119+0000","last_change":"2024-03-30T00:33:15.681942+0000","last_active":"2024-03-30T00:33:17.002119+0000","last_peered":"2024-03-30T00:33:17.002119+0000","last_clean":"2024-03-30T00:33:17.002119+0000","last_became_active":"2024-03-30T00:33:15.679854+0000","last_became_peered":"2024-03-30T00:33:15.679854+0000","last_unstale":"2024-03-30T00:33:17.002119+0000","last_undegraded":"2024-03-30T00:33:17.002119+0000","last_fullsized":"2024-03-30T00:33:17.002119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:19.771959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001108+0000","last_change":"2024-03-30T00:33:15.682968+0000","last_active":"2024-03-30T00:33:17.001108+0000","last_peered":"2024-03-30T00:33:17.001108+0000","last_clean":"2024-03-30T00:33:17.001108+0000","last_became_active":"2024-03-30T00:33:15.681216+0000","last_became_peered":"2024-03-30T00:33:15.681216+0000","last_unstale":"2024-03-30T00:33:17.001108+0000","last_undegraded":"2024-03-30T00:33:17.001108+0000","last_fullsized":"2024-03-30T00:33:17.001108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:50.038843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001697+0000","last_change":"2024-03-30T00:33:15.677727+0000","last_active":"2024-03-30T00:33:17.001697+0000","last_peered":"2024-03-30T00:33:17.001697+0000","last_clean":"2024-03-30T00:33:17.001697+0000","last_became_active":"2024-03-30T00:33:15.676758+0000","last_became_peered":"2024-03-30T00:33:15.676758+0000","last_unstale":"2024-03-30T00:33:17.001697+0000","last_undegraded":"2024-03-30T00:33:17.001697+0000","last_fullsized":"2024-03-30T00:33:17.001697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:34.745920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001691+0000","last_change":"2024-03-30T00:33:15.684545+0000","last_active":"2024-03-30T00:33:17.001691+0000","last_peered":"2024-03-30T00:33:17.001691+0000","last_clean":"2024-03-30T00:33:17.001691+0000","last_became_active":"2024-03-30T00:33:15.682989+0000","last_became_peered":"2024-03-30T00:33:15.682989+0000","last_unstale":"2024-03-30T00:33:17.001691+0000","last_undegraded":"2024-03-30T00:33:17.001691+0000","last_fullsized":"2024-03-30T00:33:17.001691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:11.120981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.686822+0000","last_change":"2024-03-30T00:33:15.678122+0000","last_active":"2024-03-30T00:33:15.686822+0000","last_peered":"2024-03-30T00:33:15.686822+0000","last_clean":"2024-03-30T00:33:15.686822+0000","last_became_active":"2024-03-30T00:33:15.677681+0000","last_became_peered":"2024-03-30T00:33:15.677681+0000","last_unstale":"2024-03-30T00:33:15.686822+0000","last_undegraded":"2024-03-30T00:33:15.686822+0000","last_fullsized":"2024-03-30T00:33:15.686822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:50.454125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290980+0000","last_change":"2024-03-30T00:33:15.687689+0000","last_active":"2024-03-30T00:33:17.290980+0000","last_peered":"2024-03-30T00:33:17.290980+0000","last_clean":"2024-03-30T00:33:17.290980+0000","last_became_active":"2024-03-30T00:33:15.686009+0000","last_became_peered":"2024-03-30T00:33:15.686009+0000","last_unstale":"2024-03-30T00:33:17.290980+0000","last_undegraded":"2024-03-30T00:33:17.290980+0000","last_fullsized":"2024-03-30T00:33:17.290980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:17.234749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659641+0000","last_change":"2024-03-30T00:33:15.686053+0000","last_active":"2024-03-30T00:33:16.659641+0000","last_peered":"2024-03-30T00:33:16.659641+0000","last_clean":"2024-03-30T00:33:16.659641+0000","last_became_active":"2024-03-30T00:33:15.685723+0000","last_became_peered":"2024-03-30T00:33:15.685723+0000","last_unstale":"2024-03-30T00:33:16.659641+0000","last_undegraded":"2024-03-30T00:33:16.659641+0000","last_fullsized":"2024-03-30T00:33:16.659641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:27.322834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660740+0000","last_change":"2024-03-30T00:33:15.674783+0000","last_active":"2024-03-30T00:33:16.660740+0000","last_peered":"2024-03-30T00:33:16.660740+0000","last_clean":"2024-03-30T00:33:16.660740+0000","last_became_active":"2024-03-30T00:33:15.674152+0000","last_became_peered":"2024-03-30T00:33:15.674152+0000","last_unstale":"2024-03-30T00:33:16.660740+0000","last_undegraded":"2024-03-30T00:33:16.660740+0000","last_fullsized":"2024-03-30T00:33:16.660740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:39.517240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.119665+0000","last_change":"2024-03-30T00:33:15.172217+0000","last_active":"2024-03-30T00:33:20.119665+0000","last_peered":"2024-03-30T00:33:20.119665+0000","last_clean":"2024-03-30T00:33:20.119665+0000","last_became_active":"2024-03-30T00:33:15.171718+0000","last_became_peered":"2024-03-30T00:33:15.171718+0000","last_unstale":"2024-03-30T00:33:20.119665+0000","last_undegraded":"2024-03-30T00:33:20.119665+0000","last_fullsized":"2024-03-30T00:33:20.119665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:11.691572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001530+0000","last_change":"2024-03-30T00:33:15.669937+0000","last_active":"2024-03-30T00:33:17.001530+0000","last_peered":"2024-03-30T00:33:17.001530+0000","last_clean":"2024-03-30T00:33:17.001530+0000","last_became_active":"2024-03-30T00:33:15.668704+0000","last_became_peered":"2024-03-30T00:33:15.668704+0000","last_unstale":"2024-03-30T00:33:17.001530+0000","last_undegraded":"2024-03-30T00:33:17.001530+0000","last_fullsized":"2024-03-30T00:33:17.001530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:23.914755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001552+0000","last_change":"2024-03-30T00:33:15.173663+0000","last_active":"2024-03-30T00:33:17.001552+0000","last_peered":"2024-03-30T00:33:17.001552+0000","last_clean":"2024-03-30T00:33:17.001552+0000","last_became_active":"2024-03-30T00:33:15.173288+0000","last_became_peered":"2024-03-30T00:33:15.173288+0000","last_unstale":"2024-03-30T00:33:17.001552+0000","last_undegraded":"2024-03-30T00:33:17.001552+0000","last_fullsized":"2024-03-30T00:33:17.001552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.435142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002383+0000","last_change":"2024-03-30T00:33:15.686757+0000","last_active":"2024-03-30T00:33:17.002383+0000","last_peered":"2024-03-30T00:33:17.002383+0000","last_clean":"2024-03-30T00:33:17.002383+0000","last_became_active":"2024-03-30T00:33:15.686306+0000","last_became_peered":"2024-03-30T00:33:15.686306+0000","last_unstale":"2024-03-30T00:33:17.002383+0000","last_undegraded":"2024-03-30T00:33:17.002383+0000","last_fullsized":"2024-03-30T00:33:17.002383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:50:21.048212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.117995+0000","last_change":"2024-03-30T00:33:15.173402+0000","last_active":"2024-03-30T00:33:20.117995+0000","last_peered":"2024-03-30T00:33:20.117995+0000","last_clean":"2024-03-30T00:33:20.117995+0000","last_became_active":"2024-03-30T00:33:15.172995+0000","last_became_peered":"2024-03-30T00:33:15.172995+0000","last_unstale":"2024-03-30T00:33:20.117995+0000","last_undegraded":"2024-03-30T00:33:20.117995+0000","last_fullsized":"2024-03-30T00:33:20.117995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:25.694792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.691152+0000","last_change":"2024-03-30T00:33:15.688292+0000","last_active":"2024-03-30T00:33:15.691152+0000","last_peered":"2024-03-30T00:33:15.691152+0000","last_clean":"2024-03-30T00:33:15.691152+0000","last_became_active":"2024-03-30T00:33:15.682722+0000","last_became_peered":"2024-03-30T00:33:15.682722+0000","last_unstale":"2024-03-30T00:33:15.691152+0000","last_undegraded":"2024-03-30T00:33:15.691152+0000","last_fullsized":"2024-03-30T00:33:15.691152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:52:10.074656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.692186+0000","last_change":"2024-03-30T00:33:15.162316+0000","last_active":"2024-03-30T00:33:15.692186+0000","last_peered":"2024-03-30T00:33:15.692186+0000","last_clean":"2024-03-30T00:33:15.692186+0000","last_became_active":"2024-03-30T00:33:15.161805+0000","last_became_peered":"2024-03-30T00:33:15.161805+0000","last_unstale":"2024-03-30T00:33:15.692186+0000","last_undegraded":"2024-03-30T00:33:15.692186+0000","last_fullsized":"2024-03-30T00:33:15.692186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:26:16.470245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.532008+0000","last_change":"2024-03-30T00:33:15.675442+0000","last_active":"2024-03-30T00:33:16.532008+0000","last_peered":"2024-03-30T00:33:16.532008+0000","last_clean":"2024-03-30T00:33:16.532008+0000","last_became_active":"2024-03-30T00:33:15.675068+0000","last_became_peered":"2024-03-30T00:33:15.675068+0000","last_unstale":"2024-03-30T00:33:16.532008+0000","last_undegraded":"2024-03-30T00:33:16.532008+0000","last_fullsized":"2024-03-30T00:33:16.532008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:36:31.353262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.107636+0000","last_change":"2024-03-30T00:33:15.172087+0000","last_active":"2024-03-30T00:33:20.107636+0000","last_peered":"2024-03-30T00:33:20.107636+0000","last_clean":"2024-03-30T00:33:20.107636+0000","last_became_active":"2024-03-30T00:33:15.171768+0000","last_became_peered":"2024-03-30T00:33:15.171768+0000","last_unstale":"2024-03-30T00:33:20.107636+0000","last_undegraded":"2024-03-30T00:33:20.107636+0000","last_fullsized":"2024-03-30T00:33:20.107636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:17.198995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.668850+0000","last_change":"2024-03-30T00:33:15.665219+0000","last_active":"2024-03-30T00:33:15.668850+0000","last_peered":"2024-03-30T00:33:15.668850+0000","last_clean":"2024-03-30T00:33:15.668850+0000","last_became_active":"2024-03-30T00:33:15.664935+0000","last_became_peered":"2024-03-30T00:33:15.664935+0000","last_unstale":"2024-03-30T00:33:15.668850+0000","last_undegraded":"2024-03-30T00:33:15.668850+0000","last_fullsized":"2024-03-30T00:33:15.668850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:37.123937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.116508+0000","last_change":"2024-03-30T00:33:15.165668+0000","last_active":"2024-03-30T00:33:20.116508+0000","last_peered":"2024-03-30T00:33:20.116508+0000","last_clean":"2024-03-30T00:33:20.116508+0000","last_became_active":"2024-03-30T00:33:15.164637+0000","last_became_peered":"2024-03-30T00:33:15.164637+0000","last_unstale":"2024-03-30T00:33:20.116508+0000","last_undegraded":"2024-03-30T00:33:20.116508+0000","last_fullsized":"2024-03-30T00:33:20.116508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:47:59.389414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531981+0000","last_change":"2024-03-30T00:33:15.662357+0000","last_active":"2024-03-30T00:33:16.531981+0000","last_peered":"2024-03-30T00:33:16.531981+0000","last_clean":"2024-03-30T00:33:16.531981+0000","last_became_active":"2024-03-30T00:33:15.662011+0000","last_became_peered":"2024-03-30T00:33:15.662011+0000","last_unstale":"2024-03-30T00:33:16.531981+0000","last_undegraded":"2024-03-30T00:33:16.531981+0000","last_fullsized":"2024-03-30T00:33:16.531981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:18.386054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000942+0000","last_change":"2024-03-30T00:33:15.177269+0000","last_active":"2024-03-30T00:33:17.000942+0000","last_peered":"2024-03-30T00:33:17.000942+0000","last_clean":"2024-03-30T00:33:17.000942+0000","last_became_active":"2024-03-30T00:33:15.176937+0000","last_became_peered":"2024-03-30T00:33:15.176937+0000","last_unstale":"2024-03-30T00:33:17.000942+0000","last_undegraded":"2024-03-30T00:33:17.000942+0000","last_fullsized":"2024-03-30T00:33:17.000942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:44.658775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000691+0000","last_change":"2024-03-30T00:33:15.675832+0000","last_active":"2024-03-30T00:33:17.000691+0000","last_peered":"2024-03-30T00:33:17.000691+0000","last_clean":"2024-03-30T00:33:17.000691+0000","last_became_active":"2024-03-30T00:33:15.675481+0000","last_became_peered":"2024-03-30T00:33:15.675481+0000","last_unstale":"2024-03-30T00:33:17.000691+0000","last_undegraded":"2024-03-30T00:33:17.000691+0000","last_fullsized":"2024-03-30T00:33:17.000691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:11.638056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001044+0000","last_change":"2024-03-30T00:33:15.171238+0000","last_active":"2024-03-30T00:33:17.001044+0000","last_peered":"2024-03-30T00:33:17.001044+0000","last_clean":"2024-03-30T00:33:17.001044+0000","last_became_active":"2024-03-30T00:33:15.170931+0000","last_became_peered":"2024-03-30T00:33:15.170931+0000","last_unstale":"2024-03-30T00:33:17.001044+0000","last_undegraded":"2024-03-30T00:33:17.001044+0000","last_fullsized":"2024-03-30T00:33:17.001044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:43:50.178363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659336+0000","last_change":"2024-03-30T00:33:15.676014+0000","last_active":"2024-03-30T00:33:16.659336+0000","last_peered":"2024-03-30T00:33:16.659336+0000","last_clean":"2024-03-30T00:33:16.659336+0000","last_became_active":"2024-03-30T00:33:15.675553+0000","last_became_peered":"2024-03-30T00:33:15.675553+0000","last_unstale":"2024-03-30T00:33:16.659336+0000","last_undegraded":"2024-03-30T00:33:16.659336+0000","last_fullsized":"2024-03-30T00:33:16.659336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:17.248242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.133355+0000","last_change":"2024-03-30T00:33:15.177033+0000","last_active":"2024-03-30T00:33:19.133355+0000","last_peered":"2024-03-30T00:33:19.133355+0000","last_clean":"2024-03-30T00:33:19.133355+0000","last_became_active":"2024-03-30T00:33:15.176602+0000","last_became_peered":"2024-03-30T00:33:15.176602+0000","last_unstale":"2024-03-30T00:33:19.133355+0000","last_undegraded":"2024-03-30T00:33:19.133355+0000","last_fullsized":"2024-03-30T00:33:19.133355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:10.804704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.691801+0000","last_change":"2024-03-30T00:33:15.683388+0000","last_active":"2024-03-30T00:33:15.691801+0000","last_peered":"2024-03-30T00:33:15.691801+0000","last_clean":"2024-03-30T00:33:15.691801+0000","last_became_active":"2024-03-30T00:33:15.682976+0000","last_became_peered":"2024-03-30T00:33:15.682976+0000","last_unstale":"2024-03-30T00:33:15.691801+0000","last_undegraded":"2024-03-30T00:33:15.691801+0000","last_fullsized":"2024-03-30T00:33:15.691801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:50:34.907265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001036+0000","last_change":"2024-03-30T00:33:15.167819+0000","last_active":"2024-03-30T00:33:17.001036+0000","last_peered":"2024-03-30T00:33:17.001036+0000","last_clean":"2024-03-30T00:33:17.001036+0000","last_became_active":"2024-03-30T00:33:15.167405+0000","last_became_peered":"2024-03-30T00:33:15.167405+0000","last_unstale":"2024-03-30T00:33:17.001036+0000","last_undegraded":"2024-03-30T00:33:17.001036+0000","last_fullsized":"2024-03-30T00:33:17.001036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:37.793792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002345+0000","last_change":"2024-03-30T00:33:15.690907+0000","last_active":"2024-03-30T00:33:17.002345+0000","last_peered":"2024-03-30T00:33:17.002345+0000","last_clean":"2024-03-30T00:33:17.002345+0000","last_became_active":"2024-03-30T00:33:15.690046+0000","last_became_peered":"2024-03-30T00:33:15.690046+0000","last_unstale":"2024-03-30T00:33:17.002345+0000","last_undegraded":"2024-03-30T00:33:17.002345+0000","last_fullsized":"2024-03-30T00:33:17.002345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:24.319685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.132456+0000","last_change":"2024-03-30T00:33:15.166633+0000","last_active":"2024-03-30T00:33:19.132456+0000","last_peered":"2024-03-30T00:33:19.132456+0000","last_clean":"2024-03-30T00:33:19.132456+0000","last_became_active":"2024-03-30T00:33:15.166308+0000","last_became_peered":"2024-03-30T00:33:15.166308+0000","last_unstale":"2024-03-30T00:33:19.132456+0000","last_undegraded":"2024-03-30T00:33:19.132456+0000","last_fullsized":"2024-03-30T00:33:19.132456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:53.102872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693949+0000","last_change":"2024-03-30T00:33:15.691880+0000","last_active":"2024-03-30T00:33:15.693949+0000","last_peered":"2024-03-30T00:33:15.693949+0000","last_clean":"2024-03-30T00:33:15.693949+0000","last_became_active":"2024-03-30T00:33:15.690780+0000","last_became_peered":"2024-03-30T00:33:15.690780+0000","last_unstale":"2024-03-30T00:33:15.693949+0000","last_undegraded":"2024-03-30T00:33:15.693949+0000","last_fullsized":"2024-03-30T00:33:15.693949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:06.532709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.107184+0000","last_change":"2024-03-30T00:33:15.164287+0000","last_active":"2024-03-30T00:33:20.107184+0000","last_peered":"2024-03-30T00:33:20.107184+0000","last_clean":"2024-03-30T00:33:20.107184+0000","last_became_active":"2024-03-30T00:33:15.163557+0000","last_became_peered":"2024-03-30T00:33:15.163557+0000","last_unstale":"2024-03-30T00:33:20.107184+0000","last_undegraded":"2024-03-30T00:33:20.107184+0000","last_fullsized":"2024-03-30T00:33:20.107184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:41.117936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.683827+0000","last_change":"2024-03-30T00:33:15.682841+0000","last_active":"2024-03-30T00:33:15.683827+0000","last_peered":"2024-03-30T00:33:15.683827+0000","last_clean":"2024-03-30T00:33:15.683827+0000","last_became_active":"2024-03-30T00:33:15.682237+0000","last_became_peered":"2024-03-30T00:33:15.682237+0000","last_unstale":"2024-03-30T00:33:15.683827+0000","last_undegraded":"2024-03-30T00:33:15.683827+0000","last_fullsized":"2024-03-30T00:33:15.683827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:59:05.719216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118639+0000","last_change":"2024-03-30T00:33:15.165193+0000","last_active":"2024-03-30T00:33:20.118639+0000","last_peered":"2024-03-30T00:33:20.118639+0000","last_clean":"2024-03-30T00:33:20.118639+0000","last_became_active":"2024-03-30T00:33:15.161741+0000","last_became_peered":"2024-03-30T00:33:15.161741+0000","last_unstale":"2024-03-30T00:33:20.118639+0000","last_undegraded":"2024-03-30T00:33:20.118639+0000","last_fullsized":"2024-03-30T00:33:20.118639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:22:11.891064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001296+0000","last_change":"2024-03-30T00:33:15.680929+0000","last_active":"2024-03-30T00:33:17.001296+0000","last_peered":"2024-03-30T00:33:17.001296+0000","last_clean":"2024-03-30T00:33:17.001296+0000","last_became_active":"2024-03-30T00:33:15.676671+0000","last_became_peered":"2024-03-30T00:33:15.676671+0000","last_unstale":"2024-03-30T00:33:17.001296+0000","last_undegraded":"2024-03-30T00:33:17.001296+0000","last_fullsized":"2024-03-30T00:33:17.001296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:28:33.819135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.132787+0000","last_change":"2024-03-30T00:33:15.168029+0000","last_active":"2024-03-30T00:33:19.132787+0000","last_peered":"2024-03-30T00:33:19.132787+0000","last_clean":"2024-03-30T00:33:19.132787+0000","last_became_active":"2024-03-30T00:33:15.167636+0000","last_became_peered":"2024-03-30T00:33:15.167636+0000","last_unstale":"2024-03-30T00:33:19.132787+0000","last_undegraded":"2024-03-30T00:33:19.132787+0000","last_fullsized":"2024-03-30T00:33:19.132787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:25:47.834696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001171+0000","last_change":"2024-03-30T00:33:15.687977+0000","last_active":"2024-03-30T00:33:17.001171+0000","last_peered":"2024-03-30T00:33:17.001171+0000","last_clean":"2024-03-30T00:33:17.001171+0000","last_became_active":"2024-03-30T00:33:15.687491+0000","last_became_peered":"2024-03-30T00:33:15.687491+0000","last_unstale":"2024-03-30T00:33:17.001171+0000","last_undegraded":"2024-03-30T00:33:17.001171+0000","last_fullsized":"2024-03-30T00:33:17.001171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:08:55.568039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.657267+0000","last_change":"2024-03-30T00:33:15.171826+0000","last_active":"2024-03-30T00:33:15.657267+0000","last_peered":"2024-03-30T00:33:15.657267+0000","last_clean":"2024-03-30T00:33:15.657267+0000","last_became_active":"2024-03-30T00:33:15.170749+0000","last_became_peered":"2024-03-30T00:33:15.170749+0000","last_unstale":"2024-03-30T00:33:15.657267+0000","last_undegraded":"2024-03-30T00:33:15.657267+0000","last_fullsized":"2024-03-30T00:33:15.657267+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:57:52.034530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000714+0000","last_change":"2024-03-30T00:33:15.681668+0000","last_active":"2024-03-30T00:33:17.000714+0000","last_peered":"2024-03-30T00:33:17.000714+0000","last_clean":"2024-03-30T00:33:17.000714+0000","last_became_active":"2024-03-30T00:33:15.681129+0000","last_became_peered":"2024-03-30T00:33:15.681129+0000","last_unstale":"2024-03-30T00:33:17.000714+0000","last_undegraded":"2024-03-30T00:33:17.000714+0000","last_fullsized":"2024-03-30T00:33:17.000714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:51:19.552243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118585+0000","last_change":"2024-03-30T00:33:15.171016+0000","last_active":"2024-03-30T00:33:20.118585+0000","last_peered":"2024-03-30T00:33:20.118585+0000","last_clean":"2024-03-30T00:33:20.118585+0000","last_became_active":"2024-03-30T00:33:15.170058+0000","last_became_peered":"2024-03-30T00:33:15.170058+0000","last_unstale":"2024-03-30T00:33:20.118585+0000","last_undegraded":"2024-03-30T00:33:20.118585+0000","last_fullsized":"2024-03-30T00:33:20.118585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:07:39.910286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531944+0000","last_change":"2024-03-30T00:33:15.687012+0000","last_active":"2024-03-30T00:33:16.531944+0000","last_peered":"2024-03-30T00:33:16.531944+0000","last_clean":"2024-03-30T00:33:16.531944+0000","last_became_active":"2024-03-30T00:33:15.684030+0000","last_became_peered":"2024-03-30T00:33:15.684030+0000","last_unstale":"2024-03-30T00:33:16.531944+0000","last_undegraded":"2024-03-30T00:33:16.531944+0000","last_fullsized":"2024-03-30T00:33:16.531944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:32.447119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001625+0000","last_change":"2024-03-30T00:33:15.163633+0000","last_active":"2024-03-30T00:33:17.001625+0000","last_peered":"2024-03-30T00:33:17.001625+0000","last_clean":"2024-03-30T00:33:17.001625+0000","last_became_active":"2024-03-30T00:33:15.161339+0000","last_became_peered":"2024-03-30T00:33:15.161339+0000","last_unstale":"2024-03-30T00:33:17.001625+0000","last_undegraded":"2024-03-30T00:33:17.001625+0000","last_fullsized":"2024-03-30T00:33:17.001625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:15:25.194650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290780+0000","last_change":"2024-03-30T00:33:15.680617+0000","last_active":"2024-03-30T00:33:17.290780+0000","last_peered":"2024-03-30T00:33:17.290780+0000","last_clean":"2024-03-30T00:33:17.290780+0000","last_became_active":"2024-03-30T00:33:15.679137+0000","last_became_peered":"2024-03-30T00:33:15.679137+0000","last_unstale":"2024-03-30T00:33:17.290780+0000","last_undegraded":"2024-03-30T00:33:17.290780+0000","last_fullsized":"2024-03-30T00:33:17.290780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:34.449218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659518+0000","last_change":"2024-03-30T00:33:15.177273+0000","last_active":"2024-03-30T00:33:16.659518+0000","last_peered":"2024-03-30T00:33:16.659518+0000","last_clean":"2024-03-30T00:33:16.659518+0000","last_became_active":"2024-03-30T00:33:15.175333+0000","last_became_peered":"2024-03-30T00:33:15.175333+0000","last_unstale":"2024-03-30T00:33:16.659518+0000","last_undegraded":"2024-03-30T00:33:16.659518+0000","last_fullsized":"2024-03-30T00:33:16.659518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:27:03.305671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.685478+0000","last_change":"2024-03-30T00:33:15.674746+0000","last_active":"2024-03-30T00:33:15.685478+0000","last_peered":"2024-03-30T00:33:15.685478+0000","last_clean":"2024-03-30T00:33:15.685478+0000","last_became_active":"2024-03-30T00:33:15.674388+0000","last_became_peered":"2024-03-30T00:33:15.674388+0000","last_unstale":"2024-03-30T00:33:15.685478+0000","last_undegraded":"2024-03-30T00:33:15.685478+0000","last_fullsized":"2024-03-30T00:33:15.685478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:21:15.024181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000989+0000","last_change":"2024-03-30T00:33:15.173640+0000","last_active":"2024-03-30T00:33:17.000989+0000","last_peered":"2024-03-30T00:33:17.000989+0000","last_clean":"2024-03-30T00:33:17.000989+0000","last_became_active":"2024-03-30T00:33:15.173104+0000","last_became_peered":"2024-03-30T00:33:15.173104+0000","last_unstale":"2024-03-30T00:33:17.000989+0000","last_undegraded":"2024-03-30T00:33:17.000989+0000","last_fullsized":"2024-03-30T00:33:17.000989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:20:14.880730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.683575+0000","last_change":"2024-03-30T00:33:15.677824+0000","last_active":"2024-03-30T00:33:15.683575+0000","last_peered":"2024-03-30T00:33:15.683575+0000","last_clean":"2024-03-30T00:33:15.683575+0000","last_became_active":"2024-03-30T00:33:15.677435+0000","last_became_peered":"2024-03-30T00:33:15.677435+0000","last_unstale":"2024-03-30T00:33:15.683575+0000","last_undegraded":"2024-03-30T00:33:15.683575+0000","last_fullsized":"2024-03-30T00:33:15.683575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:22.992989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531931+0000","last_change":"2024-03-30T00:33:15.667428+0000","last_active":"2024-03-30T00:33:16.531931+0000","last_peered":"2024-03-30T00:33:16.531931+0000","last_clean":"2024-03-30T00:33:16.531931+0000","last_became_active":"2024-03-30T00:33:15.666634+0000","last_became_peered":"2024-03-30T00:33:15.666634+0000","last_unstale":"2024-03-30T00:33:16.531931+0000","last_undegraded":"2024-03-30T00:33:16.531931+0000","last_fullsized":"2024-03-30T00:33:16.531931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:56:14.710178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000860+0000","last_change":"2024-03-30T00:33:15.173264+0000","last_active":"2024-03-30T00:33:17.000860+0000","last_peered":"2024-03-30T00:33:17.000860+0000","last_clean":"2024-03-30T00:33:17.000860+0000","last_became_active":"2024-03-30T00:33:15.172932+0000","last_became_peered":"2024-03-30T00:33:15.172932+0000","last_unstale":"2024-03-30T00:33:17.000860+0000","last_undegraded":"2024-03-30T00:33:17.000860+0000","last_fullsized":"2024-03-30T00:33:17.000860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:18:25.079284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000756+0000","last_change":"2024-03-30T00:33:15.686606+0000","last_active":"2024-03-30T00:33:17.000756+0000","last_peered":"2024-03-30T00:33:17.000756+0000","last_clean":"2024-03-30T00:33:17.000756+0000","last_became_active":"2024-03-30T00:33:15.686076+0000","last_became_peered":"2024-03-30T00:33:15.686076+0000","last_unstale":"2024-03-30T00:33:17.000756+0000","last_undegraded":"2024-03-30T00:33:17.000756+0000","last_fullsized":"2024-03-30T00:33:17.000756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:46.962659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.133123+0000","last_change":"2024-03-30T00:33:15.177263+0000","last_active":"2024-03-30T00:33:19.133123+0000","last_peered":"2024-03-30T00:33:19.133123+0000","last_clean":"2024-03-30T00:33:19.133123+0000","last_became_active":"2024-03-30T00:33:15.176743+0000","last_became_peered":"2024-03-30T00:33:15.176743+0000","last_unstale":"2024-03-30T00:33:19.133123+0000","last_undegraded":"2024-03-30T00:33:19.133123+0000","last_fullsized":"2024-03-30T00:33:19.133123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:56:47.491371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":322,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.669557+0000","last_change":"2024-03-30T00:32:12.678730+0000","last_active":"2024-03-30T00:33:15.669557+0000","last_peered":"2024-03-30T00:33:15.669557+0000","last_clean":"2024-03-30T00:33:15.669557+0000","last_became_active":"2024-03-30T00:32:12.678256+0000","last_became_peered":"2024-03-30T00:32:12.678256+0000","last_unstale":"2024-03-30T00:33:15.669557+0000","last_undegraded":"2024-03-30T00:33:15.669557+0000","last_fullsized":"2024-03-30T00:33:15.669557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_clean_scrub_stamp":"2024-03-30T00:29:17.768535+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:16.229072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000775+0000","last_change":"2024-03-30T00:33:15.685230+0000","last_active":"2024-03-30T00:33:17.000775+0000","last_peered":"2024-03-30T00:33:17.000775+0000","last_clean":"2024-03-30T00:33:17.000775+0000","last_became_active":"2024-03-30T00:33:15.684263+0000","last_became_peered":"2024-03-30T00:33:15.684263+0000","last_unstale":"2024-03-30T00:33:17.000775+0000","last_undegraded":"2024-03-30T00:33:17.000775+0000","last_fullsized":"2024-03-30T00:33:17.000775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:22:51.980318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000955+0000","last_change":"2024-03-30T00:33:15.167490+0000","last_active":"2024-03-30T00:33:17.000955+0000","last_peered":"2024-03-30T00:33:17.000955+0000","last_clean":"2024-03-30T00:33:17.000955+0000","last_became_active":"2024-03-30T00:33:15.166888+0000","last_became_peered":"2024-03-30T00:33:15.166888+0000","last_unstale":"2024-03-30T00:33:17.000955+0000","last_undegraded":"2024-03-30T00:33:17.000955+0000","last_fullsized":"2024-03-30T00:33:17.000955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:00:32.571281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131306+0000","last_change":"2024-03-30T00:33:15.178388+0000","last_active":"2024-03-30T00:33:19.131306+0000","last_peered":"2024-03-30T00:33:19.131306+0000","last_clean":"2024-03-30T00:33:19.131306+0000","last_became_active":"2024-03-30T00:33:15.178160+0000","last_became_peered":"2024-03-30T00:33:15.178160+0000","last_unstale":"2024-03-30T00:33:19.131306+0000","last_undegraded":"2024-03-30T00:33:19.131306+0000","last_fullsized":"2024-03-30T00:33:19.131306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:28:51.567598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001200+0000","last_change":"2024-03-30T00:33:15.675426+0000","last_active":"2024-03-30T00:33:17.001200+0000","last_peered":"2024-03-30T00:33:17.001200+0000","last_clean":"2024-03-30T00:33:17.001200+0000","last_became_active":"2024-03-30T00:33:15.675083+0000","last_became_peered":"2024-03-30T00:33:15.675083+0000","last_unstale":"2024-03-30T00:33:17.001200+0000","last_undegraded":"2024-03-30T00:33:17.001200+0000","last_fullsized":"2024-03-30T00:33:17.001200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:37:03.086331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118827+0000","last_change":"2024-03-30T00:33:15.171537+0000","last_active":"2024-03-30T00:33:20.118827+0000","last_peered":"2024-03-30T00:33:20.118827+0000","last_clean":"2024-03-30T00:33:20.118827+0000","last_became_active":"2024-03-30T00:33:15.171128+0000","last_became_peered":"2024-03-30T00:33:15.171128+0000","last_unstale":"2024-03-30T00:33:20.118827+0000","last_undegraded":"2024-03-30T00:33:20.118827+0000","last_fullsized":"2024-03-30T00:33:20.118827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:52.062144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001730+0000","last_change":"2024-03-30T00:33:15.674075+0000","last_active":"2024-03-30T00:33:17.001730+0000","last_peered":"2024-03-30T00:33:17.001730+0000","last_clean":"2024-03-30T00:33:17.001730+0000","last_became_active":"2024-03-30T00:33:15.672400+0000","last_became_peered":"2024-03-30T00:33:15.672400+0000","last_unstale":"2024-03-30T00:33:17.001730+0000","last_undegraded":"2024-03-30T00:33:17.001730+0000","last_fullsized":"2024-03-30T00:33:17.001730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:59.309835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131339+0000","last_change":"2024-03-30T00:33:15.168245+0000","last_active":"2024-03-30T00:33:19.131339+0000","last_peered":"2024-03-30T00:33:19.131339+0000","last_clean":"2024-03-30T00:33:19.131339+0000","last_became_active":"2024-03-30T00:33:15.167852+0000","last_became_peered":"2024-03-30T00:33:15.167852+0000","last_unstale":"2024-03-30T00:33:19.131339+0000","last_undegraded":"2024-03-30T00:33:19.131339+0000","last_fullsized":"2024-03-30T00:33:19.131339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:33:28.780614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002270+0000","last_change":"2024-03-30T00:33:15.679802+0000","last_active":"2024-03-30T00:33:17.002270+0000","last_peered":"2024-03-30T00:33:17.002270+0000","last_clean":"2024-03-30T00:33:17.002270+0000","last_became_active":"2024-03-30T00:33:15.679468+0000","last_became_peered":"2024-03-30T00:33:15.679468+0000","last_unstale":"2024-03-30T00:33:17.002270+0000","last_undegraded":"2024-03-30T00:33:17.002270+0000","last_fullsized":"2024-03-30T00:33:17.002270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:03:03.223713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131563+0000","last_change":"2024-03-30T00:33:15.170052+0000","last_active":"2024-03-30T00:33:19.131563+0000","last_peered":"2024-03-30T00:33:19.131563+0000","last_clean":"2024-03-30T00:33:19.131563+0000","last_became_active":"2024-03-30T00:33:15.169706+0000","last_became_peered":"2024-03-30T00:33:15.169706+0000","last_unstale":"2024-03-30T00:33:19.131563+0000","last_undegraded":"2024-03-30T00:33:19.131563+0000","last_fullsized":"2024-03-30T00:33:19.131563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:32.043071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.668729+0000","last_change":"2024-03-30T00:33:15.667061+0000","last_active":"2024-03-30T00:33:15.668729+0000","last_peered":"2024-03-30T00:33:15.668729+0000","last_clean":"2024-03-30T00:33:15.668729+0000","last_became_active":"2024-03-30T00:33:15.666763+0000","last_became_peered":"2024-03-30T00:33:15.666763+0000","last_unstale":"2024-03-30T00:33:15.668729+0000","last_undegraded":"2024-03-30T00:33:15.668729+0000","last_fullsized":"2024-03-30T00:33:15.668729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:46.558297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000531+0000","last_change":"2024-03-30T00:33:15.176152+0000","last_active":"2024-03-30T00:33:17.000531+0000","last_peered":"2024-03-30T00:33:17.000531+0000","last_clean":"2024-03-30T00:33:17.000531+0000","last_became_active":"2024-03-30T00:33:15.175640+0000","last_became_peered":"2024-03-30T00:33:15.175640+0000","last_unstale":"2024-03-30T00:33:17.000531+0000","last_undegraded":"2024-03-30T00:33:17.000531+0000","last_fullsized":"2024-03-30T00:33:17.000531+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:58:11.575577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290545+0000","last_change":"2024-03-30T00:33:15.672957+0000","last_active":"2024-03-30T00:33:17.290545+0000","last_peered":"2024-03-30T00:33:17.290545+0000","last_clean":"2024-03-30T00:33:17.290545+0000","last_became_active":"2024-03-30T00:33:15.668868+0000","last_became_peered":"2024-03-30T00:33:15.668868+0000","last_unstale":"2024-03-30T00:33:17.290545+0000","last_undegraded":"2024-03-30T00:33:17.290545+0000","last_fullsized":"2024-03-30T00:33:17.290545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:50.342693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.787067+0000","last_change":"2024-03-30T00:33:15.160345+0000","last_active":"2024-03-30T00:33:16.787067+0000","last_peered":"2024-03-30T00:33:16.787067+0000","last_clean":"2024-03-30T00:33:16.787067+0000","last_became_active":"2024-03-30T00:33:15.160026+0000","last_became_peered":"2024-03-30T00:33:15.160026+0000","last_unstale":"2024-03-30T00:33:16.787067+0000","last_undegraded":"2024-03-30T00:33:16.787067+0000","last_fullsized":"2024-03-30T00:33:16.787067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:06:51.177680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001362+0000","last_change":"2024-03-30T00:33:15.674058+0000","last_active":"2024-03-30T00:33:17.001362+0000","last_peered":"2024-03-30T00:33:17.001362+0000","last_clean":"2024-03-30T00:33:17.001362+0000","last_became_active":"2024-03-30T00:33:15.673165+0000","last_became_peered":"2024-03-30T00:33:15.673165+0000","last_unstale":"2024-03-30T00:33:17.001362+0000","last_undegraded":"2024-03-30T00:33:17.001362+0000","last_fullsized":"2024-03-30T00:33:17.001362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:14.869963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131675+0000","last_change":"2024-03-30T00:33:15.173178+0000","last_active":"2024-03-30T00:33:19.131675+0000","last_peered":"2024-03-30T00:33:19.131675+0000","last_clean":"2024-03-30T00:33:19.131675+0000","last_became_active":"2024-03-30T00:33:15.172830+0000","last_became_peered":"2024-03-30T00:33:15.172830+0000","last_unstale":"2024-03-30T00:33:19.131675+0000","last_undegraded":"2024-03-30T00:33:19.131675+0000","last_fullsized":"2024-03-30T00:33:19.131675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:49:09.823855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001435+0000","last_change":"2024-03-30T00:33:15.693485+0000","last_active":"2024-03-30T00:33:17.001435+0000","last_peered":"2024-03-30T00:33:17.001435+0000","last_clean":"2024-03-30T00:33:17.001435+0000","last_became_active":"2024-03-30T00:33:15.693066+0000","last_became_peered":"2024-03-30T00:33:15.693066+0000","last_unstale":"2024-03-30T00:33:17.001435+0000","last_undegraded":"2024-03-30T00:33:17.001435+0000","last_fullsized":"2024-03-30T00:33:17.001435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:38.161331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118339+0000","last_change":"2024-03-30T00:33:15.170808+0000","last_active":"2024-03-30T00:33:20.118339+0000","last_peered":"2024-03-30T00:33:20.118339+0000","last_clean":"2024-03-30T00:33:20.118339+0000","last_became_active":"2024-03-30T00:33:15.170448+0000","last_became_peered":"2024-03-30T00:33:15.170448+0000","last_unstale":"2024-03-30T00:33:20.118339+0000","last_undegraded":"2024-03-30T00:33:20.118339+0000","last_fullsized":"2024-03-30T00:33:20.118339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:08:28.179041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001385+0000","last_change":"2024-03-30T00:33:15.687619+0000","last_active":"2024-03-30T00:33:17.001385+0000","last_peered":"2024-03-30T00:33:17.001385+0000","last_clean":"2024-03-30T00:33:17.001385+0000","last_became_active":"2024-03-30T00:33:15.687313+0000","last_became_peered":"2024-03-30T00:33:15.687313+0000","last_unstale":"2024-03-30T00:33:17.001385+0000","last_undegraded":"2024-03-30T00:33:17.001385+0000","last_fullsized":"2024-03-30T00:33:17.001385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:36:51.182558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001678+0000","last_change":"2024-03-30T00:33:15.168601+0000","last_active":"2024-03-30T00:33:17.001678+0000","last_peered":"2024-03-30T00:33:17.001678+0000","last_clean":"2024-03-30T00:33:17.001678+0000","last_became_active":"2024-03-30T00:33:15.168065+0000","last_became_peered":"2024-03-30T00:33:15.168065+0000","last_unstale":"2024-03-30T00:33:17.001678+0000","last_undegraded":"2024-03-30T00:33:17.001678+0000","last_fullsized":"2024-03-30T00:33:17.001678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:22:36.477280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000984+0000","last_change":"2024-03-30T00:33:15.678299+0000","last_active":"2024-03-30T00:33:17.000984+0000","last_peered":"2024-03-30T00:33:17.000984+0000","last_clean":"2024-03-30T00:33:17.000984+0000","last_became_active":"2024-03-30T00:33:15.675499+0000","last_became_peered":"2024-03-30T00:33:15.675499+0000","last_unstale":"2024-03-30T00:33:17.000984+0000","last_undegraded":"2024-03-30T00:33:17.000984+0000","last_fullsized":"2024-03-30T00:33:17.000984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:15.317125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000985+0000","last_change":"2024-03-30T00:33:15.172058+0000","last_active":"2024-03-30T00:33:17.000985+0000","last_peered":"2024-03-30T00:33:17.000985+0000","last_clean":"2024-03-30T00:33:17.000985+0000","last_became_active":"2024-03-30T00:33:15.167857+0000","last_became_peered":"2024-03-30T00:33:15.167857+0000","last_unstale":"2024-03-30T00:33:17.000985+0000","last_undegraded":"2024-03-30T00:33:17.000985+0000","last_fullsized":"2024-03-30T00:33:17.000985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:03:35.951559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001350+0000","last_change":"2024-03-30T00:33:15.682519+0000","last_active":"2024-03-30T00:33:17.001350+0000","last_peered":"2024-03-30T00:33:17.001350+0000","last_clean":"2024-03-30T00:33:17.001350+0000","last_became_active":"2024-03-30T00:33:15.681055+0000","last_became_peered":"2024-03-30T00:33:15.681055+0000","last_unstale":"2024-03-30T00:33:17.001350+0000","last_undegraded":"2024-03-30T00:33:17.001350+0000","last_fullsized":"2024-03-30T00:33:17.001350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:38.160151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290650+0000","last_change":"2024-03-30T00:33:15.673411+0000","last_active":"2024-03-30T00:33:17.290650+0000","last_peered":"2024-03-30T00:33:17.290650+0000","last_clean":"2024-03-30T00:33:17.290650+0000","last_became_active":"2024-03-30T00:33:15.672852+0000","last_became_peered":"2024-03-30T00:33:15.672852+0000","last_unstale":"2024-03-30T00:33:17.290650+0000","last_undegraded":"2024-03-30T00:33:17.290650+0000","last_fullsized":"2024-03-30T00:33:17.290650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:55:46.705785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660735+0000","last_change":"2024-03-30T00:33:15.680624+0000","last_active":"2024-03-30T00:33:16.660735+0000","last_peered":"2024-03-30T00:33:16.660735+0000","last_clean":"2024-03-30T00:33:16.660735+0000","last_became_active":"2024-03-30T00:33:15.679900+0000","last_became_peered":"2024-03-30T00:33:15.679900+0000","last_unstale":"2024-03-30T00:33:16.660735+0000","last_undegraded":"2024-03-30T00:33:16.660735+0000","last_fullsized":"2024-03-30T00:33:16.660735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:44.484608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660521+0000","last_change":"2024-03-30T00:33:15.672612+0000","last_active":"2024-03-30T00:33:16.660521+0000","last_peered":"2024-03-30T00:33:16.660521+0000","last_clean":"2024-03-30T00:33:16.660521+0000","last_became_active":"2024-03-30T00:33:15.671536+0000","last_became_peered":"2024-03-30T00:33:15.671536+0000","last_unstale":"2024-03-30T00:33:16.660521+0000","last_undegraded":"2024-03-30T00:33:16.660521+0000","last_fullsized":"2024-03-30T00:33:16.660521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:40.783988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002065+0000","last_change":"2024-03-30T00:33:15.681577+0000","last_active":"2024-03-30T00:33:17.002065+0000","last_peered":"2024-03-30T00:33:17.002065+0000","last_clean":"2024-03-30T00:33:17.002065+0000","last_became_active":"2024-03-30T00:33:15.681313+0000","last_became_peered":"2024-03-30T00:33:15.681313+0000","last_unstale":"2024-03-30T00:33:17.002065+0000","last_undegraded":"2024-03-30T00:33:17.002065+0000","last_fullsized":"2024-03-30T00:33:17.002065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:42:15.178957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001296+0000","last_change":"2024-03-30T00:33:15.688172+0000","last_active":"2024-03-30T00:33:17.001296+0000","last_peered":"2024-03-30T00:33:17.001296+0000","last_clean":"2024-03-30T00:33:17.001296+0000","last_became_active":"2024-03-30T00:33:15.687860+0000","last_became_peered":"2024-03-30T00:33:15.687860+0000","last_unstale":"2024-03-30T00:33:17.001296+0000","last_undegraded":"2024-03-30T00:33:17.001296+0000","last_fullsized":"2024-03-30T00:33:17.001296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:28.762798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290827+0000","last_change":"2024-03-30T00:33:15.673868+0000","last_active":"2024-03-30T00:33:17.290827+0000","last_peered":"2024-03-30T00:33:17.290827+0000","last_clean":"2024-03-30T00:33:17.290827+0000","last_became_active":"2024-03-30T00:33:15.673467+0000","last_became_peered":"2024-03-30T00:33:15.673467+0000","last_unstale":"2024-03-30T00:33:17.290827+0000","last_undegraded":"2024-03-30T00:33:17.290827+0000","last_fullsized":"2024-03-30T00:33:17.290827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:07:39.448967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.661014+0000","last_change":"2024-03-30T00:33:15.684639+0000","last_active":"2024-03-30T00:33:16.661014+0000","last_peered":"2024-03-30T00:33:16.661014+0000","last_clean":"2024-03-30T00:33:16.661014+0000","last_became_active":"2024-03-30T00:33:15.684357+0000","last_became_peered":"2024-03-30T00:33:15.684357+0000","last_unstale":"2024-03-30T00:33:16.661014+0000","last_undegraded":"2024-03-30T00:33:16.661014+0000","last_fullsized":"2024-03-30T00:33:16.661014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:02:50.995640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001730+0000","last_change":"2024-03-30T00:33:15.680810+0000","last_active":"2024-03-30T00:33:17.001730+0000","last_peered":"2024-03-30T00:33:17.001730+0000","last_clean":"2024-03-30T00:33:17.001730+0000","last_became_active":"2024-03-30T00:33:15.680457+0000","last_became_peered":"2024-03-30T00:33:15.680457+0000","last_unstale":"2024-03-30T00:33:17.001730+0000","last_undegraded":"2024-03-30T00:33:17.001730+0000","last_fullsized":"2024-03-30T00:33:17.001730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:02.575683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531880+0000","last_change":"2024-03-30T00:33:15.661644+0000","last_active":"2024-03-30T00:33:16.531880+0000","last_peered":"2024-03-30T00:33:16.531880+0000","last_clean":"2024-03-30T00:33:16.531880+0000","last_became_active":"2024-03-30T00:33:15.661359+0000","last_became_peered":"2024-03-30T00:33:15.661359+0000","last_unstale":"2024-03-30T00:33:16.531880+0000","last_undegraded":"2024-03-30T00:33:16.531880+0000","last_fullsized":"2024-03-30T00:33:16.531880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:44:39.894342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000646+0000","last_change":"2024-03-30T00:33:15.671491+0000","last_active":"2024-03-30T00:33:17.000646+0000","last_peered":"2024-03-30T00:33:17.000646+0000","last_clean":"2024-03-30T00:33:17.000646+0000","last_became_active":"2024-03-30T00:33:15.671110+0000","last_became_peered":"2024-03-30T00:33:15.671110+0000","last_unstale":"2024-03-30T00:33:17.000646+0000","last_undegraded":"2024-03-30T00:33:17.000646+0000","last_fullsized":"2024-03-30T00:33:17.000646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:57.623730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693121+0000","last_change":"2024-03-30T00:33:15.686795+0000","last_active":"2024-03-30T00:33:15.693121+0000","last_peered":"2024-03-30T00:33:15.693121+0000","last_clean":"2024-03-30T00:33:15.693121+0000","last_became_active":"2024-03-30T00:33:15.685715+0000","last_became_peered":"2024-03-30T00:33:15.685715+0000","last_unstale":"2024-03-30T00:33:15.693121+0000","last_undegraded":"2024-03-30T00:33:15.693121+0000","last_fullsized":"2024-03-30T00:33:15.693121+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:37:53.838823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001008+0000","last_change":"2024-03-30T00:33:15.687369+0000","last_active":"2024-03-30T00:33:17.001008+0000","last_peered":"2024-03-30T00:33:17.001008+0000","last_clean":"2024-03-30T00:33:17.001008+0000","last_became_active":"2024-03-30T00:33:15.686852+0000","last_became_peered":"2024-03-30T00:33:15.686852+0000","last_unstale":"2024-03-30T00:33:17.001008+0000","last_undegraded":"2024-03-30T00:33:17.001008+0000","last_fullsized":"2024-03-30T00:33:17.001008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:09:21.003739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290614+0000","last_change":"2024-03-30T00:33:15.676840+0000","last_active":"2024-03-30T00:33:17.290614+0000","last_peered":"2024-03-30T00:33:17.290614+0000","last_clean":"2024-03-30T00:33:17.290614+0000","last_became_active":"2024-03-30T00:33:15.675884+0000","last_became_peered":"2024-03-30T00:33:15.675884+0000","last_unstale":"2024-03-30T00:33:17.290614+0000","last_undegraded":"2024-03-30T00:33:17.290614+0000","last_fullsized":"2024-03-30T00:33:17.290614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:02.900843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4324,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":104,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":18490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":901120,"data_stored":429559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8129999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8300000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.79}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0009999999999999}]}]},{"osd":10,"up_from":71,"seq":304942678033,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1523712,"data_stored":1021584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,8,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":1,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.8160000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":1.994},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":1.879}]},{"osd":3,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.153},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":1.075}]},{"osd":4,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":1.696},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":1.1160000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":1.7430000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":2.1970000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":2.153},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":2.181}]},{"osd":7,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.97999999999999998}]},{"osd":8,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":2.2360000000000002},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.1679999999999999}]},{"osd":9,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":2.3210000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":1.2090000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]}]},{"osd":9,"up_from":66,"seq":283467841556,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":429795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.433},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.44800000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.84899999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.47199999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.73399999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.626},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.78200000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.63700000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.89000000000000001}]},{"osd":8,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.86799999999999999}]},{"osd":10,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.91200000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103193,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":429603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59399999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.998}]},{"osd":2,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63200000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.66200000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":1.0640000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.94899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":1.04},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.92900000000000005}]},{"osd":9,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":1.079}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":1.097}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":7,"up_from":50,"seq":214748364829,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1495040,"data_stored":1019927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.40799999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.0049999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.74299999999999999,"15min":0.73199999999999998},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.80900000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.67100000000000004,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.56499999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0760000000000001,"5min":1.095,"15min":1.095},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.79600000000000004,"15min":0.79100000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1679999999999999,"5min":1.252,"15min":1.252},"last":0.92100000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.79900000000000004,"15min":0.79400000000000004},"min":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.85799999999999998}]},{"osd":8,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.90900000000000003}]},{"osd":9,"last update":"Sat Mar 30 00:32:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.79100000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.93500000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528353,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":897024,"data_stored":428977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.34100000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63300000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.38},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.41399999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66200000000000003,"5min":0.873,"15min":0.873},"last":0.35499999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.628,"15min":0.622},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.52500000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84899999999999998,"5min":0.879,"15min":0.879},"last":0.42899999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.55700000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.98799999999999999,"5min":1.002,"15min":1.002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97899999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.46600000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.442,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.505},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.64500000000000002,"15min":0.63300000000000001},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.92400000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.45500000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.60899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97199999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.749,"15min":0.749},"min":{"1min":0.51500000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":1.238,"15min":1.238},"last":0.58799999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.80100000000000005,"15min":0.78800000000000003},"min":{"1min":0.64400000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.71399999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.55900000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:33:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.67000000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0169999999999999,"5min":1.105,"15min":1.105},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.63100000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.70999999999999996}]},{"osd":8,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.65600000000000003}]},{"osd":9,"last update":"Sat Mar 30 00:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.626},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.72799999999999998}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.69699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36096,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709056,"statfs":{"total":95995035648,"available":95958073344,"internally_reserved":0,"allocated":909312,"data_stored":430541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.63700000000000001,"15min":0.59599999999999997},"min":{"1min":0.44700000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.624}]},{"osd":2,"last update":"Sat Mar 30 00:32:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.503,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88300000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.625,"15min":0.59099999999999997},"min":{"1min":0.45300000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.84799999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.621,"15min":0.58399999999999996},"min":{"1min":0.50800000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.64500000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.50600000000000001,"15min":0.48299999999999998},"min":{"1min":0.40999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.53700000000000003,"15min":0.505},"min":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.51900000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.749},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.68400000000000005,"15min":0.65700000000000003},"min":{"1min":0.52400000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3039999999999998,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.879}]},{"osd":6,"last update":"Sat Mar 30 00:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.70599999999999996,"15min":0.69299999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78400000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.64500000000000002,"15min":0.629},"min":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.65800000000000003,"15min":0.64700000000000002},"min":{"1min":0.504,"5min":0.438,"15min":0.438},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.80300000000000005}]},{"osd":8,"last update":"Sat Mar 30 00:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1,"5min":1,"15min":1},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.3570000000000002,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.95899999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.72699999999999998}]},{"osd":10,"last update":"Sat Mar 30 00:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.03}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738428,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":901120,"data_stored":429067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59099999999999997,"15min":0.57199999999999995},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66600000000000004,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96699999999999997,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.78400000000000003}]},{"osd":2,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68600000000000005,"15min":0.628},"min":{"1min":0.432,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0549999999999999,"5min":5.8550000000000004,"15min":5.8550000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.77000000000000002,"15min":0.82799999999999996},"min":{"1min":0.49199999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97799999999999998,"5min":5.282,"15min":5.282},"last":0.55800000000000005}]},{"osd":3,"last update":"Sat Mar 30 00:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.83299999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.47999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.67900000000000005,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.53000000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.91100000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.92400000000000004,"15min":0.95799999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.57199999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:32:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.95099999999999996,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.629,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98099999999999998,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.45700000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.125,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2470000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.81399999999999995}]},{"osd":8,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.505},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.69099999999999995}]},{"osd":9,"last update":"Sat Mar 30 00:32:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.65300000000000002}]},{"osd":10,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.97499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36120,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709032,"statfs":{"total":95995035648,"available":95958048768,"internally_reserved":0,"allocated":933888,"data_stored":431843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92700000000000005,"5min":1.131,"15min":1.131},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80100000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.66300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81899999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.439,"5min":0.375,"15min":0.375},"max":{"1min":0.96299999999999997,"5min":1.006,"15min":1.006},"last":0.94099999999999995}]},{"osd":3,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.435,"5min":0.371,"15min":0.371},"max":{"1min":0.89000000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71999999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.878},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.65700000000000003,"15min":0.61899999999999999},"min":{"1min":0.64900000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.0449999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.40100000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.58299999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.755,"15min":0.73699999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70099999999999996,"15min":0.68999999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.65700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.83299999999999996}]},{"osd":8,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.97999999999999998}]},{"osd":9,"last update":"Sat Mar 30 00:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.016},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.89000000000000001}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.077}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":3,"up_from":24,"seq":103079215149,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":429081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.39700000000000002},"min":{"1min":0.35799999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.44600000000000001,"15min":0.42999999999999999},"min":{"1min":0.40200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64400000000000002,"5min":0.748,"15min":0.748},"last":0.72699999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.44400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.76500000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.57799999999999996},"min":{"1min":0.503,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82399999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.53300000000000003}]},{"osd":2,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.627,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94299999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.48299999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64800000000000002,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86699999999999999,"5min":0.996,"15min":0.996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.66400000000000003,"15min":0.63},"min":{"1min":0.58199999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:32:24 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.56899999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.70799999999999996,"15min":0.68700000000000006},"min":{"1min":0.60799999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97899999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:32:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.47999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98199999999999998,"5min":1.202,"15min":1.202},"last":0.56399999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.81100000000000005,"15min":0.79900000000000004},"min":{"1min":0.63300000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76800000000000002}]},{"osd":8,"last update":"Sat Mar 30 00:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.66700000000000004}]},{"osd":9,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.624}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.749},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.65600000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986218,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":901120,"data_stored":428999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.497}]},{"osd":1,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.47799999999999998,"15min":0.44900000000000001},"min":{"1min":0.35799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.47199999999999998,"15min":0.45000000000000001},"min":{"1min":0.39600000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59699999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.84199999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.67900000000000005,"15min":0.65100000000000002},"min":{"1min":0.58699999999999997,"5min":0.439,"15min":0.439},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.68700000000000006,"15min":0.66300000000000003},"min":{"1min":0.498,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.625}]},{"osd":5,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.46899999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96299999999999997,"5min":1.137,"15min":1.137},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67100000000000004,"15min":0.66100000000000003},"min":{"1min":0.54300000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.95499999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69799999999999995,"15min":0.67900000000000005},"min":{"1min":0.56699999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.45600000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.073},{"interface":"front","average":{"1min":0.745,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.52400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.99199999999999999}]},{"osd":8,"last update":"Sat Mar 30 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":1.018}]},{"osd":9,"last update":"Sat Mar 30 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":1}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":1.087},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.034}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789991,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32592,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712560,"statfs":{"total":95995035648,"available":95961661440,"internally_reserved":0,"allocated":1515520,"data_stored":1020442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.81999999999999995,"15min":0.84799999999999998},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0009999999999999,"5min":8.077,"15min":8.077},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69199999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86099999999999999,"5min":0.878,"15min":0.878},"last":0.437},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63500000000000001,"15min":0.629},"min":{"1min":0.48599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.73799999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.38600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88900000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.86599999999999999,"15min":0.90500000000000003},"min":{"1min":0.436,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86199999999999999,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":0.54700000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.54200000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.49199999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97699999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58699999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.61499999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.76600000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:33:08 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.99399999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71299999999999997,"15min":0.70799999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.623}]},{"osd":8,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64500000000000002}]},{"osd":9,"last update":"Sat Mar 30 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67800000000000005}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:33:25.007 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:33:25.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.227+0000 7f52c2e00700 1 -- 172.21.15.99:0/1686640327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52bc106970 msgr2=0x7f52bc108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:25.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.227+0000 7f52c2e00700 1 --2- 172.21.15.99:0/1686640327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52bc106970 0x7f52bc108de0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f52b400b0a0 tx=0x7f52b400b3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.227+0000 7f52c2e00700 1 -- 172.21.15.99:0/1686640327 shutdown_connections 2024-03-30T00:33:25.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.227+0000 7f52c2e00700 1 --2- 172.21.15.99:0/1686640327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52bc106970 0x7f52bc108de0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.227+0000 7f52c2e00700 1 --2- 172.21.15.99:0/1686640327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52bc101710 0x7f52bc103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.227+0000 7f52c2e00700 1 --2- 172.21.15.99:0/1686640327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc104040 0x7f52bc106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.227+0000 7f52c2e00700 1 -- 172.21.15.99:0/1686640327 >> 172.21.15.99:0/1686640327 conn(0x7f52bc0fb100 msgr2=0x7f52bc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:25.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.227+0000 7f52c2e00700 1 -- 172.21.15.99:0/1686640327 shutdown_connections 2024-03-30T00:33:25.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.227+0000 7f52c2e00700 1 -- 172.21.15.99:0/1686640327 wait complete. 2024-03-30T00:33:25.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.228+0000 7f52c2e00700 1 Processor -- start 2024-03-30T00:33:25.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.228+0000 7f52c2e00700 1 -- start start 2024-03-30T00:33:25.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.228+0000 7f52c2e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52bc101710 0x7f52bc198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:25.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.228+0000 7f52c2e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52bc104040 0x7f52bc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:25.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.228+0000 7f52c2e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc106970 0x7f52bc1a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:25.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.228+0000 7f52c2e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc1a0940 con 0x7f52bc104040 2024-03-30T00:33:25.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.228+0000 7f52c2e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc1a0a80 con 0x7f52bc101710 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.228+0000 7f52c2e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc1a0c10 con 0x7f52bc106970 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52bb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52bc104040 0x7f52bc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52bb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52bc104040 0x7f52bc199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40464/0 (socket says 172.21.15.99:40464) 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52bb400700 1 -- 172.21.15.99:0/4052520718 learned_addr learned my addr 172.21.15.99:0/4052520718 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52c0e00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc106970 0x7f52bc1a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52bbe00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52bc101710 0x7f52bc198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52bb400700 1 -- 172.21.15.99:0/4052520718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc106970 msgr2=0x7f52bc1a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52bb400700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc106970 0x7f52bc1a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52bb400700 1 -- 172.21.15.99:0/4052520718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52bc101710 msgr2=0x7f52bc198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52bb400700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52bc101710 0x7f52bc198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52bb400700 1 -- 172.21.15.99:0/4052520718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b4009d00 con 0x7f52bc104040 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52c0e00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc106970 0x7f52bc1a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:25.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.229+0000 7f52bb400700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52bc104040 0x7f52bc199350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c920 tx=0x7f52a800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:25.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.230+0000 7f52b8c00700 1 -- 172.21.15.99:0/4052520718 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8007b00 con 0x7f52bc104040 2024-03-30T00:33:25.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.230+0000 7f52b8c00700 1 -- 172.21.15.99:0/4052520718 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52a8004cc0 con 0x7f52bc104040 2024-03-30T00:33:25.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.230+0000 7f52c2e00700 1 -- 172.21.15.99:0/4052520718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52bc1a0ef0 con 0x7f52bc104040 2024-03-30T00:33:25.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.230+0000 7f52b8c00700 1 -- 172.21.15.99:0/4052520718 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a8005590 con 0x7f52bc104040 2024-03-30T00:33:25.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.230+0000 7f52c2e00700 1 -- 172.21.15.99:0/4052520718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52bc1a13c0 con 0x7f52bc104040 2024-03-30T00:33:25.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.231+0000 7f52c2e00700 1 -- 172.21.15.99:0/4052520718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52bc1084f0 con 0x7f52bc104040 2024-03-30T00:33:25.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.232+0000 7f52b8c00700 1 -- 172.21.15.99:0/4052520718 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f52a8004750 con 0x7f52bc104040 2024-03-30T00:33:25.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.234+0000 7f52b8c00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f52a00a1230 0x7f52a00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:25.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.234+0000 7f52b8c00700 1 -- 172.21.15.99:0/4052520718 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f52a80cc880 con 0x7f52bc104040 2024-03-30T00:33:25.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.234+0000 7f52bbe00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f52a00a1230 0x7f52a00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:25.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.234+0000 7f52bbe00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f52a00a1230 0x7f52a00a36f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00bbd0 tx=0x7f52ac00b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:25.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.238+0000 7f52b8c00700 1 -- 172.21.15.99:0/4052520718 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f52a8095f60 con 0x7f52bc104040 2024-03-30T00:33:25.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.436+0000 7f52c2e00700 1 -- 172.21.15.99:0/4052520718 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52bc02cdf0 con 0x7f52a00a1230 2024-03-30T00:33:25.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.447+0000 7f52b8c00700 1 -- 172.21.15.99:0/4052520718 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335409 (secure 0 0 0) 0x7f52bc02cdf0 con 0x7f52a00a1230 2024-03-30T00:33:25.458 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:25.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.451+0000 7f52c2e00700 1 -- 172.21.15.99:0/4052520718 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f52a00a1230 msgr2=0x7f52a00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:25.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.451+0000 7f52c2e00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f52a00a1230 0x7f52a00a36f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00bbd0 tx=0x7f52ac00b5b0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.451+0000 7f52c2e00700 1 -- 172.21.15.99:0/4052520718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52bc104040 msgr2=0x7f52bc199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.451+0000 7f52c2e00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52bc104040 0x7f52bc199350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c920 tx=0x7f52a800cce0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.451+0000 7f52c2e00700 1 -- 172.21.15.99:0/4052520718 shutdown_connections 2024-03-30T00:33:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.451+0000 7f52c2e00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52bc106970 0x7f52bc1a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.451+0000 7f52c2e00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f52a00a1230 0x7f52a00a36f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.451+0000 7f52c2e00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52bc101710 0x7f52bc198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.451+0000 7f52c2e00700 1 --2- 172.21.15.99:0/4052520718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52bc104040 0x7f52bc199350 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.451+0000 7f52c2e00700 1 -- 172.21.15.99:0/4052520718 >> 172.21.15.99:0/4052520718 conn(0x7f52bc0fb100 msgr2=0x7f52bc100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.451+0000 7f52c2e00700 1 -- 172.21.15.99:0/4052520718 shutdown_connections 2024-03-30T00:33:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:25.452+0000 7f52c2e00700 1 -- 172.21.15.99:0/4052520718 wait complete. 2024-03-30T00:33:25.461 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-30T00:33:25.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:25 smithi161 ceph-mon[42221]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:25.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:25 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4114606688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:25.475 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:25 smithi099 ceph-mon[38079]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:25.475 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:25 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4114606688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:25.477 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-03-30T00:33:25.009316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595563,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":157,"num_read_kb":133,"num_write":306,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":246,"ondisk_log_size":246,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377632,"kb_used_data":12480,"kb_used_omap":18,"kb_used_meta":365037,"kb_avail":1124564192,"statfs":{"total":1151940427776,"available":1151553732608,"internally_reserved":0,"allocated":12779520,"data_stored":6933115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":373798214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,8,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5195,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":125,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.356007"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290853+0000","last_change":"2024-03-30T00:33:15.686814+0000","last_active":"2024-03-30T00:33:17.290853+0000","last_peered":"2024-03-30T00:33:17.290853+0000","last_clean":"2024-03-30T00:33:17.290853+0000","last_became_active":"2024-03-30T00:33:15.686432+0000","last_became_peered":"2024-03-30T00:33:15.686432+0000","last_unstale":"2024-03-30T00:33:17.290853+0000","last_undegraded":"2024-03-30T00:33:17.290853+0000","last_fullsized":"2024-03-30T00:33:17.290853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:35:05.898973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002227+0000","last_change":"2024-03-30T00:33:15.678089+0000","last_active":"2024-03-30T00:33:17.002227+0000","last_peered":"2024-03-30T00:33:17.002227+0000","last_clean":"2024-03-30T00:33:17.002227+0000","last_became_active":"2024-03-30T00:33:15.677254+0000","last_became_peered":"2024-03-30T00:33:15.677254+0000","last_unstale":"2024-03-30T00:33:17.002227+0000","last_undegraded":"2024-03-30T00:33:17.002227+0000","last_fullsized":"2024-03-30T00:33:17.002227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:45.922652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002176+0000","last_change":"2024-03-30T00:33:15.686777+0000","last_active":"2024-03-30T00:33:17.002176+0000","last_peered":"2024-03-30T00:33:17.002176+0000","last_clean":"2024-03-30T00:33:17.002176+0000","last_became_active":"2024-03-30T00:33:15.686461+0000","last_became_peered":"2024-03-30T00:33:15.686461+0000","last_unstale":"2024-03-30T00:33:17.002176+0000","last_undegraded":"2024-03-30T00:33:17.002176+0000","last_fullsized":"2024-03-30T00:33:17.002176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:33:27.073089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002223+0000","last_change":"2024-03-30T00:33:15.681237+0000","last_active":"2024-03-30T00:33:17.002223+0000","last_peered":"2024-03-30T00:33:17.002223+0000","last_clean":"2024-03-30T00:33:17.002223+0000","last_became_active":"2024-03-30T00:33:15.680669+0000","last_became_peered":"2024-03-30T00:33:15.680669+0000","last_unstale":"2024-03-30T00:33:17.002223+0000","last_undegraded":"2024-03-30T00:33:17.002223+0000","last_fullsized":"2024-03-30T00:33:17.002223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:59.300262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.682286+0000","last_change":"2024-03-30T00:33:15.678825+0000","last_active":"2024-03-30T00:33:15.682286+0000","last_peered":"2024-03-30T00:33:15.682286+0000","last_clean":"2024-03-30T00:33:15.682286+0000","last_became_active":"2024-03-30T00:33:15.676864+0000","last_became_peered":"2024-03-30T00:33:15.676864+0000","last_unstale":"2024-03-30T00:33:15.682286+0000","last_undegraded":"2024-03-30T00:33:15.682286+0000","last_fullsized":"2024-03-30T00:33:15.682286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:12:09.724441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.688863+0000","last_change":"2024-03-30T00:33:15.681866+0000","last_active":"2024-03-30T00:33:15.688863+0000","last_peered":"2024-03-30T00:33:15.688863+0000","last_clean":"2024-03-30T00:33:15.688863+0000","last_became_active":"2024-03-30T00:33:15.678118+0000","last_became_peered":"2024-03-30T00:33:15.678118+0000","last_unstale":"2024-03-30T00:33:15.688863+0000","last_undegraded":"2024-03-30T00:33:15.688863+0000","last_fullsized":"2024-03-30T00:33:15.688863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:24.433027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001193+0000","last_change":"2024-03-30T00:33:15.675999+0000","last_active":"2024-03-30T00:33:17.001193+0000","last_peered":"2024-03-30T00:33:17.001193+0000","last_clean":"2024-03-30T00:33:17.001193+0000","last_became_active":"2024-03-30T00:33:15.675319+0000","last_became_peered":"2024-03-30T00:33:15.675319+0000","last_unstale":"2024-03-30T00:33:17.001193+0000","last_undegraded":"2024-03-30T00:33:17.001193+0000","last_fullsized":"2024-03-30T00:33:17.001193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:42.668747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000423+0000","last_change":"2024-03-30T00:33:15.679211+0000","last_active":"2024-03-30T00:33:17.000423+0000","last_peered":"2024-03-30T00:33:17.000423+0000","last_clean":"2024-03-30T00:33:17.000423+0000","last_became_active":"2024-03-30T00:33:15.675061+0000","last_became_peered":"2024-03-30T00:33:15.675061+0000","last_unstale":"2024-03-30T00:33:17.000423+0000","last_undegraded":"2024-03-30T00:33:17.000423+0000","last_fullsized":"2024-03-30T00:33:17.000423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:43.973542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.692417+0000","last_change":"2024-03-30T00:33:15.680842+0000","last_active":"2024-03-30T00:33:15.692417+0000","last_peered":"2024-03-30T00:33:15.692417+0000","last_clean":"2024-03-30T00:33:15.692417+0000","last_became_active":"2024-03-30T00:33:15.680307+0000","last_became_peered":"2024-03-30T00:33:15.680307+0000","last_unstale":"2024-03-30T00:33:15.692417+0000","last_undegraded":"2024-03-30T00:33:15.692417+0000","last_fullsized":"2024-03-30T00:33:15.692417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:59:40.331779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693556+0000","last_change":"2024-03-30T00:33:15.692397+0000","last_active":"2024-03-30T00:33:15.693556+0000","last_peered":"2024-03-30T00:33:15.693556+0000","last_clean":"2024-03-30T00:33:15.693556+0000","last_became_active":"2024-03-30T00:33:15.691226+0000","last_became_peered":"2024-03-30T00:33:15.691226+0000","last_unstale":"2024-03-30T00:33:15.693556+0000","last_undegraded":"2024-03-30T00:33:15.693556+0000","last_fullsized":"2024-03-30T00:33:15.693556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:48.816644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001314+0000","last_change":"2024-03-30T00:33:15.688417+0000","last_active":"2024-03-30T00:33:17.001314+0000","last_peered":"2024-03-30T00:33:17.001314+0000","last_clean":"2024-03-30T00:33:17.001314+0000","last_became_active":"2024-03-30T00:33:15.687755+0000","last_became_peered":"2024-03-30T00:33:15.687755+0000","last_unstale":"2024-03-30T00:33:17.001314+0000","last_undegraded":"2024-03-30T00:33:17.001314+0000","last_fullsized":"2024-03-30T00:33:17.001314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:30:19.782280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002119+0000","last_change":"2024-03-30T00:33:15.681942+0000","last_active":"2024-03-30T00:33:17.002119+0000","last_peered":"2024-03-30T00:33:17.002119+0000","last_clean":"2024-03-30T00:33:17.002119+0000","last_became_active":"2024-03-30T00:33:15.679854+0000","last_became_peered":"2024-03-30T00:33:15.679854+0000","last_unstale":"2024-03-30T00:33:17.002119+0000","last_undegraded":"2024-03-30T00:33:17.002119+0000","last_fullsized":"2024-03-30T00:33:17.002119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:19.771959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001108+0000","last_change":"2024-03-30T00:33:15.682968+0000","last_active":"2024-03-30T00:33:17.001108+0000","last_peered":"2024-03-30T00:33:17.001108+0000","last_clean":"2024-03-30T00:33:17.001108+0000","last_became_active":"2024-03-30T00:33:15.681216+0000","last_became_peered":"2024-03-30T00:33:15.681216+0000","last_unstale":"2024-03-30T00:33:17.001108+0000","last_undegraded":"2024-03-30T00:33:17.001108+0000","last_fullsized":"2024-03-30T00:33:17.001108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:50.038843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001697+0000","last_change":"2024-03-30T00:33:15.677727+0000","last_active":"2024-03-30T00:33:17.001697+0000","last_peered":"2024-03-30T00:33:17.001697+0000","last_clean":"2024-03-30T00:33:17.001697+0000","last_became_active":"2024-03-30T00:33:15.676758+0000","last_became_peered":"2024-03-30T00:33:15.676758+0000","last_unstale":"2024-03-30T00:33:17.001697+0000","last_undegraded":"2024-03-30T00:33:17.001697+0000","last_fullsized":"2024-03-30T00:33:17.001697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:34.745920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001691+0000","last_change":"2024-03-30T00:33:15.684545+0000","last_active":"2024-03-30T00:33:17.001691+0000","last_peered":"2024-03-30T00:33:17.001691+0000","last_clean":"2024-03-30T00:33:17.001691+0000","last_became_active":"2024-03-30T00:33:15.682989+0000","last_became_peered":"2024-03-30T00:33:15.682989+0000","last_unstale":"2024-03-30T00:33:17.001691+0000","last_undegraded":"2024-03-30T00:33:17.001691+0000","last_fullsized":"2024-03-30T00:33:17.001691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:11.120981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.686822+0000","last_change":"2024-03-30T00:33:15.678122+0000","last_active":"2024-03-30T00:33:15.686822+0000","last_peered":"2024-03-30T00:33:15.686822+0000","last_clean":"2024-03-30T00:33:15.686822+0000","last_became_active":"2024-03-30T00:33:15.677681+0000","last_became_peered":"2024-03-30T00:33:15.677681+0000","last_unstale":"2024-03-30T00:33:15.686822+0000","last_undegraded":"2024-03-30T00:33:15.686822+0000","last_fullsized":"2024-03-30T00:33:15.686822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:50.454125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290980+0000","last_change":"2024-03-30T00:33:15.687689+0000","last_active":"2024-03-30T00:33:17.290980+0000","last_peered":"2024-03-30T00:33:17.290980+0000","last_clean":"2024-03-30T00:33:17.290980+0000","last_became_active":"2024-03-30T00:33:15.686009+0000","last_became_peered":"2024-03-30T00:33:15.686009+0000","last_unstale":"2024-03-30T00:33:17.290980+0000","last_undegraded":"2024-03-30T00:33:17.290980+0000","last_fullsized":"2024-03-30T00:33:17.290980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:17.234749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659641+0000","last_change":"2024-03-30T00:33:15.686053+0000","last_active":"2024-03-30T00:33:16.659641+0000","last_peered":"2024-03-30T00:33:16.659641+0000","last_clean":"2024-03-30T00:33:16.659641+0000","last_became_active":"2024-03-30T00:33:15.685723+0000","last_became_peered":"2024-03-30T00:33:15.685723+0000","last_unstale":"2024-03-30T00:33:16.659641+0000","last_undegraded":"2024-03-30T00:33:16.659641+0000","last_fullsized":"2024-03-30T00:33:16.659641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:27.322834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660740+0000","last_change":"2024-03-30T00:33:15.674783+0000","last_active":"2024-03-30T00:33:16.660740+0000","last_peered":"2024-03-30T00:33:16.660740+0000","last_clean":"2024-03-30T00:33:16.660740+0000","last_became_active":"2024-03-30T00:33:15.674152+0000","last_became_peered":"2024-03-30T00:33:15.674152+0000","last_unstale":"2024-03-30T00:33:16.660740+0000","last_undegraded":"2024-03-30T00:33:16.660740+0000","last_fullsized":"2024-03-30T00:33:16.660740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:39.517240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.119665+0000","last_change":"2024-03-30T00:33:15.172217+0000","last_active":"2024-03-30T00:33:20.119665+0000","last_peered":"2024-03-30T00:33:20.119665+0000","last_clean":"2024-03-30T00:33:20.119665+0000","last_became_active":"2024-03-30T00:33:15.171718+0000","last_became_peered":"2024-03-30T00:33:15.171718+0000","last_unstale":"2024-03-30T00:33:20.119665+0000","last_undegraded":"2024-03-30T00:33:20.119665+0000","last_fullsized":"2024-03-30T00:33:20.119665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:11.691572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001530+0000","last_change":"2024-03-30T00:33:15.669937+0000","last_active":"2024-03-30T00:33:17.001530+0000","last_peered":"2024-03-30T00:33:17.001530+0000","last_clean":"2024-03-30T00:33:17.001530+0000","last_became_active":"2024-03-30T00:33:15.668704+0000","last_became_peered":"2024-03-30T00:33:15.668704+0000","last_unstale":"2024-03-30T00:33:17.001530+0000","last_undegraded":"2024-03-30T00:33:17.001530+0000","last_fullsized":"2024-03-30T00:33:17.001530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:23.914755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:18.428835+0000","last_change":"2024-03-30T00:33:15.173663+0000","last_active":"2024-03-30T00:33:18.428835+0000","last_peered":"2024-03-30T00:33:18.428835+0000","last_clean":"2024-03-30T00:33:18.428835+0000","last_became_active":"2024-03-30T00:33:15.173288+0000","last_became_peered":"2024-03-30T00:33:15.173288+0000","last_unstale":"2024-03-30T00:33:18.428835+0000","last_undegraded":"2024-03-30T00:33:18.428835+0000","last_fullsized":"2024-03-30T00:33:18.428835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.435142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002383+0000","last_change":"2024-03-30T00:33:15.686757+0000","last_active":"2024-03-30T00:33:17.002383+0000","last_peered":"2024-03-30T00:33:17.002383+0000","last_clean":"2024-03-30T00:33:17.002383+0000","last_became_active":"2024-03-30T00:33:15.686306+0000","last_became_peered":"2024-03-30T00:33:15.686306+0000","last_unstale":"2024-03-30T00:33:17.002383+0000","last_undegraded":"2024-03-30T00:33:17.002383+0000","last_fullsized":"2024-03-30T00:33:17.002383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:50:21.048212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.117995+0000","last_change":"2024-03-30T00:33:15.173402+0000","last_active":"2024-03-30T00:33:20.117995+0000","last_peered":"2024-03-30T00:33:20.117995+0000","last_clean":"2024-03-30T00:33:20.117995+0000","last_became_active":"2024-03-30T00:33:15.172995+0000","last_became_peered":"2024-03-30T00:33:15.172995+0000","last_unstale":"2024-03-30T00:33:20.117995+0000","last_undegraded":"2024-03-30T00:33:20.117995+0000","last_fullsized":"2024-03-30T00:33:20.117995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:25.694792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.691152+0000","last_change":"2024-03-30T00:33:15.688292+0000","last_active":"2024-03-30T00:33:15.691152+0000","last_peered":"2024-03-30T00:33:15.691152+0000","last_clean":"2024-03-30T00:33:15.691152+0000","last_became_active":"2024-03-30T00:33:15.682722+0000","last_became_peered":"2024-03-30T00:33:15.682722+0000","last_unstale":"2024-03-30T00:33:15.691152+0000","last_undegraded":"2024-03-30T00:33:15.691152+0000","last_fullsized":"2024-03-30T00:33:15.691152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:52:10.074656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.692186+0000","last_change":"2024-03-30T00:33:15.162316+0000","last_active":"2024-03-30T00:33:15.692186+0000","last_peered":"2024-03-30T00:33:15.692186+0000","last_clean":"2024-03-30T00:33:15.692186+0000","last_became_active":"2024-03-30T00:33:15.161805+0000","last_became_peered":"2024-03-30T00:33:15.161805+0000","last_unstale":"2024-03-30T00:33:15.692186+0000","last_undegraded":"2024-03-30T00:33:15.692186+0000","last_fullsized":"2024-03-30T00:33:15.692186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:26:16.470245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.532008+0000","last_change":"2024-03-30T00:33:15.675442+0000","last_active":"2024-03-30T00:33:16.532008+0000","last_peered":"2024-03-30T00:33:16.532008+0000","last_clean":"2024-03-30T00:33:16.532008+0000","last_became_active":"2024-03-30T00:33:15.675068+0000","last_became_peered":"2024-03-30T00:33:15.675068+0000","last_unstale":"2024-03-30T00:33:16.532008+0000","last_undegraded":"2024-03-30T00:33:16.532008+0000","last_fullsized":"2024-03-30T00:33:16.532008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:36:31.353262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.107636+0000","last_change":"2024-03-30T00:33:15.172087+0000","last_active":"2024-03-30T00:33:20.107636+0000","last_peered":"2024-03-30T00:33:20.107636+0000","last_clean":"2024-03-30T00:33:20.107636+0000","last_became_active":"2024-03-30T00:33:15.171768+0000","last_became_peered":"2024-03-30T00:33:15.171768+0000","last_unstale":"2024-03-30T00:33:20.107636+0000","last_undegraded":"2024-03-30T00:33:20.107636+0000","last_fullsized":"2024-03-30T00:33:20.107636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:17.198995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.668850+0000","last_change":"2024-03-30T00:33:15.665219+0000","last_active":"2024-03-30T00:33:15.668850+0000","last_peered":"2024-03-30T00:33:15.668850+0000","last_clean":"2024-03-30T00:33:15.668850+0000","last_became_active":"2024-03-30T00:33:15.664935+0000","last_became_peered":"2024-03-30T00:33:15.664935+0000","last_unstale":"2024-03-30T00:33:15.668850+0000","last_undegraded":"2024-03-30T00:33:15.668850+0000","last_fullsized":"2024-03-30T00:33:15.668850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:37.123937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.116508+0000","last_change":"2024-03-30T00:33:15.165668+0000","last_active":"2024-03-30T00:33:20.116508+0000","last_peered":"2024-03-30T00:33:20.116508+0000","last_clean":"2024-03-30T00:33:20.116508+0000","last_became_active":"2024-03-30T00:33:15.164637+0000","last_became_peered":"2024-03-30T00:33:15.164637+0000","last_unstale":"2024-03-30T00:33:20.116508+0000","last_undegraded":"2024-03-30T00:33:20.116508+0000","last_fullsized":"2024-03-30T00:33:20.116508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:47:59.389414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531981+0000","last_change":"2024-03-30T00:33:15.662357+0000","last_active":"2024-03-30T00:33:16.531981+0000","last_peered":"2024-03-30T00:33:16.531981+0000","last_clean":"2024-03-30T00:33:16.531981+0000","last_became_active":"2024-03-30T00:33:15.662011+0000","last_became_peered":"2024-03-30T00:33:15.662011+0000","last_unstale":"2024-03-30T00:33:16.531981+0000","last_undegraded":"2024-03-30T00:33:16.531981+0000","last_fullsized":"2024-03-30T00:33:16.531981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:18.386054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118436+0000","last_change":"2024-03-30T00:33:15.177269+0000","last_active":"2024-03-30T00:33:20.118436+0000","last_peered":"2024-03-30T00:33:20.118436+0000","last_clean":"2024-03-30T00:33:20.118436+0000","last_became_active":"2024-03-30T00:33:15.176937+0000","last_became_peered":"2024-03-30T00:33:15.176937+0000","last_unstale":"2024-03-30T00:33:20.118436+0000","last_undegraded":"2024-03-30T00:33:20.118436+0000","last_fullsized":"2024-03-30T00:33:20.118436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:44.658775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000691+0000","last_change":"2024-03-30T00:33:15.675832+0000","last_active":"2024-03-30T00:33:17.000691+0000","last_peered":"2024-03-30T00:33:17.000691+0000","last_clean":"2024-03-30T00:33:17.000691+0000","last_became_active":"2024-03-30T00:33:15.675481+0000","last_became_peered":"2024-03-30T00:33:15.675481+0000","last_unstale":"2024-03-30T00:33:17.000691+0000","last_undegraded":"2024-03-30T00:33:17.000691+0000","last_fullsized":"2024-03-30T00:33:17.000691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:11.638056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001044+0000","last_change":"2024-03-30T00:33:15.171238+0000","last_active":"2024-03-30T00:33:17.001044+0000","last_peered":"2024-03-30T00:33:17.001044+0000","last_clean":"2024-03-30T00:33:17.001044+0000","last_became_active":"2024-03-30T00:33:15.170931+0000","last_became_peered":"2024-03-30T00:33:15.170931+0000","last_unstale":"2024-03-30T00:33:17.001044+0000","last_undegraded":"2024-03-30T00:33:17.001044+0000","last_fullsized":"2024-03-30T00:33:17.001044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:43:50.178363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659336+0000","last_change":"2024-03-30T00:33:15.676014+0000","last_active":"2024-03-30T00:33:16.659336+0000","last_peered":"2024-03-30T00:33:16.659336+0000","last_clean":"2024-03-30T00:33:16.659336+0000","last_became_active":"2024-03-30T00:33:15.675553+0000","last_became_peered":"2024-03-30T00:33:15.675553+0000","last_unstale":"2024-03-30T00:33:16.659336+0000","last_undegraded":"2024-03-30T00:33:16.659336+0000","last_fullsized":"2024-03-30T00:33:16.659336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:17.248242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.133355+0000","last_change":"2024-03-30T00:33:15.177033+0000","last_active":"2024-03-30T00:33:19.133355+0000","last_peered":"2024-03-30T00:33:19.133355+0000","last_clean":"2024-03-30T00:33:19.133355+0000","last_became_active":"2024-03-30T00:33:15.176602+0000","last_became_peered":"2024-03-30T00:33:15.176602+0000","last_unstale":"2024-03-30T00:33:19.133355+0000","last_undegraded":"2024-03-30T00:33:19.133355+0000","last_fullsized":"2024-03-30T00:33:19.133355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:10.804704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.691801+0000","last_change":"2024-03-30T00:33:15.683388+0000","last_active":"2024-03-30T00:33:15.691801+0000","last_peered":"2024-03-30T00:33:15.691801+0000","last_clean":"2024-03-30T00:33:15.691801+0000","last_became_active":"2024-03-30T00:33:15.682976+0000","last_became_peered":"2024-03-30T00:33:15.682976+0000","last_unstale":"2024-03-30T00:33:15.691801+0000","last_undegraded":"2024-03-30T00:33:15.691801+0000","last_fullsized":"2024-03-30T00:33:15.691801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:50:34.907265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131200+0000","last_change":"2024-03-30T00:33:15.167819+0000","last_active":"2024-03-30T00:33:19.131200+0000","last_peered":"2024-03-30T00:33:19.131200+0000","last_clean":"2024-03-30T00:33:19.131200+0000","last_became_active":"2024-03-30T00:33:15.167405+0000","last_became_peered":"2024-03-30T00:33:15.167405+0000","last_unstale":"2024-03-30T00:33:19.131200+0000","last_undegraded":"2024-03-30T00:33:19.131200+0000","last_fullsized":"2024-03-30T00:33:19.131200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:37.793792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002345+0000","last_change":"2024-03-30T00:33:15.690907+0000","last_active":"2024-03-30T00:33:17.002345+0000","last_peered":"2024-03-30T00:33:17.002345+0000","last_clean":"2024-03-30T00:33:17.002345+0000","last_became_active":"2024-03-30T00:33:15.690046+0000","last_became_peered":"2024-03-30T00:33:15.690046+0000","last_unstale":"2024-03-30T00:33:17.002345+0000","last_undegraded":"2024-03-30T00:33:17.002345+0000","last_fullsized":"2024-03-30T00:33:17.002345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:24.319685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.132456+0000","last_change":"2024-03-30T00:33:15.166633+0000","last_active":"2024-03-30T00:33:19.132456+0000","last_peered":"2024-03-30T00:33:19.132456+0000","last_clean":"2024-03-30T00:33:19.132456+0000","last_became_active":"2024-03-30T00:33:15.166308+0000","last_became_peered":"2024-03-30T00:33:15.166308+0000","last_unstale":"2024-03-30T00:33:19.132456+0000","last_undegraded":"2024-03-30T00:33:19.132456+0000","last_fullsized":"2024-03-30T00:33:19.132456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:53.102872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693949+0000","last_change":"2024-03-30T00:33:15.691880+0000","last_active":"2024-03-30T00:33:15.693949+0000","last_peered":"2024-03-30T00:33:15.693949+0000","last_clean":"2024-03-30T00:33:15.693949+0000","last_became_active":"2024-03-30T00:33:15.690780+0000","last_became_peered":"2024-03-30T00:33:15.690780+0000","last_unstale":"2024-03-30T00:33:15.693949+0000","last_undegraded":"2024-03-30T00:33:15.693949+0000","last_fullsized":"2024-03-30T00:33:15.693949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:06.532709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.107184+0000","last_change":"2024-03-30T00:33:15.164287+0000","last_active":"2024-03-30T00:33:20.107184+0000","last_peered":"2024-03-30T00:33:20.107184+0000","last_clean":"2024-03-30T00:33:20.107184+0000","last_became_active":"2024-03-30T00:33:15.163557+0000","last_became_peered":"2024-03-30T00:33:15.163557+0000","last_unstale":"2024-03-30T00:33:20.107184+0000","last_undegraded":"2024-03-30T00:33:20.107184+0000","last_fullsized":"2024-03-30T00:33:20.107184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:41.117936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.683827+0000","last_change":"2024-03-30T00:33:15.682841+0000","last_active":"2024-03-30T00:33:15.683827+0000","last_peered":"2024-03-30T00:33:15.683827+0000","last_clean":"2024-03-30T00:33:15.683827+0000","last_became_active":"2024-03-30T00:33:15.682237+0000","last_became_peered":"2024-03-30T00:33:15.682237+0000","last_unstale":"2024-03-30T00:33:15.683827+0000","last_undegraded":"2024-03-30T00:33:15.683827+0000","last_fullsized":"2024-03-30T00:33:15.683827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:59:05.719216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118639+0000","last_change":"2024-03-30T00:33:15.165193+0000","last_active":"2024-03-30T00:33:20.118639+0000","last_peered":"2024-03-30T00:33:20.118639+0000","last_clean":"2024-03-30T00:33:20.118639+0000","last_became_active":"2024-03-30T00:33:15.161741+0000","last_became_peered":"2024-03-30T00:33:15.161741+0000","last_unstale":"2024-03-30T00:33:20.118639+0000","last_undegraded":"2024-03-30T00:33:20.118639+0000","last_fullsized":"2024-03-30T00:33:20.118639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:22:11.891064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001296+0000","last_change":"2024-03-30T00:33:15.680929+0000","last_active":"2024-03-30T00:33:17.001296+0000","last_peered":"2024-03-30T00:33:17.001296+0000","last_clean":"2024-03-30T00:33:17.001296+0000","last_became_active":"2024-03-30T00:33:15.676671+0000","last_became_peered":"2024-03-30T00:33:15.676671+0000","last_unstale":"2024-03-30T00:33:17.001296+0000","last_undegraded":"2024-03-30T00:33:17.001296+0000","last_fullsized":"2024-03-30T00:33:17.001296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:28:33.819135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.132787+0000","last_change":"2024-03-30T00:33:15.168029+0000","last_active":"2024-03-30T00:33:19.132787+0000","last_peered":"2024-03-30T00:33:19.132787+0000","last_clean":"2024-03-30T00:33:19.132787+0000","last_became_active":"2024-03-30T00:33:15.167636+0000","last_became_peered":"2024-03-30T00:33:15.167636+0000","last_unstale":"2024-03-30T00:33:19.132787+0000","last_undegraded":"2024-03-30T00:33:19.132787+0000","last_fullsized":"2024-03-30T00:33:19.132787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:25:47.834696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001171+0000","last_change":"2024-03-30T00:33:15.687977+0000","last_active":"2024-03-30T00:33:17.001171+0000","last_peered":"2024-03-30T00:33:17.001171+0000","last_clean":"2024-03-30T00:33:17.001171+0000","last_became_active":"2024-03-30T00:33:15.687491+0000","last_became_peered":"2024-03-30T00:33:15.687491+0000","last_unstale":"2024-03-30T00:33:17.001171+0000","last_undegraded":"2024-03-30T00:33:17.001171+0000","last_fullsized":"2024-03-30T00:33:17.001171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:08:55.568039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:23.136131+0000","last_change":"2024-03-30T00:33:15.171826+0000","last_active":"2024-03-30T00:33:23.136131+0000","last_peered":"2024-03-30T00:33:23.136131+0000","last_clean":"2024-03-30T00:33:23.136131+0000","last_became_active":"2024-03-30T00:33:15.170749+0000","last_became_peered":"2024-03-30T00:33:15.170749+0000","last_unstale":"2024-03-30T00:33:23.136131+0000","last_undegraded":"2024-03-30T00:33:23.136131+0000","last_fullsized":"2024-03-30T00:33:23.136131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:57:52.034530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000714+0000","last_change":"2024-03-30T00:33:15.681668+0000","last_active":"2024-03-30T00:33:17.000714+0000","last_peered":"2024-03-30T00:33:17.000714+0000","last_clean":"2024-03-30T00:33:17.000714+0000","last_became_active":"2024-03-30T00:33:15.681129+0000","last_became_peered":"2024-03-30T00:33:15.681129+0000","last_unstale":"2024-03-30T00:33:17.000714+0000","last_undegraded":"2024-03-30T00:33:17.000714+0000","last_fullsized":"2024-03-30T00:33:17.000714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:51:19.552243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118585+0000","last_change":"2024-03-30T00:33:15.171016+0000","last_active":"2024-03-30T00:33:20.118585+0000","last_peered":"2024-03-30T00:33:20.118585+0000","last_clean":"2024-03-30T00:33:20.118585+0000","last_became_active":"2024-03-30T00:33:15.170058+0000","last_became_peered":"2024-03-30T00:33:15.170058+0000","last_unstale":"2024-03-30T00:33:20.118585+0000","last_undegraded":"2024-03-30T00:33:20.118585+0000","last_fullsized":"2024-03-30T00:33:20.118585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:07:39.910286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531944+0000","last_change":"2024-03-30T00:33:15.687012+0000","last_active":"2024-03-30T00:33:16.531944+0000","last_peered":"2024-03-30T00:33:16.531944+0000","last_clean":"2024-03-30T00:33:16.531944+0000","last_became_active":"2024-03-30T00:33:15.684030+0000","last_became_peered":"2024-03-30T00:33:15.684030+0000","last_unstale":"2024-03-30T00:33:16.531944+0000","last_undegraded":"2024-03-30T00:33:16.531944+0000","last_fullsized":"2024-03-30T00:33:16.531944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:32.447119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001625+0000","last_change":"2024-03-30T00:33:15.163633+0000","last_active":"2024-03-30T00:33:17.001625+0000","last_peered":"2024-03-30T00:33:17.001625+0000","last_clean":"2024-03-30T00:33:17.001625+0000","last_became_active":"2024-03-30T00:33:15.161339+0000","last_became_peered":"2024-03-30T00:33:15.161339+0000","last_unstale":"2024-03-30T00:33:17.001625+0000","last_undegraded":"2024-03-30T00:33:17.001625+0000","last_fullsized":"2024-03-30T00:33:17.001625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:15:25.194650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290780+0000","last_change":"2024-03-30T00:33:15.680617+0000","last_active":"2024-03-30T00:33:17.290780+0000","last_peered":"2024-03-30T00:33:17.290780+0000","last_clean":"2024-03-30T00:33:17.290780+0000","last_became_active":"2024-03-30T00:33:15.679137+0000","last_became_peered":"2024-03-30T00:33:15.679137+0000","last_unstale":"2024-03-30T00:33:17.290780+0000","last_undegraded":"2024-03-30T00:33:17.290780+0000","last_fullsized":"2024-03-30T00:33:17.290780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:34.449218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659518+0000","last_change":"2024-03-30T00:33:15.177273+0000","last_active":"2024-03-30T00:33:16.659518+0000","last_peered":"2024-03-30T00:33:16.659518+0000","last_clean":"2024-03-30T00:33:16.659518+0000","last_became_active":"2024-03-30T00:33:15.175333+0000","last_became_peered":"2024-03-30T00:33:15.175333+0000","last_unstale":"2024-03-30T00:33:16.659518+0000","last_undegraded":"2024-03-30T00:33:16.659518+0000","last_fullsized":"2024-03-30T00:33:16.659518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:27:03.305671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.685478+0000","last_change":"2024-03-30T00:33:15.674746+0000","last_active":"2024-03-30T00:33:15.685478+0000","last_peered":"2024-03-30T00:33:15.685478+0000","last_clean":"2024-03-30T00:33:15.685478+0000","last_became_active":"2024-03-30T00:33:15.674388+0000","last_became_peered":"2024-03-30T00:33:15.674388+0000","last_unstale":"2024-03-30T00:33:15.685478+0000","last_undegraded":"2024-03-30T00:33:15.685478+0000","last_fullsized":"2024-03-30T00:33:15.685478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:21:15.024181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.054408+0000","last_change":"2024-03-30T00:33:15.173640+0000","last_active":"2024-03-30T00:33:19.054408+0000","last_peered":"2024-03-30T00:33:19.054408+0000","last_clean":"2024-03-30T00:33:19.054408+0000","last_became_active":"2024-03-30T00:33:15.173104+0000","last_became_peered":"2024-03-30T00:33:15.173104+0000","last_unstale":"2024-03-30T00:33:19.054408+0000","last_undegraded":"2024-03-30T00:33:19.054408+0000","last_fullsized":"2024-03-30T00:33:19.054408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:20:14.880730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.683575+0000","last_change":"2024-03-30T00:33:15.677824+0000","last_active":"2024-03-30T00:33:15.683575+0000","last_peered":"2024-03-30T00:33:15.683575+0000","last_clean":"2024-03-30T00:33:15.683575+0000","last_became_active":"2024-03-30T00:33:15.677435+0000","last_became_peered":"2024-03-30T00:33:15.677435+0000","last_unstale":"2024-03-30T00:33:15.683575+0000","last_undegraded":"2024-03-30T00:33:15.683575+0000","last_fullsized":"2024-03-30T00:33:15.683575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:22.992989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531931+0000","last_change":"2024-03-30T00:33:15.667428+0000","last_active":"2024-03-30T00:33:16.531931+0000","last_peered":"2024-03-30T00:33:16.531931+0000","last_clean":"2024-03-30T00:33:16.531931+0000","last_became_active":"2024-03-30T00:33:15.666634+0000","last_became_peered":"2024-03-30T00:33:15.666634+0000","last_unstale":"2024-03-30T00:33:16.531931+0000","last_undegraded":"2024-03-30T00:33:16.531931+0000","last_fullsized":"2024-03-30T00:33:16.531931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:56:14.710178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:22.132841+0000","last_change":"2024-03-30T00:33:15.173264+0000","last_active":"2024-03-30T00:33:22.132841+0000","last_peered":"2024-03-30T00:33:22.132841+0000","last_clean":"2024-03-30T00:33:22.132841+0000","last_became_active":"2024-03-30T00:33:15.172932+0000","last_became_peered":"2024-03-30T00:33:15.172932+0000","last_unstale":"2024-03-30T00:33:22.132841+0000","last_undegraded":"2024-03-30T00:33:22.132841+0000","last_fullsized":"2024-03-30T00:33:22.132841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:18:25.079284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000756+0000","last_change":"2024-03-30T00:33:15.686606+0000","last_active":"2024-03-30T00:33:17.000756+0000","last_peered":"2024-03-30T00:33:17.000756+0000","last_clean":"2024-03-30T00:33:17.000756+0000","last_became_active":"2024-03-30T00:33:15.686076+0000","last_became_peered":"2024-03-30T00:33:15.686076+0000","last_unstale":"2024-03-30T00:33:17.000756+0000","last_undegraded":"2024-03-30T00:33:17.000756+0000","last_fullsized":"2024-03-30T00:33:17.000756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:46.962659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.119040+0000","last_change":"2024-03-30T00:33:15.177263+0000","last_active":"2024-03-30T00:33:20.119040+0000","last_peered":"2024-03-30T00:33:20.119040+0000","last_clean":"2024-03-30T00:33:20.119040+0000","last_became_active":"2024-03-30T00:33:15.176743+0000","last_became_peered":"2024-03-30T00:33:15.176743+0000","last_unstale":"2024-03-30T00:33:20.119040+0000","last_undegraded":"2024-03-30T00:33:20.119040+0000","last_fullsized":"2024-03-30T00:33:20.119040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:56:47.491371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":322,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.669557+0000","last_change":"2024-03-30T00:32:12.678730+0000","last_active":"2024-03-30T00:33:15.669557+0000","last_peered":"2024-03-30T00:33:15.669557+0000","last_clean":"2024-03-30T00:33:15.669557+0000","last_became_active":"2024-03-30T00:32:12.678256+0000","last_became_peered":"2024-03-30T00:32:12.678256+0000","last_unstale":"2024-03-30T00:33:15.669557+0000","last_undegraded":"2024-03-30T00:33:15.669557+0000","last_fullsized":"2024-03-30T00:33:15.669557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_clean_scrub_stamp":"2024-03-30T00:29:17.768535+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:16.229072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000775+0000","last_change":"2024-03-30T00:33:15.685230+0000","last_active":"2024-03-30T00:33:17.000775+0000","last_peered":"2024-03-30T00:33:17.000775+0000","last_clean":"2024-03-30T00:33:17.000775+0000","last_became_active":"2024-03-30T00:33:15.684263+0000","last_became_peered":"2024-03-30T00:33:15.684263+0000","last_unstale":"2024-03-30T00:33:17.000775+0000","last_undegraded":"2024-03-30T00:33:17.000775+0000","last_fullsized":"2024-03-30T00:33:17.000775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:22:51.980318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.115562+0000","last_change":"2024-03-30T00:33:15.167490+0000","last_active":"2024-03-30T00:33:20.115562+0000","last_peered":"2024-03-30T00:33:20.115562+0000","last_clean":"2024-03-30T00:33:20.115562+0000","last_became_active":"2024-03-30T00:33:15.166888+0000","last_became_peered":"2024-03-30T00:33:15.166888+0000","last_unstale":"2024-03-30T00:33:20.115562+0000","last_undegraded":"2024-03-30T00:33:20.115562+0000","last_fullsized":"2024-03-30T00:33:20.115562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:00:32.571281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131306+0000","last_change":"2024-03-30T00:33:15.178388+0000","last_active":"2024-03-30T00:33:19.131306+0000","last_peered":"2024-03-30T00:33:19.131306+0000","last_clean":"2024-03-30T00:33:19.131306+0000","last_became_active":"2024-03-30T00:33:15.178160+0000","last_became_peered":"2024-03-30T00:33:15.178160+0000","last_unstale":"2024-03-30T00:33:19.131306+0000","last_undegraded":"2024-03-30T00:33:19.131306+0000","last_fullsized":"2024-03-30T00:33:19.131306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:28:51.567598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001200+0000","last_change":"2024-03-30T00:33:15.675426+0000","last_active":"2024-03-30T00:33:17.001200+0000","last_peered":"2024-03-30T00:33:17.001200+0000","last_clean":"2024-03-30T00:33:17.001200+0000","last_became_active":"2024-03-30T00:33:15.675083+0000","last_became_peered":"2024-03-30T00:33:15.675083+0000","last_unstale":"2024-03-30T00:33:17.001200+0000","last_undegraded":"2024-03-30T00:33:17.001200+0000","last_fullsized":"2024-03-30T00:33:17.001200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:37:03.086331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118827+0000","last_change":"2024-03-30T00:33:15.171537+0000","last_active":"2024-03-30T00:33:20.118827+0000","last_peered":"2024-03-30T00:33:20.118827+0000","last_clean":"2024-03-30T00:33:20.118827+0000","last_became_active":"2024-03-30T00:33:15.171128+0000","last_became_peered":"2024-03-30T00:33:15.171128+0000","last_unstale":"2024-03-30T00:33:20.118827+0000","last_undegraded":"2024-03-30T00:33:20.118827+0000","last_fullsized":"2024-03-30T00:33:20.118827+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:52.062144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001730+0000","last_change":"2024-03-30T00:33:15.674075+0000","last_active":"2024-03-30T00:33:17.001730+0000","last_peered":"2024-03-30T00:33:17.001730+0000","last_clean":"2024-03-30T00:33:17.001730+0000","last_became_active":"2024-03-30T00:33:15.672400+0000","last_became_peered":"2024-03-30T00:33:15.672400+0000","last_unstale":"2024-03-30T00:33:17.001730+0000","last_undegraded":"2024-03-30T00:33:17.001730+0000","last_fullsized":"2024-03-30T00:33:17.001730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:59.309835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131339+0000","last_change":"2024-03-30T00:33:15.168245+0000","last_active":"2024-03-30T00:33:19.131339+0000","last_peered":"2024-03-30T00:33:19.131339+0000","last_clean":"2024-03-30T00:33:19.131339+0000","last_became_active":"2024-03-30T00:33:15.167852+0000","last_became_peered":"2024-03-30T00:33:15.167852+0000","last_unstale":"2024-03-30T00:33:19.131339+0000","last_undegraded":"2024-03-30T00:33:19.131339+0000","last_fullsized":"2024-03-30T00:33:19.131339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:33:28.780614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002270+0000","last_change":"2024-03-30T00:33:15.679802+0000","last_active":"2024-03-30T00:33:17.002270+0000","last_peered":"2024-03-30T00:33:17.002270+0000","last_clean":"2024-03-30T00:33:17.002270+0000","last_became_active":"2024-03-30T00:33:15.679468+0000","last_became_peered":"2024-03-30T00:33:15.679468+0000","last_unstale":"2024-03-30T00:33:17.002270+0000","last_undegraded":"2024-03-30T00:33:17.002270+0000","last_fullsized":"2024-03-30T00:33:17.002270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:03:03.223713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131563+0000","last_change":"2024-03-30T00:33:15.170052+0000","last_active":"2024-03-30T00:33:19.131563+0000","last_peered":"2024-03-30T00:33:19.131563+0000","last_clean":"2024-03-30T00:33:19.131563+0000","last_became_active":"2024-03-30T00:33:15.169706+0000","last_became_peered":"2024-03-30T00:33:15.169706+0000","last_unstale":"2024-03-30T00:33:19.131563+0000","last_undegraded":"2024-03-30T00:33:19.131563+0000","last_fullsized":"2024-03-30T00:33:19.131563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:32.043071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.668729+0000","last_change":"2024-03-30T00:33:15.667061+0000","last_active":"2024-03-30T00:33:15.668729+0000","last_peered":"2024-03-30T00:33:15.668729+0000","last_clean":"2024-03-30T00:33:15.668729+0000","last_became_active":"2024-03-30T00:33:15.666763+0000","last_became_peered":"2024-03-30T00:33:15.666763+0000","last_unstale":"2024-03-30T00:33:15.668729+0000","last_undegraded":"2024-03-30T00:33:15.668729+0000","last_fullsized":"2024-03-30T00:33:15.668729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:46.558297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.133165+0000","last_change":"2024-03-30T00:33:15.176152+0000","last_active":"2024-03-30T00:33:19.133165+0000","last_peered":"2024-03-30T00:33:19.133165+0000","last_clean":"2024-03-30T00:33:19.133165+0000","last_became_active":"2024-03-30T00:33:15.175640+0000","last_became_peered":"2024-03-30T00:33:15.175640+0000","last_unstale":"2024-03-30T00:33:19.133165+0000","last_undegraded":"2024-03-30T00:33:19.133165+0000","last_fullsized":"2024-03-30T00:33:19.133165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:58:11.575577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290545+0000","last_change":"2024-03-30T00:33:15.672957+0000","last_active":"2024-03-30T00:33:17.290545+0000","last_peered":"2024-03-30T00:33:17.290545+0000","last_clean":"2024-03-30T00:33:17.290545+0000","last_became_active":"2024-03-30T00:33:15.668868+0000","last_became_peered":"2024-03-30T00:33:15.668868+0000","last_unstale":"2024-03-30T00:33:17.290545+0000","last_undegraded":"2024-03-30T00:33:17.290545+0000","last_fullsized":"2024-03-30T00:33:17.290545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:50.342693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.787067+0000","last_change":"2024-03-30T00:33:15.160345+0000","last_active":"2024-03-30T00:33:16.787067+0000","last_peered":"2024-03-30T00:33:16.787067+0000","last_clean":"2024-03-30T00:33:16.787067+0000","last_became_active":"2024-03-30T00:33:15.160026+0000","last_became_peered":"2024-03-30T00:33:15.160026+0000","last_unstale":"2024-03-30T00:33:16.787067+0000","last_undegraded":"2024-03-30T00:33:16.787067+0000","last_fullsized":"2024-03-30T00:33:16.787067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:06:51.177680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001362+0000","last_change":"2024-03-30T00:33:15.674058+0000","last_active":"2024-03-30T00:33:17.001362+0000","last_peered":"2024-03-30T00:33:17.001362+0000","last_clean":"2024-03-30T00:33:17.001362+0000","last_became_active":"2024-03-30T00:33:15.673165+0000","last_became_peered":"2024-03-30T00:33:15.673165+0000","last_unstale":"2024-03-30T00:33:17.001362+0000","last_undegraded":"2024-03-30T00:33:17.001362+0000","last_fullsized":"2024-03-30T00:33:17.001362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:14.869963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131675+0000","last_change":"2024-03-30T00:33:15.173178+0000","last_active":"2024-03-30T00:33:19.131675+0000","last_peered":"2024-03-30T00:33:19.131675+0000","last_clean":"2024-03-30T00:33:19.131675+0000","last_became_active":"2024-03-30T00:33:15.172830+0000","last_became_peered":"2024-03-30T00:33:15.172830+0000","last_unstale":"2024-03-30T00:33:19.131675+0000","last_undegraded":"2024-03-30T00:33:19.131675+0000","last_fullsized":"2024-03-30T00:33:19.131675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:49:09.823855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001435+0000","last_change":"2024-03-30T00:33:15.693485+0000","last_active":"2024-03-30T00:33:17.001435+0000","last_peered":"2024-03-30T00:33:17.001435+0000","last_clean":"2024-03-30T00:33:17.001435+0000","last_became_active":"2024-03-30T00:33:15.693066+0000","last_became_peered":"2024-03-30T00:33:15.693066+0000","last_unstale":"2024-03-30T00:33:17.001435+0000","last_undegraded":"2024-03-30T00:33:17.001435+0000","last_fullsized":"2024-03-30T00:33:17.001435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:38.161331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118339+0000","last_change":"2024-03-30T00:33:15.170808+0000","last_active":"2024-03-30T00:33:20.118339+0000","last_peered":"2024-03-30T00:33:20.118339+0000","last_clean":"2024-03-30T00:33:20.118339+0000","last_became_active":"2024-03-30T00:33:15.170448+0000","last_became_peered":"2024-03-30T00:33:15.170448+0000","last_unstale":"2024-03-30T00:33:20.118339+0000","last_undegraded":"2024-03-30T00:33:20.118339+0000","last_fullsized":"2024-03-30T00:33:20.118339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:08:28.179041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001385+0000","last_change":"2024-03-30T00:33:15.687619+0000","last_active":"2024-03-30T00:33:17.001385+0000","last_peered":"2024-03-30T00:33:17.001385+0000","last_clean":"2024-03-30T00:33:17.001385+0000","last_became_active":"2024-03-30T00:33:15.687313+0000","last_became_peered":"2024-03-30T00:33:15.687313+0000","last_unstale":"2024-03-30T00:33:17.001385+0000","last_undegraded":"2024-03-30T00:33:17.001385+0000","last_fullsized":"2024-03-30T00:33:17.001385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:36:51.182558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001678+0000","last_change":"2024-03-30T00:33:15.168601+0000","last_active":"2024-03-30T00:33:17.001678+0000","last_peered":"2024-03-30T00:33:17.001678+0000","last_clean":"2024-03-30T00:33:17.001678+0000","last_became_active":"2024-03-30T00:33:15.168065+0000","last_became_peered":"2024-03-30T00:33:15.168065+0000","last_unstale":"2024-03-30T00:33:17.001678+0000","last_undegraded":"2024-03-30T00:33:17.001678+0000","last_fullsized":"2024-03-30T00:33:17.001678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:22:36.477280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.054439+0000","last_change":"2024-03-30T00:33:15.678299+0000","last_active":"2024-03-30T00:33:19.054439+0000","last_peered":"2024-03-30T00:33:19.054439+0000","last_clean":"2024-03-30T00:33:19.054439+0000","last_became_active":"2024-03-30T00:33:15.675499+0000","last_became_peered":"2024-03-30T00:33:15.675499+0000","last_unstale":"2024-03-30T00:33:19.054439+0000","last_undegraded":"2024-03-30T00:33:19.054439+0000","last_fullsized":"2024-03-30T00:33:19.054439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:15.317125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000985+0000","last_change":"2024-03-30T00:33:15.172058+0000","last_active":"2024-03-30T00:33:17.000985+0000","last_peered":"2024-03-30T00:33:17.000985+0000","last_clean":"2024-03-30T00:33:17.000985+0000","last_became_active":"2024-03-30T00:33:15.167857+0000","last_became_peered":"2024-03-30T00:33:15.167857+0000","last_unstale":"2024-03-30T00:33:17.000985+0000","last_undegraded":"2024-03-30T00:33:17.000985+0000","last_fullsized":"2024-03-30T00:33:17.000985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:03:35.951559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001350+0000","last_change":"2024-03-30T00:33:15.682519+0000","last_active":"2024-03-30T00:33:17.001350+0000","last_peered":"2024-03-30T00:33:17.001350+0000","last_clean":"2024-03-30T00:33:17.001350+0000","last_became_active":"2024-03-30T00:33:15.681055+0000","last_became_peered":"2024-03-30T00:33:15.681055+0000","last_unstale":"2024-03-30T00:33:17.001350+0000","last_undegraded":"2024-03-30T00:33:17.001350+0000","last_fullsized":"2024-03-30T00:33:17.001350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:38.160151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290650+0000","last_change":"2024-03-30T00:33:15.673411+0000","last_active":"2024-03-30T00:33:17.290650+0000","last_peered":"2024-03-30T00:33:17.290650+0000","last_clean":"2024-03-30T00:33:17.290650+0000","last_became_active":"2024-03-30T00:33:15.672852+0000","last_became_peered":"2024-03-30T00:33:15.672852+0000","last_unstale":"2024-03-30T00:33:17.290650+0000","last_undegraded":"2024-03-30T00:33:17.290650+0000","last_fullsized":"2024-03-30T00:33:17.290650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:55:46.705785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660735+0000","last_change":"2024-03-30T00:33:15.680624+0000","last_active":"2024-03-30T00:33:16.660735+0000","last_peered":"2024-03-30T00:33:16.660735+0000","last_clean":"2024-03-30T00:33:16.660735+0000","last_became_active":"2024-03-30T00:33:15.679900+0000","last_became_peered":"2024-03-30T00:33:15.679900+0000","last_unstale":"2024-03-30T00:33:16.660735+0000","last_undegraded":"2024-03-30T00:33:16.660735+0000","last_fullsized":"2024-03-30T00:33:16.660735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:44.484608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660521+0000","last_change":"2024-03-30T00:33:15.672612+0000","last_active":"2024-03-30T00:33:16.660521+0000","last_peered":"2024-03-30T00:33:16.660521+0000","last_clean":"2024-03-30T00:33:16.660521+0000","last_became_active":"2024-03-30T00:33:15.671536+0000","last_became_peered":"2024-03-30T00:33:15.671536+0000","last_unstale":"2024-03-30T00:33:16.660521+0000","last_undegraded":"2024-03-30T00:33:16.660521+0000","last_fullsized":"2024-03-30T00:33:16.660521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:40.783988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002065+0000","last_change":"2024-03-30T00:33:15.681577+0000","last_active":"2024-03-30T00:33:17.002065+0000","last_peered":"2024-03-30T00:33:17.002065+0000","last_clean":"2024-03-30T00:33:17.002065+0000","last_became_active":"2024-03-30T00:33:15.681313+0000","last_became_peered":"2024-03-30T00:33:15.681313+0000","last_unstale":"2024-03-30T00:33:17.002065+0000","last_undegraded":"2024-03-30T00:33:17.002065+0000","last_fullsized":"2024-03-30T00:33:17.002065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:42:15.178957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001296+0000","last_change":"2024-03-30T00:33:15.688172+0000","last_active":"2024-03-30T00:33:17.001296+0000","last_peered":"2024-03-30T00:33:17.001296+0000","last_clean":"2024-03-30T00:33:17.001296+0000","last_became_active":"2024-03-30T00:33:15.687860+0000","last_became_peered":"2024-03-30T00:33:15.687860+0000","last_unstale":"2024-03-30T00:33:17.001296+0000","last_undegraded":"2024-03-30T00:33:17.001296+0000","last_fullsized":"2024-03-30T00:33:17.001296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:28.762798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290827+0000","last_change":"2024-03-30T00:33:15.673868+0000","last_active":"2024-03-30T00:33:17.290827+0000","last_peered":"2024-03-30T00:33:17.290827+0000","last_clean":"2024-03-30T00:33:17.290827+0000","last_became_active":"2024-03-30T00:33:15.673467+0000","last_became_peered":"2024-03-30T00:33:15.673467+0000","last_unstale":"2024-03-30T00:33:17.290827+0000","last_undegraded":"2024-03-30T00:33:17.290827+0000","last_fullsized":"2024-03-30T00:33:17.290827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:07:39.448967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.661014+0000","last_change":"2024-03-30T00:33:15.684639+0000","last_active":"2024-03-30T00:33:16.661014+0000","last_peered":"2024-03-30T00:33:16.661014+0000","last_clean":"2024-03-30T00:33:16.661014+0000","last_became_active":"2024-03-30T00:33:15.684357+0000","last_became_peered":"2024-03-30T00:33:15.684357+0000","last_unstale":"2024-03-30T00:33:16.661014+0000","last_undegraded":"2024-03-30T00:33:16.661014+0000","last_fullsized":"2024-03-30T00:33:16.661014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:02:50.995640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001730+0000","last_change":"2024-03-30T00:33:15.680810+0000","last_active":"2024-03-30T00:33:17.001730+0000","last_peered":"2024-03-30T00:33:17.001730+0000","last_clean":"2024-03-30T00:33:17.001730+0000","last_became_active":"2024-03-30T00:33:15.680457+0000","last_became_peered":"2024-03-30T00:33:15.680457+0000","last_unstale":"2024-03-30T00:33:17.001730+0000","last_undegraded":"2024-03-30T00:33:17.001730+0000","last_fullsized":"2024-03-30T00:33:17.001730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:02.575683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531880+0000","last_change":"2024-03-30T00:33:15.661644+0000","last_active":"2024-03-30T00:33:16.531880+0000","last_peered":"2024-03-30T00:33:16.531880+0000","last_clean":"2024-03-30T00:33:16.531880+0000","last_became_active":"2024-03-30T00:33:15.661359+0000","last_became_peered":"2024-03-30T00:33:15.661359+0000","last_unstale":"2024-03-30T00:33:16.531880+0000","last_undegraded":"2024-03-30T00:33:16.531880+0000","last_fullsized":"2024-03-30T00:33:16.531880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:44:39.894342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000646+0000","last_change":"2024-03-30T00:33:15.671491+0000","last_active":"2024-03-30T00:33:17.000646+0000","last_peered":"2024-03-30T00:33:17.000646+0000","last_clean":"2024-03-30T00:33:17.000646+0000","last_became_active":"2024-03-30T00:33:15.671110+0000","last_became_peered":"2024-03-30T00:33:15.671110+0000","last_unstale":"2024-03-30T00:33:17.000646+0000","last_undegraded":"2024-03-30T00:33:17.000646+0000","last_fullsized":"2024-03-30T00:33:17.000646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:57.623730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693121+0000","last_change":"2024-03-30T00:33:15.686795+0000","last_active":"2024-03-30T00:33:15.693121+0000","last_peered":"2024-03-30T00:33:15.693121+0000","last_clean":"2024-03-30T00:33:15.693121+0000","last_became_active":"2024-03-30T00:33:15.685715+0000","last_became_peered":"2024-03-30T00:33:15.685715+0000","last_unstale":"2024-03-30T00:33:15.693121+0000","last_undegraded":"2024-03-30T00:33:15.693121+0000","last_fullsized":"2024-03-30T00:33:15.693121+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:37:53.838823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001008+0000","last_change":"2024-03-30T00:33:15.687369+0000","last_active":"2024-03-30T00:33:17.001008+0000","last_peered":"2024-03-30T00:33:17.001008+0000","last_clean":"2024-03-30T00:33:17.001008+0000","last_became_active":"2024-03-30T00:33:15.686852+0000","last_became_peered":"2024-03-30T00:33:15.686852+0000","last_unstale":"2024-03-30T00:33:17.001008+0000","last_undegraded":"2024-03-30T00:33:17.001008+0000","last_fullsized":"2024-03-30T00:33:17.001008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:09:21.003739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290614+0000","last_change":"2024-03-30T00:33:15.676840+0000","last_active":"2024-03-30T00:33:17.290614+0000","last_peered":"2024-03-30T00:33:17.290614+0000","last_clean":"2024-03-30T00:33:17.290614+0000","last_became_active":"2024-03-30T00:33:15.675884+0000","last_became_peered":"2024-03-30T00:33:15.675884+0000","last_unstale":"2024-03-30T00:33:17.290614+0000","last_undegraded":"2024-03-30T00:33:17.290614+0000","last_fullsized":"2024-03-30T00:33:17.290614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:02.900843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5078,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":124,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":21836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":901120,"data_stored":429559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8129999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8300000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.79}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0009999999999999}]}]},{"osd":10,"up_from":71,"seq":304942678033,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1523712,"data_stored":1021584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,8,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":1,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.8160000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":1.994},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":1.879}]},{"osd":3,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":1.153},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":1.075}]},{"osd":4,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":1.696},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":1.1160000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":1.7430000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":2.1970000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":2.153},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":2.181}]},{"osd":7,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.97999999999999998}]},{"osd":8,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":2.2360000000000002},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.1679999999999999}]},{"osd":9,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":2.3210000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":1.2090000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]}]},{"osd":9,"up_from":66,"seq":283467841556,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":429795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.433},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.44800000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.84899999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.47199999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.73399999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.626},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.78200000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.63700000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.89000000000000001}]},{"osd":8,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.86799999999999999}]},{"osd":10,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.91200000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103193,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":429603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59399999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.998}]},{"osd":2,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63200000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.66200000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":1.0640000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77400000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.94899999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":1.04},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.92900000000000005}]},{"osd":9,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":1.079}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":1.097}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]}]},{"osd":7,"up_from":50,"seq":214748364830,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1021158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.46000000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.622}]},{"osd":3,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.74299999999999999,"15min":0.73199999999999998},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.67100000000000004,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.56499999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0760000000000001,"5min":1.095,"15min":1.095},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.79600000000000004,"15min":0.79100000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1679999999999999,"5min":1.252,"15min":1.252},"last":0.84899999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.79900000000000004,"15min":0.79400000000000004},"min":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.64300000000000002}]},{"osd":8,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.77600000000000002}]},{"osd":9,"last update":"Sat Mar 30 00:32:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.75},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.67200000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528354,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":905216,"data_stored":429184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.34100000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63300000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.41399999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66200000000000003,"5min":0.873,"15min":0.873},"last":0.45700000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.628,"15min":0.622},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.52500000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84899999999999998,"5min":0.879,"15min":0.879},"last":0.58299999999999996}]},{"osd":2,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.55700000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.98799999999999999,"5min":1.002,"15min":1.002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.53700000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97899999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.442,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.64500000000000002,"15min":0.63300000000000001},"min":{"1min":0.54200000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.92400000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.55900000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.60899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97199999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.749,"15min":0.749},"min":{"1min":0.51500000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":1.238,"15min":1.238},"last":0.54100000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.80100000000000005,"15min":0.78800000000000003},"min":{"1min":0.64400000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.71399999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.91600000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:33:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.67000000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0169999999999999,"5min":1.105,"15min":1.105},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.63100000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.71999999999999997}]},{"osd":8,"last update":"Sat Mar 30 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.70499999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.495},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.748}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.629}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36096,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709056,"statfs":{"total":95995035648,"available":95958073344,"internally_reserved":0,"allocated":909312,"data_stored":430541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.63700000000000001,"15min":0.59599999999999997},"min":{"1min":0.44700000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.624}]},{"osd":2,"last update":"Sat Mar 30 00:32:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.503,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88300000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.625,"15min":0.59099999999999997},"min":{"1min":0.45300000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.84799999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.621,"15min":0.58399999999999996},"min":{"1min":0.50800000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.40300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.64500000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.50600000000000001,"15min":0.48299999999999998},"min":{"1min":0.40999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.53700000000000003,"15min":0.505},"min":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.51900000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.749},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.68400000000000005,"15min":0.65700000000000003},"min":{"1min":0.52400000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.3039999999999998,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.879}]},{"osd":6,"last update":"Sat Mar 30 00:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.70599999999999996,"15min":0.69299999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78400000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.64500000000000002,"15min":0.629},"min":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.65800000000000003,"15min":0.64700000000000002},"min":{"1min":0.504,"5min":0.438,"15min":0.438},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.80300000000000005}]},{"osd":8,"last update":"Sat Mar 30 00:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1,"5min":1,"15min":1},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.3570000000000002,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.95899999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":3.2719999999999998,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.72699999999999998}]},{"osd":10,"last update":"Sat Mar 30 00:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.03}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":925696,"data_stored":430455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59099999999999997,"15min":0.57199999999999995},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.623},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66600000000000004,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96699999999999997,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.755}]},{"osd":2,"last update":"Sat Mar 30 00:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68999999999999995,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88400000000000001,"5min":5.8550000000000004,"15min":5.8550000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.72699999999999998,"15min":0.81299999999999994},"min":{"1min":0.41999999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.872,"5min":5.282,"15min":5.282},"last":0.63900000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.59499999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.47999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.67900000000000005,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.57599999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.53000000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.91100000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.92400000000000004,"15min":0.95799999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.438}]},{"osd":6,"last update":"Sat Mar 30 00:32:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.95099999999999996,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.629,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98099999999999998,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.74399999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.125,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2470000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.81399999999999995}]},{"osd":8,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.72299999999999998}]},{"osd":9,"last update":"Sat Mar 30 00:32:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.77500000000000002}]},{"osd":10,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.82099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36120,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709032,"statfs":{"total":95995035648,"available":95958048768,"internally_reserved":0,"allocated":933888,"data_stored":431843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92700000000000005,"5min":1.131,"15min":1.131},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80100000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.66300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39100000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81899999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.439,"5min":0.375,"15min":0.375},"max":{"1min":0.96299999999999997,"5min":1.006,"15min":1.006},"last":0.94099999999999995}]},{"osd":3,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.43099999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.435,"5min":0.371,"15min":0.371},"max":{"1min":0.89000000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71999999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.878},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.65700000000000003,"15min":0.61899999999999999},"min":{"1min":0.64900000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.0449999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.40100000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.58299999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:32:46 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.755,"15min":0.73699999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.70099999999999996,"15min":0.68999999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.65700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.83299999999999996}]},{"osd":8,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.97999999999999998}]},{"osd":9,"last update":"Sat Mar 30 00:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.016},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.89000000000000001}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.077}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":3,"up_from":24,"seq":103079215150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":429378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.39700000000000002},"min":{"1min":0.35799999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.44600000000000001,"15min":0.42999999999999999},"min":{"1min":0.40200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64400000000000002,"5min":0.748,"15min":0.748},"last":0.50700000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.44400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.76500000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59799999999999998,"15min":0.57799999999999996},"min":{"1min":0.503,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82399999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.86499999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:32:43 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.67800000000000005,"15min":0.67500000000000004},"min":{"1min":0.627,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94299999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65200000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64800000000000002,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86699999999999999,"5min":0.996,"15min":0.996},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.66400000000000003,"15min":0.63},"min":{"1min":0.58199999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.748}]},{"osd":5,"last update":"Sat Mar 30 00:32:24 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.56899999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.70799999999999996,"15min":0.68700000000000006},"min":{"1min":0.60799999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97899999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72699999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:32:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.47999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.98199999999999998,"5min":1.202,"15min":1.202},"last":0.80700000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.81100000000000005,"15min":0.79900000000000004},"min":{"1min":0.63300000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.88100000000000001}]},{"osd":8,"last update":"Sat Mar 30 00:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.66700000000000004}]},{"osd":9,"last update":"Sat Mar 30 00:32:50 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.62}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.77700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986219,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":909312,"data_stored":429573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.497}]},{"osd":1,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.47799999999999998,"15min":0.44900000000000001},"min":{"1min":0.35799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.47199999999999998,"15min":0.45000000000000001},"min":{"1min":0.39600000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.71399999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.93899999999999995}]},{"osd":3,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.67900000000000005,"15min":0.65100000000000002},"min":{"1min":0.58699999999999997,"5min":0.439,"15min":0.439},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.68700000000000006,"15min":0.66300000000000003},"min":{"1min":0.498,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.74299999999999999}]},{"osd":5,"last update":"Sat Mar 30 00:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68899999999999995,"15min":0.67700000000000005},"min":{"1min":0.42999999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94399999999999995,"5min":1.137,"15min":1.137},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.69699999999999995,"15min":0.67000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.99099999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.69799999999999995,"15min":0.67900000000000005},"min":{"1min":0.56699999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.82399999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.45600000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.52400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.133}]},{"osd":8,"last update":"Sat Mar 30 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.107},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":1.089}]},{"osd":9,"last update":"Sat Mar 30 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.872},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":1.0549999999999999}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":1.167},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.91600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789991,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32592,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712560,"statfs":{"total":95995035648,"available":95961661440,"internally_reserved":0,"allocated":1515520,"data_stored":1020442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.81999999999999995,"15min":0.84799999999999998},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0009999999999999,"5min":8.077,"15min":8.077},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69199999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.48299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86099999999999999,"5min":0.878,"15min":0.878},"last":0.437},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63500000000000001,"15min":0.629},"min":{"1min":0.48599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.73799999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.38600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88900000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.86599999999999999,"15min":0.90500000000000003},"min":{"1min":0.436,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86199999999999999,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":0.54700000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:32:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.54200000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0169999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.49199999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97699999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58699999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68999999999999995,"15min":0.67400000000000004},"min":{"1min":0.61499999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.76600000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:33:08 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.99399999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71299999999999997,"15min":0.70799999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.623}]},{"osd":8,"last update":"Sat Mar 30 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.64500000000000002}]},{"osd":9,"last update":"Sat Mar 30 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67800000000000005}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.72199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:33:25.483 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T00:33:25.483 INFO:tasks.cephadm:Setting up client nodes... 2024-03-30T00:33:25.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-30T00:33:25.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:25 smithi142 ceph-mon[41779]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:33:25.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:25 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4114606688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:25.871 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:26.346 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:26 smithi099 ceph-mon[38079]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:33:26.346 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:26 smithi099 ceph-mon[38079]: pgmap v261: 97 pgs: 97 active+clean; 582 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2024-03-30T00:33:26.346 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:26 smithi099 ceph-mon[38079]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:33:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:26 smithi161 ceph-mon[42221]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:33:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:26 smithi161 ceph-mon[42221]: pgmap v261: 97 pgs: 97 active+clean; 582 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2024-03-30T00:33:26.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:26 smithi161 ceph-mon[42221]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:33:26.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:26 smithi142 ceph-mon[41779]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:33:26.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:26 smithi142 ceph-mon[41779]: pgmap v261: 97 pgs: 97 active+clean; 582 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2024-03-30T00:33:26.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:26 smithi142 ceph-mon[41779]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:33:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.070+0000 7f2519800700 1 -- 172.21.15.99:0/562287514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2514100dc0 msgr2=0x7f25141011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.070+0000 7f2519800700 1 --2- 172.21.15.99:0/562287514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2514100dc0 0x7f25141011e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f25040099c0 tx=0x7f2504009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.070+0000 7f2519800700 1 -- 172.21.15.99:0/562287514 shutdown_connections 2024-03-30T00:33:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.070+0000 7f2519800700 1 --2- 172.21.15.99:0/562287514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2514102960 0x7f2514109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.070+0000 7f2519800700 1 --2- 172.21.15.99:0/562287514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2514100dc0 0x7f25141011e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.070+0000 7f2519800700 1 --2- 172.21.15.99:0/562287514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2514101fc0 0x7f2514102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:27.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.070+0000 7f2519800700 1 -- 172.21.15.99:0/562287514 >> 172.21.15.99:0/562287514 conn(0x7f25140fc360 msgr2=0x7f25140fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:27.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.070+0000 7f2519800700 1 -- 172.21.15.99:0/562287514 shutdown_connections 2024-03-30T00:33:27.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.070+0000 7f2519800700 1 -- 172.21.15.99:0/562287514 wait complete. 2024-03-30T00:33:27.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.071+0000 7f2519800700 1 Processor -- start 2024-03-30T00:33:27.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.071+0000 7f2519800700 1 -- start start 2024-03-30T00:33:27.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2519800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2514100dc0 0x7f2514198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:27.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2519800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2514101fc0 0x7f25141993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:27.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2519800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2514102960 0x7f25141a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:27.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2519800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141a09a0 con 0x7f2514101fc0 2024-03-30T00:33:27.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2519800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141a0ae0 con 0x7f2514100dc0 2024-03-30T00:33:27.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2519800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141a0c70 con 0x7f2514102960 2024-03-30T00:33:27.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2512000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2514101fc0 0x7f25141993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:27.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2512000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2514101fc0 0x7f25141993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40476/0 (socket says 172.21.15.99:40476) 2024-03-30T00:33:27.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2512000700 1 -- 172.21.15.99:0/771687664 learned_addr learned my addr 172.21.15.99:0/771687664 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:27.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2513400700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2514102960 0x7f25141a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:27.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2512a00700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2514100dc0 0x7f2514198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:27.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2512000700 1 -- 172.21.15.99:0/771687664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2514102960 msgr2=0x7f25141a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:27.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2512000700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2514102960 0x7f25141a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:27.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2512000700 1 -- 172.21.15.99:0/771687664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2514100dc0 msgr2=0x7f2514198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:27.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2512000700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2514100dc0 0x7f2514198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:27.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2512000700 1 -- 172.21.15.99:0/771687664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25040095f0 con 0x7f2514101fc0 2024-03-30T00:33:27.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.072+0000 7f2512a00700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2514100dc0 0x7f2514198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.073+0000 7f2512000700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2514101fc0 0x7f25141993a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f250000e9d0 tx=0x7f250000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.073+0000 7f250b400700 1 -- 172.21.15.99:0/771687664 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500009790 con 0x7f2514101fc0 2024-03-30T00:33:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.073+0000 7f250b400700 1 -- 172.21.15.99:0/771687664 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f250000cbd0 con 0x7f2514101fc0 2024-03-30T00:33:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.073+0000 7f2519800700 1 -- 172.21.15.99:0/771687664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25141a0f50 con 0x7f2514101fc0 2024-03-30T00:33:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.073+0000 7f250b400700 1 -- 172.21.15.99:0/771687664 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500018680 con 0x7f2514101fc0 2024-03-30T00:33:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.073+0000 7f2519800700 1 -- 172.21.15.99:0/771687664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25141a13c0 con 0x7f2514101fc0 2024-03-30T00:33:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.074+0000 7f2519800700 1 -- 172.21.15.99:0/771687664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251404ea80 con 0x7f2514101fc0 2024-03-30T00:33:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.076+0000 7f250b400700 1 -- 172.21.15.99:0/771687664 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f250000cd40 con 0x7f2514101fc0 2024-03-30T00:33:27.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.077+0000 7f250b400700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f24f80a11b0 0x7f24f80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:27.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.077+0000 7f250b400700 1 -- 172.21.15.99:0/771687664 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2500014070 con 0x7f2514101fc0 2024-03-30T00:33:27.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.077+0000 7f2512a00700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f24f80a11b0 0x7f24f80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:27.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.078+0000 7f2512a00700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f24f80a11b0 0x7f24f80a3670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2504009990 tx=0x7f250401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:27.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.081+0000 7f250b400700 1 -- 172.21.15.99:0/771687664 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f25000529f0 con 0x7f2514101fc0 2024-03-30T00:33:27.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.365+0000 7f2519800700 1 -- 172.21.15.99:0/771687664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f251410c4c0 con 0x7f2514101fc0 2024-03-30T00:33:27.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.372+0000 7f250b400700 1 -- 172.21.15.99:0/771687664 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f25000962f0 con 0x7f2514101fc0 2024-03-30T00:33:27.379 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-03-30T00:33:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDXXQdmCS/gFRAAJtzoyyhIO8miVP5Z9Ar85w== 2024-03-30T00:33:27.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 -- 172.21.15.99:0/771687664 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f24f80a11b0 msgr2=0x7f24f80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:27.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f24f80a11b0 0x7f24f80a3670 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2504009990 tx=0x7f250401a040 comp rx=0 tx=0).stop 2024-03-30T00:33:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 -- 172.21.15.99:0/771687664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2514101fc0 msgr2=0x7f25141993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2514101fc0 0x7f25141993a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f250000e9d0 tx=0x7f250000ece0 comp rx=0 tx=0).stop 2024-03-30T00:33:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 -- 172.21.15.99:0/771687664 shutdown_connections 2024-03-30T00:33:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2514102960 0x7f25141a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f24f80a11b0 0x7f24f80a3670 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2514100dc0 0x7f2514198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 --2- 172.21.15.99:0/771687664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2514101fc0 0x7f25141993a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 -- 172.21.15.99:0/771687664 >> 172.21.15.99:0/771687664 conn(0x7f25140fc360 msgr2=0x7f2514107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 -- 172.21.15.99:0/771687664 shutdown_connections 2024-03-30T00:33:27.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:27.375+0000 7f2519800700 1 -- 172.21.15.99:0/771687664 wait complete. 2024-03-30T00:33:27.757 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:33:27.757 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-30T00:33:27.757 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-30T00:33:27.851 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-30T00:33:27.851 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-30T00:33:27.851 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph mgr dump --format=json 2024-03-30T00:33:28.249 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:28.408 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:28 smithi099 ceph-mon[38079]: pgmap v262: 97 pgs: 97 active+clean; 623 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.9 KiB/s wr, 18 op/s 2024-03-30T00:33:28.408 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:28 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/771687664' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-30T00:33:28.408 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:28 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/771687664' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-30T00:33:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:28 smithi161 ceph-mon[42221]: pgmap v262: 97 pgs: 97 active+clean; 623 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.9 KiB/s wr, 18 op/s 2024-03-30T00:33:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:28 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/771687664' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-30T00:33:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:28 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/771687664' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-30T00:33:28.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:28 smithi142 ceph-mon[41779]: pgmap v262: 97 pgs: 97 active+clean; 623 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.9 KiB/s wr, 18 op/s 2024-03-30T00:33:28.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:28 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/771687664' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-30T00:33:28.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:28 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/771687664' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-30T00:33:29.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.301+0000 7f6f7ca00700 1 -- 172.21.15.99:0/3456562313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f78106ef0 msgr2=0x7f6f781072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:29.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.301+0000 7f6f7ca00700 1 --2- 172.21.15.99:0/3456562313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f78106ef0 0x7f6f781072d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f680099c0 tx=0x7f6f68009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:29.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.301+0000 7f6f7ca00700 1 -- 172.21.15.99:0/3456562313 shutdown_connections 2024-03-30T00:33:29.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.301+0000 7f6f7ca00700 1 --2- 172.21.15.99:0/3456562313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f78101820 0x7f6f7810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:29.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.301+0000 7f6f7ca00700 1 --2- 172.21.15.99:0/3456562313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f78106ef0 0x7f6f781072d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:29.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.301+0000 7f6f7ca00700 1 --2- 172.21.15.99:0/3456562313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f78100e60 0x7f6f781012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:29.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.301+0000 7f6f7ca00700 1 -- 172.21.15.99:0/3456562313 >> 172.21.15.99:0/3456562313 conn(0x7f6f78077fa0 msgr2=0x7f6f780783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:29.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.301+0000 7f6f7ca00700 1 -- 172.21.15.99:0/3456562313 shutdown_connections 2024-03-30T00:33:29.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.301+0000 7f6f7ca00700 1 -- 172.21.15.99:0/3456562313 wait complete. 2024-03-30T00:33:29.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.302+0000 7f6f7ca00700 1 Processor -- start 2024-03-30T00:33:29.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.302+0000 7f6f7ca00700 1 -- start start 2024-03-30T00:33:29.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.303+0000 7f6f7ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f78100e60 0x7f6f780730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:29.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.303+0000 7f6f7ca00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f78101820 0x7f6f780735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:29.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.303+0000 7f6f7ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f78106ef0 0x7f6f78073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:29.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.303+0000 7f6f7ca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78075020 con 0x7f6f78106ef0 2024-03-30T00:33:29.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.303+0000 7f6f7ca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f781a9ab0 con 0x7f6f78100e60 2024-03-30T00:33:29.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.303+0000 7f6f7ca00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f781a9c20 con 0x7f6f78101820 2024-03-30T00:33:29.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.303+0000 7f6f76800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f78106ef0 0x7f6f78073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:29.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.303+0000 7f6f76800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f78106ef0 0x7f6f78073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40484/0 (socket says 172.21.15.99:40484) 2024-03-30T00:33:29.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.303+0000 7f6f76800700 1 -- 172.21.15.99:0/2007378225 learned_addr learned my addr 172.21.15.99:0/2007378225 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:29.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.303+0000 7f6f75e00700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f78100e60 0x7f6f780730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:29.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.303+0000 7f6f75400700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f78101820 0x7f6f780735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:29.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.304+0000 7f6f76800700 1 -- 172.21.15.99:0/2007378225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f78101820 msgr2=0x7f6f780735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:29.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.304+0000 7f6f76800700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f78101820 0x7f6f780735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:29.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.304+0000 7f6f76800700 1 -- 172.21.15.99:0/2007378225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f78100e60 msgr2=0x7f6f780730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:29.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.304+0000 7f6f76800700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f78100e60 0x7f6f780730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:29.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.304+0000 7f6f76800700 1 -- 172.21.15.99:0/2007378225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f680095f0 con 0x7f6f78106ef0 2024-03-30T00:33:29.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.304+0000 7f6f76800700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f78106ef0 0x7f6f78073b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000cca0 tx=0x7f6f7000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:29.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.304+0000 7f6f6ea00700 1 -- 172.21.15.99:0/2007378225 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000ab40 con 0x7f6f78106ef0 2024-03-30T00:33:29.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.304+0000 7f6f6ea00700 1 -- 172.21.15.99:0/2007378225 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6f7000aca0 con 0x7f6f78106ef0 2024-03-30T00:33:29.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.304+0000 7f6f7ca00700 1 -- 172.21.15.99:0/2007378225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f781a9f00 con 0x7f6f78106ef0 2024-03-30T00:33:29.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.304+0000 7f6f6ea00700 1 -- 172.21.15.99:0/2007378225 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f700115e0 con 0x7f6f78106ef0 2024-03-30T00:33:29.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.304+0000 7f6f7ca00700 1 -- 172.21.15.99:0/2007378225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f781aa380 con 0x7f6f78106ef0 2024-03-30T00:33:29.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.305+0000 7f6f7ca00700 1 -- 172.21.15.99:0/2007378225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f781092d0 con 0x7f6f78106ef0 2024-03-30T00:33:29.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.307+0000 7f6f6ea00700 1 -- 172.21.15.99:0/2007378225 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f6f70012420 con 0x7f6f78106ef0 2024-03-30T00:33:29.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.308+0000 7f6f6ea00700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6f5c0a11b0 0x7f6f5c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:29.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.308+0000 7f6f6ea00700 1 -- 172.21.15.99:0/2007378225 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6f700ccb50 con 0x7f6f78106ef0 2024-03-30T00:33:29.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.309+0000 7f6f75e00700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6f5c0a11b0 0x7f6f5c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:29.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.309+0000 7f6f75e00700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6f5c0a11b0 0x7f6f5c0a3670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6f68000c00 tx=0x7f6f68009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:29.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.312+0000 7f6f6ea00700 1 -- 172.21.15.99:0/2007378225 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6f70096340 con 0x7f6f78106ef0 2024-03-30T00:33:29.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.573+0000 7f6f7ca00700 1 -- 172.21.15.99:0/2007378225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6f781024f0 con 0x7f6f78106ef0 2024-03-30T00:33:29.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.578+0000 7f6f6ea00700 1 -- 172.21.15.99:0/2007378225 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260275 (secure 0 0 0) 0x7f6f70099960 con 0x7f6f78106ef0 2024-03-30T00:33:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:29.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 -- 172.21.15.99:0/2007378225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6f5c0a11b0 msgr2=0x7f6f5c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:29.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6f5c0a11b0 0x7f6f5c0a3670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6f68000c00 tx=0x7f6f68009f10 comp rx=0 tx=0).stop 2024-03-30T00:33:29.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 -- 172.21.15.99:0/2007378225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f78106ef0 msgr2=0x7f6f78073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:29.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f78106ef0 0x7f6f78073b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000cca0 tx=0x7f6f7000a060 comp rx=0 tx=0).stop 2024-03-30T00:33:29.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 -- 172.21.15.99:0/2007378225 shutdown_connections 2024-03-30T00:33:29.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f78106ef0 0x7f6f78073b30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:29.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6f5c0a11b0 0x7f6f5c0a3670 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:29.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f78100e60 0x7f6f780730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:29.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 --2- 172.21.15.99:0/2007378225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f78101820 0x7f6f780735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:29.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 -- 172.21.15.99:0/2007378225 >> 172.21.15.99:0/2007378225 conn(0x7f6f78077fa0 msgr2=0x7f6f780ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:29.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 -- 172.21.15.99:0/2007378225 shutdown_connections 2024-03-30T00:33:29.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:29.583+0000 7f6f7ca00700 1 -- 172.21.15.99:0/2007378225 wait complete. 2024-03-30T00:33:29.972 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1501731896},{"type":"v1","addr":"172.21.15.99:6801","nonce":1501731896}]},"active_addr":"172.21.15.99:6801/1501731896","active_change":"2024-03-30T00:27:08.933104+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14220,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.99:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":1235964974}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3421332352}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":429965107}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":1630436412}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":2648617909}]}]} 2024-03-30T00:33:29.977 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-30T00:33:29.977 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-30T00:33:29.977 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-03-30T00:33:30.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:30 smithi099 ceph-mon[38079]: pgmap v263: 97 pgs: 97 active+clean; 623 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.3 KiB/s wr, 16 op/s 2024-03-30T00:33:30.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:30 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2007378225' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-30T00:33:30.336 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:30.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:30 smithi161 ceph-mon[42221]: pgmap v263: 97 pgs: 97 active+clean; 623 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.3 KiB/s wr, 16 op/s 2024-03-30T00:33:30.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:30 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2007378225' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-30T00:33:30.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:30 smithi142 ceph-mon[41779]: pgmap v263: 97 pgs: 97 active+clean; 623 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.3 KiB/s wr, 16 op/s 2024-03-30T00:33:30.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:30 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2007378225' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-30T00:33:31.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.485+0000 7f89d7e00700 1 -- 172.21.15.99:0/818271210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89d8100e10 msgr2=0x7f89d81011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:31.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.485+0000 7f89d7e00700 1 --2- 172.21.15.99:0/818271210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89d8100e10 0x7f89d81011f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009990 tx=0x7f89c8009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:31.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.485+0000 7f89d7e00700 1 -- 172.21.15.99:0/818271210 shutdown_connections 2024-03-30T00:33:31.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.485+0000 7f89d7e00700 1 --2- 172.21.15.99:0/818271210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d8105ef0 0x7f89d810dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:31.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.485+0000 7f89d7e00700 1 --2- 172.21.15.99:0/818271210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89d8100e10 0x7f89d81011f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:31.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.485+0000 7f89d7e00700 1 --2- 172.21.15.99:0/818271210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d81017c0 0x7f89d8105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:31.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.485+0000 7f89d7e00700 1 -- 172.21.15.99:0/818271210 >> 172.21.15.99:0/818271210 conn(0x7f89d80fc6e0 msgr2=0x7f89d80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:31.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.485+0000 7f89d7e00700 1 -- 172.21.15.99:0/818271210 shutdown_connections 2024-03-30T00:33:31.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.485+0000 7f89d7e00700 1 -- 172.21.15.99:0/818271210 wait complete. 2024-03-30T00:33:31.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.486+0000 7f89d7e00700 1 Processor -- start 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d7e00700 1 -- start start 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d7e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d8100e10 0x7f89d81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89d81017c0 0x7f89d81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d8105ef0 0x7f89d819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d7e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d819e0b0 con 0x7f89d81017c0 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d7e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d819e220 con 0x7f89d8105ef0 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d7e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d819e3e0 con 0x7f89d8100e10 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d6000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89d81017c0 0x7f89d81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d6000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89d81017c0 0x7f89d81996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40496/0 (socket says 172.21.15.99:40496) 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d6000700 1 -- 172.21.15.99:0/2881443716 learned_addr learned my addr 172.21.15.99:0/2881443716 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d6a00700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d8100e10 0x7f89d81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.487+0000 7f89d7400700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d8105ef0 0x7f89d819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.488+0000 7f89d6000700 1 -- 172.21.15.99:0/2881443716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d8100e10 msgr2=0x7f89d81991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.488+0000 7f89d6000700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d8100e10 0x7f89d81991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:31.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.488+0000 7f89d6000700 1 -- 172.21.15.99:0/2881443716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d8105ef0 msgr2=0x7f89d819daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:31.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.488+0000 7f89d6000700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d8105ef0 0x7f89d819daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:31.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.488+0000 7f89d6000700 1 -- 172.21.15.99:0/2881443716 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c80095f0 con 0x7f89d81017c0 2024-03-30T00:33:31.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.488+0000 7f89d6000700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89d81017c0 0x7f89d81996f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f89c400d980 tx=0x7f89c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:31.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.488+0000 7f89cf400700 1 -- 172.21.15.99:0/2881443716 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400f890 con 0x7f89d81017c0 2024-03-30T00:33:31.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.488+0000 7f89cf400700 1 -- 172.21.15.99:0/2881443716 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89c400fe80 con 0x7f89d81017c0 2024-03-30T00:33:31.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.488+0000 7f89cf400700 1 -- 172.21.15.99:0/2881443716 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c400e3b0 con 0x7f89d81017c0 2024-03-30T00:33:31.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.488+0000 7f89d7e00700 1 -- 172.21.15.99:0/2881443716 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d819e6c0 con 0x7f89d81017c0 2024-03-30T00:33:31.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.488+0000 7f89d7e00700 1 -- 172.21.15.99:0/2881443716 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d81a5f20 con 0x7f89d81017c0 2024-03-30T00:33:31.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.489+0000 7f89d7e00700 1 -- 172.21.15.99:0/2881443716 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d81022f0 con 0x7f89d81017c0 2024-03-30T00:33:31.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.491+0000 7f89cf400700 1 -- 172.21.15.99:0/2881443716 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f89c400f9f0 con 0x7f89d81017c0 2024-03-30T00:33:31.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.492+0000 7f89cf400700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f89bc0a11b0 0x7f89bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:31.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.493+0000 7f89cf400700 1 -- 172.21.15.99:0/2881443716 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f89c40cc410 con 0x7f89d81017c0 2024-03-30T00:33:31.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.493+0000 7f89d6a00700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f89bc0a11b0 0x7f89bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:31.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.493+0000 7f89d6a00700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f89bc0a11b0 0x7f89bc0a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f89c8006010 tx=0x7f89c800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:31.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.497+0000 7f89cf400700 1 -- 172.21.15.99:0/2881443716 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f89c4095af0 con 0x7f89d81017c0 2024-03-30T00:33:31.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.706+0000 7f89d7e00700 1 -- 172.21.15.99:0/2881443716 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89d8110a10 con 0x7f89d81017c0 2024-03-30T00:33:31.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.707+0000 7f89cf400700 1 -- 172.21.15.99:0/2881443716 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22120 (secure 0 0 0) 0x7f89c4099110 con 0x7f89d81017c0 2024-03-30T00:33:31.714 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:31.715 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":83,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:33:16.651168+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:33:31.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 -- 172.21.15.99:0/2881443716 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f89bc0a11b0 msgr2=0x7f89bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:31.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f89bc0a11b0 0x7f89bc0a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f89c8006010 tx=0x7f89c800b260 comp rx=0 tx=0).stop 2024-03-30T00:33:31.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 -- 172.21.15.99:0/2881443716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89d81017c0 msgr2=0x7f89d81996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:31.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89d81017c0 0x7f89d81996f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f89c400d980 tx=0x7f89c400dc90 comp rx=0 tx=0).stop 2024-03-30T00:33:31.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 -- 172.21.15.99:0/2881443716 shutdown_connections 2024-03-30T00:33:31.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f89bc0a11b0 0x7f89bc0a3670 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:31.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89d8105ef0 0x7f89d819daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:31.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89d8100e10 0x7f89d81991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:31.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 --2- 172.21.15.99:0/2881443716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89d81017c0 0x7f89d81996f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:31.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 -- 172.21.15.99:0/2881443716 >> 172.21.15.99:0/2881443716 conn(0x7f89d80fc6e0 msgr2=0x7f89d80fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:31.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 -- 172.21.15.99:0/2881443716 shutdown_connections 2024-03-30T00:33:31.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:31.710+0000 7f89d7e00700 1 -- 172.21.15.99:0/2881443716 wait complete. 2024-03-30T00:33:32.145 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-30T00:33:32.146 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-03-30T00:33:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:32 smithi161 ceph-mon[42221]: pgmap v264: 97 pgs: 97 active+clean; 631 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-03-30T00:33:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:32 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2881443716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:32.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:32 smithi099 ceph-mon[38079]: pgmap v264: 97 pgs: 97 active+clean; 631 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-03-30T00:33:32.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:32 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2881443716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:32.506 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:32.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:32 smithi142 ceph-mon[41779]: pgmap v264: 97 pgs: 97 active+clean; 631 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2024-03-30T00:33:32.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:32 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2881443716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:33.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.625+0000 7f5427a00700 1 -- 172.21.15.99:0/3227726533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5420072fe0 msgr2=0x7f54200733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:33.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.625+0000 7f5427a00700 1 --2- 172.21.15.99:0/3227726533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5420072fe0 0x7f54200733c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5414009990 tx=0x7f5414009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.625+0000 7f5427a00700 1 -- 172.21.15.99:0/3227726533 shutdown_connections 2024-03-30T00:33:33.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.625+0000 7f5427a00700 1 --2- 172.21.15.99:0/3227726533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f542010d810 0x7f542010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.625+0000 7f5427a00700 1 --2- 172.21.15.99:0/3227726533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5420072fe0 0x7f54200733c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.625+0000 7f5427a00700 1 --2- 172.21.15.99:0/3227726533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5420073900 0x7f542010d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.625+0000 7f5427a00700 1 -- 172.21.15.99:0/3227726533 >> 172.21.15.99:0/3227726533 conn(0x7f5420077fa0 msgr2=0x7f54200783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:33.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.625+0000 7f5427a00700 1 -- 172.21.15.99:0/3227726533 shutdown_connections 2024-03-30T00:33:33.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.625+0000 7f5427a00700 1 -- 172.21.15.99:0/3227726533 wait complete. 2024-03-30T00:33:33.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.626+0000 7f5427a00700 1 Processor -- start 2024-03-30T00:33:33.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.626+0000 7f5427a00700 1 -- start start 2024-03-30T00:33:33.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.626+0000 7f5427a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5420072fe0 0x7f5420199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:33.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f5427a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5420073900 0x7f54201998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:33.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f5427a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f542010d810 0x7f542019dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:33.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f5427a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019e230 con 0x7f5420073900 2024-03-30T00:33:33.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f5427a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019e3a0 con 0x7f5420072fe0 2024-03-30T00:33:33.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f5427a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542019e560 con 0x7f542010d810 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f541fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5420073900 0x7f54201998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f541fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5420073900 0x7f54201998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40504/0 (socket says 172.21.15.99:40504) 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f541fe00700 1 -- 172.21.15.99:0/1413273137 learned_addr learned my addr 172.21.15.99:0/1413273137 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f5425000700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5420072fe0 0x7f5420199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f5425a00700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f542010d810 0x7f542019dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f541fe00700 1 -- 172.21.15.99:0/1413273137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f542010d810 msgr2=0x7f542019dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f541fe00700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f542010d810 0x7f542019dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f541fe00700 1 -- 172.21.15.99:0/1413273137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5420072fe0 msgr2=0x7f5420199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f541fe00700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5420072fe0 0x7f5420199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.627+0000 7f541fe00700 1 -- 172.21.15.99:0/1413273137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54140095f0 con 0x7f5420073900 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.628+0000 7f541fe00700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5420073900 0x7f54201998c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c9b0 tx=0x7f540c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.628+0000 7f541d600700 1 -- 172.21.15.99:0/1413273137 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c007b00 con 0x7f5420073900 2024-03-30T00:33:33.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.628+0000 7f541d600700 1 -- 172.21.15.99:0/1413273137 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f540c00ce50 con 0x7f5420073900 2024-03-30T00:33:33.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.628+0000 7f5427a00700 1 -- 172.21.15.99:0/1413273137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542019e840 con 0x7f5420073900 2024-03-30T00:33:33.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.628+0000 7f541d600700 1 -- 172.21.15.99:0/1413273137 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c018420 con 0x7f5420073900 2024-03-30T00:33:33.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.628+0000 7f5427a00700 1 -- 172.21.15.99:0/1413273137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54201a60a0 con 0x7f5420073900 2024-03-30T00:33:33.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.629+0000 7f5427a00700 1 -- 172.21.15.99:0/1413273137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5420108a20 con 0x7f5420073900 2024-03-30T00:33:33.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.631+0000 7f541d600700 1 -- 172.21.15.99:0/1413273137 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f540c00f450 con 0x7f5420073900 2024-03-30T00:33:33.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.632+0000 7f541d600700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f54080a1230 0x7f54080a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:33.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.632+0000 7f541d600700 1 -- 172.21.15.99:0/1413273137 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f540c0cc890 con 0x7f5420073900 2024-03-30T00:33:33.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.632+0000 7f5425000700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f54080a1230 0x7f54080a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:33.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.633+0000 7f5425000700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f54080a1230 0x7f54080a36f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f541400b5c0 tx=0x7f5414005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:33.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.636+0000 7f541d600700 1 -- 172.21.15.99:0/1413273137 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f540c095f70 con 0x7f5420073900 2024-03-30T00:33:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.840+0000 7f5427a00700 1 -- 172.21.15.99:0/1413273137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5420074670 con 0x7f5420073900 2024-03-30T00:33:33.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.842+0000 7f541d600700 1 -- 172.21.15.99:0/1413273137 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22120 (secure 0 0 0) 0x7f540c00f700 con 0x7f5420073900 2024-03-30T00:33:33.850 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:33.850 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":83,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:33:16.651168+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:33:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 -- 172.21.15.99:0/1413273137 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f54080a1230 msgr2=0x7f54080a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f54080a1230 0x7f54080a36f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f541400b5c0 tx=0x7f5414005fb0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 -- 172.21.15.99:0/1413273137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5420073900 msgr2=0x7f54201998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5420073900 0x7f54201998c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f540c00c9b0 tx=0x7f540c00ccc0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 -- 172.21.15.99:0/1413273137 shutdown_connections 2024-03-30T00:33:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f542010d810 0x7f542019dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f54080a1230 0x7f54080a36f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5420072fe0 0x7f5420199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 --2- 172.21.15.99:0/1413273137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5420073900 0x7f54201998c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 -- 172.21.15.99:0/1413273137 >> 172.21.15.99:0/1413273137 conn(0x7f5420077fa0 msgr2=0x7f54200fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 -- 172.21.15.99:0/1413273137 shutdown_connections 2024-03-30T00:33:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:33.844+0000 7f5427a00700 1 -- 172.21.15.99:0/1413273137 wait complete. 2024-03-30T00:33:34.251 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-03-30T00:33:34.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-03-30T00:33:34.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-03-30T00:33:34.253 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-03-30T00:33:34.253 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-03-30T00:33:34.253 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-03-30T00:33:34.254 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.6 flush_pg_stats 2024-03-30T00:33:34.254 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.7 flush_pg_stats 2024-03-30T00:33:34.255 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.8 flush_pg_stats 2024-03-30T00:33:34.255 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.9 flush_pg_stats 2024-03-30T00:33:34.256 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.10 flush_pg_stats 2024-03-30T00:33:34.256 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph tell osd.11 flush_pg_stats 2024-03-30T00:33:34.411 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:34 smithi099 ceph-mon[38079]: pgmap v265: 97 pgs: 97 active+clean; 631 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-03-30T00:33:34.411 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:34 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1413273137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:34.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:34 smithi161 ceph-mon[42221]: pgmap v265: 97 pgs: 97 active+clean; 631 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-03-30T00:33:34.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:34 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1413273137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:34.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:34 smithi142 ceph-mon[41779]: pgmap v265: 97 pgs: 97 active+clean; 631 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2024-03-30T00:33:34.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:34 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1413273137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:33:35.786 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:36.097 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:36.097 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:36.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:36 smithi099 ceph-mon[38079]: pgmap v266: 97 pgs: 97 active+clean; 631 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-03-30T00:33:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:36.398 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:36.449 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:36.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:36 smithi161 ceph-mon[42221]: pgmap v266: 97 pgs: 97 active+clean; 631 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-03-30T00:33:36.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:36 smithi142 ceph-mon[41779]: pgmap v266: 97 pgs: 97 active+clean; 631 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-03-30T00:33:37.531 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:37.605 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:37.606 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:37.683 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:38.212 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:38 smithi099 ceph-mon[38079]: pgmap v267: 97 pgs: 97 active+clean; 632 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-30T00:33:38.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:38 smithi161 ceph-mon[42221]: pgmap v267: 97 pgs: 97 active+clean; 632 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-30T00:33:38.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:38 smithi142 ceph-mon[41779]: pgmap v267: 97 pgs: 97 active+clean; 632 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-03-30T00:33:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.579+0000 7fc632c00700 1 -- 172.21.15.99:0/488015164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc62c106ef0 msgr2=0x7fc62c1072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.579+0000 7fc632c00700 1 --2- 172.21.15.99:0/488015164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc62c106ef0 0x7fc62c1072d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0099c0 tx=0x7fc61c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.579+0000 7fc632c00700 1 -- 172.21.15.99:0/488015164 shutdown_connections 2024-03-30T00:33:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.579+0000 7fc632c00700 1 --2- 172.21.15.99:0/488015164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc62c101820 0x7fc62c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.579+0000 7fc632c00700 1 --2- 172.21.15.99:0/488015164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc62c106ef0 0x7fc62c1072d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.579+0000 7fc632c00700 1 --2- 172.21.15.99:0/488015164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc62c100e60 0x7fc62c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.579+0000 7fc632c00700 1 -- 172.21.15.99:0/488015164 >> 172.21.15.99:0/488015164 conn(0x7fc62c077fa0 msgr2=0x7fc62c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.579+0000 7fc632c00700 1 -- 172.21.15.99:0/488015164 shutdown_connections 2024-03-30T00:33:38.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.579+0000 7fc632c00700 1 -- 172.21.15.99:0/488015164 wait complete. 2024-03-30T00:33:38.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.580+0000 7fc632c00700 1 Processor -- start 2024-03-30T00:33:38.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.580+0000 7fc632c00700 1 -- start start 2024-03-30T00:33:38.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.581+0000 7fc632c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc62c100e60 0x7fc62c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:38.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.581+0000 7fc632c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc62c101820 0x7fc62c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:38.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.581+0000 7fc632c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc62c106ef0 0x7fc62c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:38.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.581+0000 7fc632c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c1a5530 con 0x7fc62c101820 2024-03-30T00:33:38.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.581+0000 7fc632c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c1a56a0 con 0x7fc62c106ef0 2024-03-30T00:33:38.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.581+0000 7fc632c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c1a5860 con 0x7fc62c100e60 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.581+0000 7fc62b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc62c101820 0x7fc62c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.581+0000 7fc62b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc62c101820 0x7fc62c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42668/0 (socket says 172.21.15.99:42668) 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.581+0000 7fc62b400700 1 -- 172.21.15.99:0/4168361821 learned_addr learned my addr 172.21.15.99:0/4168361821 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.581+0000 7fc62be00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc62c100e60 0x7fc62c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.581+0000 7fc630c00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc62c106ef0 0x7fc62c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc62b400700 1 -- 172.21.15.99:0/4168361821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc62c100e60 msgr2=0x7fc62c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc62b400700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc62c100e60 0x7fc62c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc62b400700 1 -- 172.21.15.99:0/4168361821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc62c106ef0 msgr2=0x7fc62c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc62b400700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc62c106ef0 0x7fc62c073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc62b400700 1 -- 172.21.15.99:0/4168361821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c0095f0 con 0x7fc62c101820 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc62be00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc62c100e60 0x7fc62c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc630c00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc62c106ef0 0x7fc62c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:38.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc62b400700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc62c101820 0x7fc62c0735f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc61800b550 tx=0x7fc61800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:38.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc628c00700 1 -- 172.21.15.99:0/4168361821 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618011830 con 0x7fc62c101820 2024-03-30T00:33:38.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc628c00700 1 -- 172.21.15.99:0/4168361821 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc618011e20 con 0x7fc62c101820 2024-03-30T00:33:38.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc628c00700 1 -- 172.21.15.99:0/4168361821 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618010dd0 con 0x7fc62c101820 2024-03-30T00:33:38.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc632c00700 1 -- 172.21.15.99:0/4168361821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62c1a5b40 con 0x7fc62c101820 2024-03-30T00:33:38.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.582+0000 7fc632c00700 1 -- 172.21.15.99:0/4168361821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c1a5ff0 con 0x7fc62c101820 2024-03-30T00:33:38.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.583+0000 7fc632c00700 1 -- 172.21.15.99:0/4168361821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc62c110f60 con 0x7fc62c101820 2024-03-30T00:33:38.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.585+0000 7fc628c00700 1 -- 172.21.15.99:0/4168361821 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc618011990 con 0x7fc62c101820 2024-03-30T00:33:38.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.586+0000 7fc628c00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc6100a11b0 0x7fc6100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:38.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.587+0000 7fc628c00700 1 -- 172.21.15.99:0/4168361821 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc6180cc910 con 0x7fc62c101820 2024-03-30T00:33:38.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.587+0000 7fc62be00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc6100a11b0 0x7fc6100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.587+0000 7fc628c00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080] conn(0x7fc6100aa990 0x7fc6100acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.587+0000 7fc628c00700 1 -- 172.21.15.99:0/4168361821 --> [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6100ad460 con 0x7fc6100aa990 2024-03-30T00:33:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.587+0000 7fc628c00700 1 -- 172.21.15.99:0/4168361821 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6180ccc80 con 0x7fc62c101820 2024-03-30T00:33:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.587+0000 7fc630c00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080] conn(0x7fc6100aa990 0x7fc6100acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.587+0000 7fc62be00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc6100a11b0 0x7fc6100a3670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc61c000c00 tx=0x7fc61c003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.588+0000 7fc630c00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080] conn(0x7fc6100aa990 0x7fc6100acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:38.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.588+0000 7fc628c00700 1 -- 172.21.15.99:0/4168361821 <== osd.6 v2:172.21.15.99:6818/2055935080 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc6100ad460 con 0x7fc6100aa990 2024-03-30T00:33:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.614+0000 7fc632c00700 1 -- 172.21.15.99:0/4168361821 --> [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc62c061430 con 0x7fc6100aa990 2024-03-30T00:33:38.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.615+0000 7fc628c00700 1 -- 172.21.15.99:0/4168361821 <== osd.6 v2:172.21.15.99:6818/2055935080 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc62c061430 con 0x7fc6100aa990 2024-03-30T00:33:38.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.615+0000 7fc632c00700 1 -- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080] conn(0x7fc6100aa990 msgr2=0x7fc6100acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.615+0000 7fc632c00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080] conn(0x7fc6100aa990 0x7fc6100acdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.615+0000 7fc632c00700 1 -- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc6100a11b0 msgr2=0x7fc6100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.615+0000 7fc632c00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc6100a11b0 0x7fc6100a3670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc61c000c00 tx=0x7fc61c003680 comp rx=0 tx=0).stop 2024-03-30T00:33:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.615+0000 7fc632c00700 1 -- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc62c101820 msgr2=0x7fc62c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.615+0000 7fc632c00700 1 --2- 172.21.15.99:0/4168361821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc62c101820 0x7fc62c0735f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc61800b550 tx=0x7fc61800b910 comp rx=0 tx=0).stop 2024-03-30T00:33:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.615+0000 7fc630c00700 1 -- 172.21.15.99:0/4168361821 reap_dead start 2024-03-30T00:33:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.616+0000 7fc632c00700 1 -- 172.21.15.99:0/4168361821 shutdown_connections 2024-03-30T00:33:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.616+0000 7fc632c00700 1 -- 172.21.15.99:0/4168361821 >> 172.21.15.99:0/4168361821 conn(0x7fc62c077fa0 msgr2=0x7fc62c0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.616+0000 7fc632c00700 1 -- 172.21.15.99:0/4168361821 shutdown_connections 2024-03-30T00:33:38.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:38.616+0000 7fc632c00700 1 -- 172.21.15.99:0/4168361821 wait complete. 2024-03-30T00:33:39.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.074+0000 7fc4d2000700 1 -- 172.21.15.99:0/2224654407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4cc100f70 msgr2=0x7fc4cc101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.074+0000 7fc4d2000700 1 --2- 172.21.15.99:0/2224654407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4cc100f70 0x7fc4cc101350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009990 tx=0x7fc4bc009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.074+0000 7fc4d2000700 1 -- 172.21.15.99:0/2224654407 shutdown_connections 2024-03-30T00:33:39.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.074+0000 7fc4d2000700 1 --2- 172.21.15.99:0/2224654407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4cc106050 0x7fc4cc10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.074+0000 7fc4d2000700 1 --2- 172.21.15.99:0/2224654407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4cc100f70 0x7fc4cc101350 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.074+0000 7fc4d2000700 1 --2- 172.21.15.99:0/2224654407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4cc101920 0x7fc4cc105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.074+0000 7fc4d2000700 1 -- 172.21.15.99:0/2224654407 >> 172.21.15.99:0/2224654407 conn(0x7fc4cc0fc7c0 msgr2=0x7fc4cc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.074+0000 7fc4d2000700 1 -- 172.21.15.99:0/2224654407 shutdown_connections 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.074+0000 7fc4d2000700 1 -- 172.21.15.99:0/2224654407 wait complete. 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.075+0000 7fc4d2000700 1 Processor -- start 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.075+0000 7fc4d2000700 1 -- start start 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.075+0000 7fc4d2000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4cc100f70 0x7fc4cc1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.075+0000 7fc4d2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4cc101920 0x7fc4cc199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4d2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4cc106050 0x7fc4cc19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4d2000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc19e1e0 con 0x7fc4cc101920 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4d2000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc19e350 con 0x7fc4cc106050 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4d2000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc19e510 con 0x7fc4cc100f70 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4caa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4cc101920 0x7fc4cc199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4cb400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4cc100f70 0x7fc4cc1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4cb400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4cc100f70 0x7fc4cc1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:38136/0 (socket says 172.21.15.99:38136) 2024-03-30T00:33:39.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4cb400700 1 -- 172.21.15.99:0/2996515086 learned_addr learned my addr 172.21.15.99:0/2996515086 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4cbe00700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4cc106050 0x7fc4cc19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4caa00700 1 -- 172.21.15.99:0/2996515086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4cc100f70 msgr2=0x7fc4cc1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4caa00700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc4cc100f70 0x7fc4cc1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4caa00700 1 -- 172.21.15.99:0/2996515086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4cc106050 msgr2=0x7fc4cc19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.076+0000 7fc4caa00700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4cc106050 0x7fc4cc19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.077+0000 7fc4caa00700 1 -- 172.21.15.99:0/2996515086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc0095f0 con 0x7fc4cc101920 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.077+0000 7fc4cbe00700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4cc106050 0x7fc4cc19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.077+0000 7fc4caa00700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4cc101920 0x7fc4cc199820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800d720 tx=0x7fc4b800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.077+0000 7fc4c3e00700 1 -- 172.21.15.99:0/2996515086 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8004220 con 0x7fc4cc101920 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.077+0000 7fc4c3e00700 1 -- 172.21.15.99:0/2996515086 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4b8004cc0 con 0x7fc4cc101920 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.077+0000 7fc4d2000700 1 -- 172.21.15.99:0/2996515086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4cc19e7f0 con 0x7fc4cc101920 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.077+0000 7fc4c3e00700 1 -- 172.21.15.99:0/2996515086 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8005620 con 0x7fc4cc101920 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.078+0000 7fc4d2000700 1 -- 172.21.15.99:0/2996515086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc1a6050 con 0x7fc4cc101920 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.079+0000 7fc4d2000700 1 -- 172.21.15.99:0/2996515086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc4cc110e20 con 0x7fc4cc101920 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.082+0000 7fc4c3e00700 1 -- 172.21.15.99:0/2996515086 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc4b8004750 con 0x7fc4cc101920 2024-03-30T00:33:39.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.085+0000 7fc4c3e00700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc4b00a1300 0x7fc4b00a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.085+0000 7fc4c3e00700 1 -- 172.21.15.99:0/2996515086 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc4b80cc5b0 con 0x7fc4cc101920 2024-03-30T00:33:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.085+0000 7fc4cb400700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc4b00a1300 0x7fc4b00a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.085+0000 7fc4c3e00700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041] conn(0x7fc4b00aaae0 0x7fc4b00acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.085+0000 7fc4c3e00700 1 -- 172.21.15.99:0/2996515086 --> [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4b00ad5b0 con 0x7fc4b00aaae0 2024-03-30T00:33:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.085+0000 7fc4c3e00700 1 -- 172.21.15.99:0/2996515086 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4b80cecc0 con 0x7fc4cc101920 2024-03-30T00:33:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.086+0000 7fc4cbe00700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041] conn(0x7fc4b00aaae0 0x7fc4b00acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.086+0000 7fc4cb400700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc4b00a1300 0x7fc4b00a37c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc006010 tx=0x7fc4bc00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.087+0000 7fc4cbe00700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041] conn(0x7fc4b00aaae0 0x7fc4b00acf00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:39.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.088+0000 7fc4c3e00700 1 -- 172.21.15.99:0/2996515086 <== osd.2 v2:172.21.15.161:6800/1904456041 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc4b00ad5b0 con 0x7fc4b00aaae0 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.115+0000 7fc4d2000700 1 -- 172.21.15.99:0/2996515086 --> [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc4cc0fdda0 con 0x7fc4b00aaae0 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.116+0000 7fc4c3e00700 1 -- 172.21.15.99:0/2996515086 <== osd.2 v2:172.21.15.161:6800/1904456041 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc4cc0fdda0 con 0x7fc4b00aaae0 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.117+0000 7fc4d2000700 1 -- 172.21.15.99:0/2996515086 >> [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041] conn(0x7fc4b00aaae0 msgr2=0x7fc4b00acf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.117+0000 7fc4d2000700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041] conn(0x7fc4b00aaae0 0x7fc4b00acf00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.117+0000 7fc4d2000700 1 -- 172.21.15.99:0/2996515086 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc4b00a1300 msgr2=0x7fc4b00a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.117+0000 7fc4d2000700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc4b00a1300 0x7fc4b00a37c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc006010 tx=0x7fc4bc00b540 comp rx=0 tx=0).stop 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.117+0000 7fc4d2000700 1 -- 172.21.15.99:0/2996515086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4cc101920 msgr2=0x7fc4cc199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.117+0000 7fc4d2000700 1 --2- 172.21.15.99:0/2996515086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4cc101920 0x7fc4cc199820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800d720 tx=0x7fc4b800da30 comp rx=0 tx=0).stop 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.117+0000 7fc4cbe00700 1 -- 172.21.15.99:0/2996515086 reap_dead start 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.117+0000 7fc4d2000700 1 -- 172.21.15.99:0/2996515086 shutdown_connections 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.117+0000 7fc4d2000700 1 -- 172.21.15.99:0/2996515086 >> 172.21.15.99:0/2996515086 conn(0x7fc4cc0fc7c0 msgr2=0x7fc4cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.117+0000 7fc4d2000700 1 -- 172.21.15.99:0/2996515086 shutdown_connections 2024-03-30T00:33:39.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.117+0000 7fc4d2000700 1 -- 172.21.15.99:0/2996515086 wait complete. 2024-03-30T00:33:39.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.512+0000 7f8697e00700 1 -- 172.21.15.99:0/3562126536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86981072f0 msgr2=0x7f869810dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.512+0000 7f8697e00700 1 --2- 172.21.15.99:0/3562126536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86981072f0 0x7f869810dc70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f869000b0d0 tx=0x7f869000b3e0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.513+0000 7f8697e00700 1 -- 172.21.15.99:0/3562126536 shutdown_connections 2024-03-30T00:33:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.513+0000 7f8697e00700 1 --2- 172.21.15.99:0/3562126536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86981072f0 0x7f869810dc70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.513+0000 7f8697e00700 1 --2- 172.21.15.99:0/3562126536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8698101d60 0x7f8698102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.513+0000 7f8697e00700 1 --2- 172.21.15.99:0/3562126536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8698102710 0x7f8698106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.513+0000 7f8697e00700 1 -- 172.21.15.99:0/3562126536 >> 172.21.15.99:0/3562126536 conn(0x7f86980fb5f0 msgr2=0x7f86980fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.513+0000 7f8697e00700 1 -- 172.21.15.99:0/3562126536 shutdown_connections 2024-03-30T00:33:39.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.513+0000 7f8697e00700 1 -- 172.21.15.99:0/3562126536 wait complete. 2024-03-30T00:33:39.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.514+0000 7f8697e00700 1 Processor -- start 2024-03-30T00:33:39.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.514+0000 7f8697e00700 1 -- start start 2024-03-30T00:33:39.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.514+0000 7f8697e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8698101d60 0x7f8698199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.514+0000 7f8697e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8698102710 0x7f86981998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.514+0000 7f8697e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86981072f0 0x7f869819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.514+0000 7f8697e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869819e220 con 0x7f8698102710 2024-03-30T00:33:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.514+0000 7f8697e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869819e390 con 0x7f86981072f0 2024-03-30T00:33:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.514+0000 7f8697e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869819e550 con 0x7f8698101d60 2024-03-30T00:33:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.514+0000 7f8696000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8698102710 0x7f86981998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.515+0000 7f8696000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8698102710 0x7f86981998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42724/0 (socket says 172.21.15.99:42724) 2024-03-30T00:33:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.515+0000 7f8696000700 1 -- 172.21.15.99:0/762964683 learned_addr learned my addr 172.21.15.99:0/762964683 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.515+0000 7f8697400700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86981072f0 0x7f869819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.515+0000 7f8696a00700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8698101d60 0x7f8698199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.515+0000 7f8697400700 1 -- 172.21.15.99:0/762964683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8698101d60 msgr2=0x7f8698199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.515+0000 7f8697400700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8698101d60 0x7f8698199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.515+0000 7f8697400700 1 -- 172.21.15.99:0/762964683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8698102710 msgr2=0x7f86981998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.515+0000 7f8697400700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8698102710 0x7f86981998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.515+0000 7f8697400700 1 -- 172.21.15.99:0/762964683 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8690009d00 con 0x7f86981072f0 2024-03-30T00:33:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.515+0000 7f8697400700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86981072f0 0x7f869819dc10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f869000b770 tx=0x7f8690007cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.516+0000 7f868f400700 1 -- 172.21.15.99:0/762964683 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000e040 con 0x7f86981072f0 2024-03-30T00:33:39.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.516+0000 7f868f400700 1 -- 172.21.15.99:0/762964683 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f869000bc50 con 0x7f86981072f0 2024-03-30T00:33:39.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.516+0000 7f8697e00700 1 -- 172.21.15.99:0/762964683 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869819e7d0 con 0x7f86981072f0 2024-03-30T00:33:39.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.516+0000 7f868f400700 1 -- 172.21.15.99:0/762964683 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690005760 con 0x7f86981072f0 2024-03-30T00:33:39.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.516+0000 7f8697e00700 1 -- 172.21.15.99:0/762964683 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86981a6090 con 0x7f86981072f0 2024-03-30T00:33:39.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.517+0000 7f8697e00700 1 -- 172.21.15.99:0/762964683 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8698069030 con 0x7f86981072f0 2024-03-30T00:33:39.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.519+0000 7f868f400700 1 -- 172.21.15.99:0/762964683 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8690004a50 con 0x7f86981072f0 2024-03-30T00:33:39.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.520+0000 7f868f400700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f867c0a11b0 0x7f867c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.520+0000 7f868f400700 1 -- 172.21.15.99:0/762964683 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f86900ce140 con 0x7f86981072f0 2024-03-30T00:33:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.520+0000 7f8696a00700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f867c0a11b0 0x7f867c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.521+0000 7f868f400700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367] conn(0x7f867c0aa860 0x7f867c0acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.521+0000 7f868f400700 1 -- 172.21.15.99:0/762964683 --> [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f867c0ad330 con 0x7f867c0aa860 2024-03-30T00:33:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.521+0000 7f868f400700 1 -- 172.21.15.99:0/762964683 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86900ce4b0 con 0x7f86981072f0 2024-03-30T00:33:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.521+0000 7f8696a00700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f867c0a11b0 0x7f867c0a3670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f869806a1d0 tx=0x7f8688009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.521+0000 7f8696000700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367] conn(0x7f867c0aa860 0x7f867c0acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.522+0000 7f8696000700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367] conn(0x7f867c0aa860 0x7f867c0acc80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:39.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.523+0000 7f868f400700 1 -- 172.21.15.99:0/762964683 <== osd.5 v2:172.21.15.161:6808/235239367 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f867c0ad330 con 0x7f867c0aa860 2024-03-30T00:33:39.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.539+0000 7f8697e00700 1 -- 172.21.15.99:0/762964683 --> [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8698110e70 con 0x7f867c0aa860 2024-03-30T00:33:39.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.540+0000 7f868f400700 1 -- 172.21.15.99:0/762964683 <== osd.5 v2:172.21.15.161:6808/235239367 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8698110e70 con 0x7f867c0aa860 2024-03-30T00:33:39.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.540+0000 7f8697e00700 1 -- 172.21.15.99:0/762964683 >> [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367] conn(0x7f867c0aa860 msgr2=0x7f867c0acc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.540+0000 7f8697e00700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367] conn(0x7f867c0aa860 0x7f867c0acc80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.540+0000 7f8697e00700 1 -- 172.21.15.99:0/762964683 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f867c0a11b0 msgr2=0x7f867c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.540+0000 7f8697e00700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f867c0a11b0 0x7f867c0a3670 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f869806a1d0 tx=0x7f8688009260 comp rx=0 tx=0).stop 2024-03-30T00:33:39.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.540+0000 7f8697e00700 1 -- 172.21.15.99:0/762964683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86981072f0 msgr2=0x7f869819dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.540+0000 7f8697e00700 1 --2- 172.21.15.99:0/762964683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86981072f0 0x7f869819dc10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f869000b770 tx=0x7f8690007cc0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.540+0000 7f8697400700 1 -- 172.21.15.99:0/762964683 reap_dead start 2024-03-30T00:33:39.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.540+0000 7f8697e00700 1 -- 172.21.15.99:0/762964683 shutdown_connections 2024-03-30T00:33:39.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.540+0000 7f8697e00700 1 -- 172.21.15.99:0/762964683 >> 172.21.15.99:0/762964683 conn(0x7f86980fb5f0 msgr2=0x7f86981003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:39.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.540+0000 7f8697e00700 1 -- 172.21.15.99:0/762964683 shutdown_connections 2024-03-30T00:33:39.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.541+0000 7f8697e00700 1 -- 172.21.15.99:0/762964683 wait complete. 2024-03-30T00:33:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.037+0000 7fa7b9600700 1 -- 172.21.15.99:0/2948426075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7b4101820 msgr2=0x7fa7b410dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.037+0000 7fa7b9600700 1 --2- 172.21.15.99:0/2948426075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7b4101820 0x7fa7b410dc10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00b0d0 tx=0x7fa7ac00b3e0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.038+0000 7fa7b9600700 1 -- 172.21.15.99:0/2948426075 shutdown_connections 2024-03-30T00:33:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.038+0000 7fa7b9600700 1 --2- 172.21.15.99:0/2948426075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7b4101820 0x7fa7b410dc10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.038+0000 7fa7b9600700 1 --2- 172.21.15.99:0/2948426075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7b4106ef0 0x7fa7b41072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.038+0000 7fa7b9600700 1 --2- 172.21.15.99:0/2948426075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b4100e60 0x7fa7b41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.038+0000 7fa7b9600700 1 -- 172.21.15.99:0/2948426075 >> 172.21.15.99:0/2948426075 conn(0x7fa7b4077fa0 msgr2=0x7fa7b40783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.038+0000 7fa7b9600700 1 -- 172.21.15.99:0/2948426075 shutdown_connections 2024-03-30T00:33:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.038+0000 7fa7b9600700 1 -- 172.21.15.99:0/2948426075 wait complete. 2024-03-30T00:33:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.039+0000 7fa7b9600700 1 Processor -- start 2024-03-30T00:33:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.040+0000 7fa7b9600700 1 -- start start 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.040+0000 7fa7b9600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b4100e60 0x7fa7b41993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.040+0000 7fa7b9600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7b4101820 0x7fa7b4199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.040+0000 7fa7b9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7b4106ef0 0x7fa7b419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.040+0000 7fa7b9600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b419e2c0 con 0x7fa7b4101820 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.040+0000 7fa7b9600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b419e430 con 0x7fa7b4106ef0 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.040+0000 7fa7b9600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b419e5f0 con 0x7fa7b4100e60 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.040+0000 7fa7b2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7b4101820 0x7fa7b4199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.040+0000 7fa7b2a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b4100e60 0x7fa7b41993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.041+0000 7fa7b2a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b4100e60 0x7fa7b41993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:38122/0 (socket says 172.21.15.99:38122) 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.041+0000 7fa7b2a00700 1 -- 172.21.15.99:0/743439327 learned_addr learned my addr 172.21.15.99:0/743439327 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.041+0000 7fa7b2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7b4101820 0x7fa7b4199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42692/0 (socket says 172.21.15.99:42692) 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.041+0000 7fa7b3400700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7b4106ef0 0x7fa7b419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.041+0000 7fa7b2a00700 1 -- 172.21.15.99:0/743439327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7b4106ef0 msgr2=0x7fa7b419dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.041+0000 7fa7b2a00700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7b4106ef0 0x7fa7b419dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.041+0000 7fa7b2a00700 1 -- 172.21.15.99:0/743439327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7b4101820 msgr2=0x7fa7b4199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.041+0000 7fa7b2a00700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7b4101820 0x7fa7b4199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.041+0000 7fa7b2a00700 1 -- 172.21.15.99:0/743439327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac009d00 con 0x7fa7b4100e60 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.041+0000 7fa7b2000700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7b4101820 0x7fa7b4199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.042+0000 7fa7b2a00700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b4100e60 0x7fa7b41993c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400e9a0 tx=0x7fa7a400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.042+0000 7fa7ab400700 1 -- 172.21.15.99:0/743439327 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4009790 con 0x7fa7b4100e60 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.042+0000 7fa7ab400700 1 -- 172.21.15.99:0/743439327 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7a4010430 con 0x7fa7b4100e60 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.042+0000 7fa7b9600700 1 -- 172.21.15.99:0/743439327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b419e8d0 con 0x7fa7b4100e60 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.042+0000 7fa7ab400700 1 -- 172.21.15.99:0/743439327 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a400cea0 con 0x7fa7b4100e60 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.042+0000 7fa7b9600700 1 -- 172.21.15.99:0/743439327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b41a6130 con 0x7fa7b4100e60 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.043+0000 7fa7b9600700 1 -- 172.21.15.99:0/743439327 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7b41024f0 con 0x7fa7b4100e60 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.045+0000 7fa7ab400700 1 -- 172.21.15.99:0/743439327 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa7a40105a0 con 0x7fa7b4100e60 2024-03-30T00:33:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.047+0000 7fa7ab400700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa7980a11b0 0x7fa7980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.047+0000 7fa7ab400700 1 -- 172.21.15.99:0/743439327 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa7a4014070 con 0x7fa7b4100e60 2024-03-30T00:33:39.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.047+0000 7fa7b2000700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa7980a11b0 0x7fa7980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.047+0000 7fa7ab400700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815] conn(0x7fa7980aa990 0x7fa7980acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:39.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.047+0000 7fa7ab400700 1 -- 172.21.15.99:0/743439327 --> [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7980ad460 con 0x7fa7980aa990 2024-03-30T00:33:39.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.047+0000 7fa7ab400700 1 -- 172.21.15.99:0/743439327 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7a40cd150 con 0x7fa7b4100e60 2024-03-30T00:33:39.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.047+0000 7fa7b2000700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa7980a11b0 0x7fa7980a3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa7b419a920 tx=0x7fa7a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:39.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.048+0000 7fa7b3400700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815] conn(0x7fa7980aa990 0x7fa7980acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:39.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.048+0000 7fa7b3400700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815] conn(0x7fa7980aa990 0x7fa7980acdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:39.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.049+0000 7fa7ab400700 1 -- 172.21.15.99:0/743439327 <== osd.7 v2:172.21.15.142:6816/445950815 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa7980ad460 con 0x7fa7980aa990 2024-03-30T00:33:39.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.067+0000 7fa7b9600700 1 -- 172.21.15.99:0/743439327 --> [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7b4110e70 con 0x7fa7980aa990 2024-03-30T00:33:39.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.067+0000 7fa7ab400700 1 -- 172.21.15.99:0/743439327 <== osd.7 v2:172.21.15.142:6816/445950815 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa7b4110e70 con 0x7fa7980aa990 2024-03-30T00:33:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.068+0000 7fa7b9600700 1 -- 172.21.15.99:0/743439327 >> [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815] conn(0x7fa7980aa990 msgr2=0x7fa7980acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.068+0000 7fa7b9600700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815] conn(0x7fa7980aa990 0x7fa7980acdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.068+0000 7fa7b9600700 1 -- 172.21.15.99:0/743439327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa7980a11b0 msgr2=0x7fa7980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.068+0000 7fa7b9600700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa7980a11b0 0x7fa7980a3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa7b419a920 tx=0x7fa7a0009260 comp rx=0 tx=0).stop 2024-03-30T00:33:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.068+0000 7fa7b9600700 1 -- 172.21.15.99:0/743439327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b4100e60 msgr2=0x7fa7b41993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.068+0000 7fa7b9600700 1 --2- 172.21.15.99:0/743439327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b4100e60 0x7fa7b41993c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400e9a0 tx=0x7fa7a400ecb0 comp rx=0 tx=0).stop 2024-03-30T00:33:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.068+0000 7fa7b3400700 1 -- 172.21.15.99:0/743439327 reap_dead start 2024-03-30T00:33:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.068+0000 7fa7b9600700 1 -- 172.21.15.99:0/743439327 shutdown_connections 2024-03-30T00:33:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.068+0000 7fa7b9600700 1 -- 172.21.15.99:0/743439327 >> 172.21.15.99:0/743439327 conn(0x7fa7b4077fa0 msgr2=0x7fa7b410c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.068+0000 7fa7b9600700 1 -- 172.21.15.99:0/743439327 shutdown_connections 2024-03-30T00:33:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:39.068+0000 7fa7b9600700 1 -- 172.21.15.99:0/743439327 wait complete. 2024-03-30T00:33:40.368 INFO:teuthology.orchestra.run.smithi099.stdout:193273528358 2024-03-30T00:33:40.368 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.6 2024-03-30T00:33:40.429 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:40 smithi099 ceph-mon[38079]: pgmap v268: 97 pgs: 97 active+clean; 632 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-30T00:33:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:40 smithi161 ceph-mon[42221]: pgmap v268: 97 pgs: 97 active+clean; 632 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-30T00:33:40.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:40 smithi142 ceph-mon[41779]: pgmap v268: 97 pgs: 97 active+clean; 632 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-30T00:33:40.775 INFO:teuthology.orchestra.run.smithi099.stdout:77309411384 2024-03-30T00:33:40.775 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-03-30T00:33:41.193 INFO:teuthology.orchestra.run.smithi099.stdout:158913789995 2024-03-30T00:33:41.194 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-03-30T00:33:42.022 INFO:teuthology.orchestra.run.smithi099.stdout:214748364834 2024-03-30T00:33:42.022 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.7 2024-03-30T00:33:42.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.772+0000 7f52db200700 1 -- 172.21.15.99:0/1896271095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52d4101820 msgr2=0x7f52d410dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.772+0000 7f52db200700 1 --2- 172.21.15.99:0/1896271095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52d4101820 0x7f52d410dc10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00b0d0 tx=0x7f52cc00b3e0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.772+0000 7f52db200700 1 -- 172.21.15.99:0/1896271095 shutdown_connections 2024-03-30T00:33:42.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.772+0000 7f52db200700 1 --2- 172.21.15.99:0/1896271095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52d4101820 0x7f52d410dc10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.772+0000 7f52db200700 1 --2- 172.21.15.99:0/1896271095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52d4106ef0 0x7f52d41072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.773+0000 7f52db200700 1 --2- 172.21.15.99:0/1896271095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d4100e60 0x7f52d41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.773+0000 7f52db200700 1 -- 172.21.15.99:0/1896271095 >> 172.21.15.99:0/1896271095 conn(0x7f52d4077fa0 msgr2=0x7f52d40783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:42.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.773+0000 7f52db200700 1 -- 172.21.15.99:0/1896271095 shutdown_connections 2024-03-30T00:33:42.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.773+0000 7f52db200700 1 -- 172.21.15.99:0/1896271095 wait complete. 2024-03-30T00:33:42.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.774+0000 7f52db200700 1 Processor -- start 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.774+0000 7f52db200700 1 -- start start 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.774+0000 7f52db200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52d4100e60 0x7f52d41972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.774+0000 7f52db200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d4101820 0x7f52d4197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.774+0000 7f52db200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52d4106ef0 0x7f52d419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.774+0000 7f52db200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d419c1a0 con 0x7f52d4100e60 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.774+0000 7f52db200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d419c310 con 0x7f52d4106ef0 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.774+0000 7f52db200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d419c4d0 con 0x7f52d4101820 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.774+0000 7f52d3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52d4100e60 0x7f52d41972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.774+0000 7f52d3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52d4100e60 0x7f52d41972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42770/0 (socket says 172.21.15.99:42770) 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.774+0000 7f52d3e00700 1 -- 172.21.15.99:0/2422638480 learned_addr learned my addr 172.21.15.99:0/2422638480 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52d9200700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52d4106ef0 0x7f52d419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52d3e00700 1 -- 172.21.15.99:0/2422638480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d4101820 msgr2=0x7f52d4197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52d3e00700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52d4101820 0x7f52d4197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52d3e00700 1 -- 172.21.15.99:0/2422638480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52d4106ef0 msgr2=0x7f52d419bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52d3e00700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52d4106ef0 0x7f52d419bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52d3e00700 1 -- 172.21.15.99:0/2422638480 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc009d00 con 0x7f52d4100e60 2024-03-30T00:33:42.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52d3e00700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52d4100e60 0x7f52d41972f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f52c400d980 tx=0x7f52c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:42.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52d0c00700 1 -- 172.21.15.99:0/2422638480 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4004220 con 0x7f52d4100e60 2024-03-30T00:33:42.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52d0c00700 1 -- 172.21.15.99:0/2422638480 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52c4007640 con 0x7f52d4100e60 2024-03-30T00:33:42.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52d0c00700 1 -- 172.21.15.99:0/2422638480 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40055e0 con 0x7f52d4100e60 2024-03-30T00:33:42.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52db200700 1 -- 172.21.15.99:0/2422638480 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d419c7b0 con 0x7f52d4100e60 2024-03-30T00:33:42.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.775+0000 7f52db200700 1 -- 172.21.15.99:0/2422638480 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d41a4010 con 0x7f52d4100e60 2024-03-30T00:33:42.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.776+0000 7f52db200700 1 -- 172.21.15.99:0/2422638480 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52d4110ec0 con 0x7f52d4100e60 2024-03-30T00:33:42.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.779+0000 7f52d0c00700 1 -- 172.21.15.99:0/2422638480 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f52c4004750 con 0x7f52d4100e60 2024-03-30T00:33:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.780+0000 7f52d0c00700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f52b80a12b0 0x7f52b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.780+0000 7f52d0c00700 1 -- 172.21.15.99:0/2422638480 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f52c40cccf0 con 0x7f52d4100e60 2024-03-30T00:33:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.780+0000 7f52d3400700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f52b80a12b0 0x7f52b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.780+0000 7f52d0c00700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857] conn(0x7f52b80aaa90 0x7f52b80aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.780+0000 7f52d0c00700 1 -- 172.21.15.99:0/2422638480 --> [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52b80ad560 con 0x7f52b80aaa90 2024-03-30T00:33:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.780+0000 7f52d0c00700 1 -- 172.21.15.99:0/2422638480 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52c40cd0a0 con 0x7f52d4100e60 2024-03-30T00:33:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.781+0000 7f52d9200700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857] conn(0x7f52b80aaa90 0x7f52b80aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.781+0000 7f52d3400700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f52b80a12b0 0x7f52b80a3770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f52c0005950 tx=0x7f52c00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.781+0000 7f52d9200700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857] conn(0x7f52b80aaa90 0x7f52b80aceb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.782+0000 7f52d0c00700 1 -- 172.21.15.99:0/2422638480 <== osd.0 v2:172.21.15.99:6802/3911742857 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f52b80ad560 con 0x7f52b80aaa90 2024-03-30T00:33:42.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.799+0000 7f52db200700 1 -- 172.21.15.99:0/2422638480 --> [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52d4061430 con 0x7f52b80aaa90 2024-03-30T00:33:42.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52d0c00700 1 -- 172.21.15.99:0/2422638480 <== osd.0 v2:172.21.15.99:6802/3911742857 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f52d4061430 con 0x7f52b80aaa90 2024-03-30T00:33:42.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52db200700 1 -- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857] conn(0x7f52b80aaa90 msgr2=0x7f52b80aceb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52db200700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857] conn(0x7f52b80aaa90 0x7f52b80aceb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52db200700 1 -- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f52b80a12b0 msgr2=0x7f52b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52db200700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f52b80a12b0 0x7f52b80a3770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f52c0005950 tx=0x7f52c00058e0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52db200700 1 -- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52d4100e60 msgr2=0x7f52d41972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52db200700 1 --2- 172.21.15.99:0/2422638480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52d4100e60 0x7f52d41972f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f52c400d980 tx=0x7f52c400dc90 comp rx=0 tx=0).stop 2024-03-30T00:33:42.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52d9200700 1 -- 172.21.15.99:0/2422638480 reap_dead start 2024-03-30T00:33:42.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52db200700 1 -- 172.21.15.99:0/2422638480 shutdown_connections 2024-03-30T00:33:42.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52db200700 1 -- 172.21.15.99:0/2422638480 >> 172.21.15.99:0/2422638480 conn(0x7f52d4077fa0 msgr2=0x7f52d40ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:42.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52db200700 1 -- 172.21.15.99:0/2422638480 shutdown_connections 2024-03-30T00:33:42.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.800+0000 7f52db200700 1 -- 172.21.15.99:0/2422638480 wait complete. 2024-03-30T00:33:42.185 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:42.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.979+0000 7f83bc200700 1 -- 172.21.15.99:0/772323271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83b4069010 msgr2=0x7f83b4069490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.979+0000 7f83bc200700 1 --2- 172.21.15.99:0/772323271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83b4069010 0x7f83b4069490 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f83a00099c0 tx=0x7f83a0009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.979+0000 7f83bc200700 1 -- 172.21.15.99:0/772323271 shutdown_connections 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.979+0000 7f83bc200700 1 --2- 172.21.15.99:0/772323271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b40699d0 0x7f83b410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.979+0000 7f83bc200700 1 --2- 172.21.15.99:0/772323271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83b4106df0 0x7f83b41071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.979+0000 7f83bc200700 1 --2- 172.21.15.99:0/772323271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83b4069010 0x7f83b4069490 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.979+0000 7f83bc200700 1 -- 172.21.15.99:0/772323271 >> 172.21.15.99:0/772323271 conn(0x7f83b4076c80 msgr2=0x7f83b4077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.979+0000 7f83bc200700 1 -- 172.21.15.99:0/772323271 shutdown_connections 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.979+0000 7f83bc200700 1 -- 172.21.15.99:0/772323271 wait complete. 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.980+0000 7f83bc200700 1 Processor -- start 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.980+0000 7f83bc200700 1 -- start start 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.981+0000 7f83bc200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b4069010 0x7f83b40ffd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.981+0000 7f83bc200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83b40699d0 0x7f83b41002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.981+0000 7f83bc200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83b4106df0 0x7f83b41007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.981+0000 7f83bc200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b41a53d0 con 0x7f83b4106df0 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.981+0000 7f83bc200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b41a5540 con 0x7f83b40699d0 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.981+0000 7f83bc200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b41a5700 con 0x7f83b4069010 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.981+0000 7f83ba200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83b4106df0 0x7f83b41007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.981+0000 7f83ba200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83b4106df0 0x7f83b41007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42750/0 (socket says 172.21.15.99:42750) 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.981+0000 7f83ba200700 1 -- 172.21.15.99:0/3101401162 learned_addr learned my addr 172.21.15.99:0/3101401162 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.981+0000 7f83b8e00700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83b40699d0 0x7f83b41002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.982+0000 7f83b9800700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b4069010 0x7f83b40ffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.982+0000 7f83ba200700 1 -- 172.21.15.99:0/3101401162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b4069010 msgr2=0x7f83b40ffd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.982+0000 7f83ba200700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b4069010 0x7f83b40ffd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.982+0000 7f83ba200700 1 -- 172.21.15.99:0/3101401162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83b40699d0 msgr2=0x7f83b41002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.982+0000 7f83ba200700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83b40699d0 0x7f83b41002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.982+0000 7f83ba200700 1 -- 172.21.15.99:0/3101401162 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a00095f0 con 0x7f83b4106df0 2024-03-30T00:33:42.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.982+0000 7f83ba200700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83b4106df0 0x7f83b41007f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00ef30 tx=0x7f83ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.983+0000 7f83b2000700 1 -- 172.21.15.99:0/3101401162 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac018070 con 0x7f83b4106df0 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.983+0000 7f83b2000700 1 -- 172.21.15.99:0/3101401162 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f83ac00f040 con 0x7f83b4106df0 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.983+0000 7f83b2000700 1 -- 172.21.15.99:0/3101401162 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00cbc0 con 0x7f83b4106df0 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.983+0000 7f83bc200700 1 -- 172.21.15.99:0/3101401162 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b41a5980 con 0x7f83b4106df0 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.983+0000 7f83bc200700 1 -- 172.21.15.99:0/3101401162 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b41a5e00 con 0x7f83b4106df0 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.984+0000 7f83bc200700 1 -- 172.21.15.99:0/3101401162 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83b41024a0 con 0x7f83b4106df0 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.987+0000 7f83b2000700 1 -- 172.21.15.99:0/3101401162 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f83ac00cd40 con 0x7f83b4106df0 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.988+0000 7f83b2000700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f839c0a1230 0x7f839c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.988+0000 7f83b2000700 1 -- 172.21.15.99:0/3101401162 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f83ac0ccd00 con 0x7f83b4106df0 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.989+0000 7f83b9800700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f839c0a1230 0x7f839c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.989+0000 7f83b2000700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097] conn(0x7f839c0aaa10 0x7f839c0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.989+0000 7f83b2000700 1 -- 172.21.15.99:0/3101401162 --> [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f839c0ad4e0 con 0x7f839c0aaa10 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.989+0000 7f83b2000700 1 -- 172.21.15.99:0/3101401162 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83ac0cd1a0 con 0x7f83b4106df0 2024-03-30T00:33:42.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.989+0000 7f83b8e00700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097] conn(0x7f839c0aaa10 0x7f839c0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.989+0000 7f83b9800700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f839c0a1230 0x7f839c0a36f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83a8005950 tx=0x7f83a800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:42.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.990+0000 7f83b8e00700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097] conn(0x7f839c0aaa10 0x7f839c0ace30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:42.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:40.991+0000 7f83b2000700 1 -- 172.21.15.99:0/3101401162 <== osd.11 v2:172.21.15.161:6824/3737118097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f839c0ad4e0 con 0x7f839c0aaa10 2024-03-30T00:33:42.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.009+0000 7f83bc200700 1 -- 172.21.15.99:0/3101401162 --> [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f83b4110e20 con 0x7f839c0aaa10 2024-03-30T00:33:42.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.010+0000 7f83b2000700 1 -- 172.21.15.99:0/3101401162 <== osd.11 v2:172.21.15.161:6824/3737118097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f83b4110e20 con 0x7f839c0aaa10 2024-03-30T00:33:42.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.010+0000 7f83bc200700 1 -- 172.21.15.99:0/3101401162 >> [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097] conn(0x7f839c0aaa10 msgr2=0x7f839c0ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.010+0000 7f83bc200700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097] conn(0x7f839c0aaa10 0x7f839c0ace30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.011+0000 7f83bc200700 1 -- 172.21.15.99:0/3101401162 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f839c0a1230 msgr2=0x7f839c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.011+0000 7f83bc200700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f839c0a1230 0x7f839c0a36f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f83a8005950 tx=0x7f83a800b410 comp rx=0 tx=0).stop 2024-03-30T00:33:42.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.011+0000 7f83bc200700 1 -- 172.21.15.99:0/3101401162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83b4106df0 msgr2=0x7f83b41007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.011+0000 7f83bc200700 1 --2- 172.21.15.99:0/3101401162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83b4106df0 0x7f83b41007f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00ef30 tx=0x7f83ac00c3a0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.011+0000 7f83ba200700 1 -- 172.21.15.99:0/3101401162 reap_dead start 2024-03-30T00:33:42.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.011+0000 7f83bc200700 1 -- 172.21.15.99:0/3101401162 shutdown_connections 2024-03-30T00:33:42.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.011+0000 7f83bc200700 1 -- 172.21.15.99:0/3101401162 >> 172.21.15.99:0/3101401162 conn(0x7f83b4076c80 msgr2=0x7f83b40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:42.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.011+0000 7f83bc200700 1 -- 172.21.15.99:0/3101401162 shutdown_connections 2024-03-30T00:33:42.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:41.011+0000 7f83bc200700 1 -- 172.21.15.99:0/3101401162 wait complete. 2024-03-30T00:33:42.418 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:42 smithi099 ceph-mon[38079]: pgmap v269: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-30T00:33:42.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.114+0000 7f091e600700 1 -- 172.21.15.99:0/2677320164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0918106e10 msgr2=0x7f09181071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.114+0000 7f091e600700 1 --2- 172.21.15.99:0/2677320164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0918106e10 0x7f09181071f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0908009990 tx=0x7f0908009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.114+0000 7f091e600700 1 -- 172.21.15.99:0/2677320164 shutdown_connections 2024-03-30T00:33:42.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.114+0000 7f091e600700 1 --2- 172.21.15.99:0/2677320164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09180690f0 0x7f091810dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.114+0000 7f091e600700 1 --2- 172.21.15.99:0/2677320164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0918106e10 0x7f09181071f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.114+0000 7f091e600700 1 --2- 172.21.15.99:0/2677320164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0918068750 0x7f0918068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.114+0000 7f091e600700 1 -- 172.21.15.99:0/2677320164 >> 172.21.15.99:0/2677320164 conn(0x7f0918075d50 msgr2=0x7f0918076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:42.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.114+0000 7f091e600700 1 -- 172.21.15.99:0/2677320164 shutdown_connections 2024-03-30T00:33:42.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.114+0000 7f091e600700 1 -- 172.21.15.99:0/2677320164 wait complete. 2024-03-30T00:33:42.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.115+0000 7f091e600700 1 Processor -- start 2024-03-30T00:33:42.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.115+0000 7f091e600700 1 -- start start 2024-03-30T00:33:42.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f091e600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0918068750 0x7f0918199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f091e600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09180690f0 0x7f0918199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f091e600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0918106e10 0x7f091819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f091e600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091819e1c0 con 0x7f0918106e10 2024-03-30T00:33:42.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f091e600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091819e330 con 0x7f0918068750 2024-03-30T00:33:42.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f091e600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091819e4f0 con 0x7f09180690f0 2024-03-30T00:33:42.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f0917e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0918106e10 0x7f091819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f0916a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09180690f0 0x7f0918199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f0917e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0918106e10 0x7f091819dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42794/0 (socket says 172.21.15.99:42794) 2024-03-30T00:33:42.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f0917e00700 1 -- 172.21.15.99:0/1871882553 learned_addr learned my addr 172.21.15.99:0/1871882553 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:42.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f0917400700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0918068750 0x7f0918199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f0917e00700 1 -- 172.21.15.99:0/1871882553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09180690f0 msgr2=0x7f0918199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f0917e00700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09180690f0 0x7f0918199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f0917e00700 1 -- 172.21.15.99:0/1871882553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0918068750 msgr2=0x7f0918199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f0917e00700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0918068750 0x7f0918199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.116+0000 7f0917e00700 1 -- 172.21.15.99:0/1871882553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09080095f0 con 0x7f0918106e10 2024-03-30T00:33:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.117+0000 7f0917e00700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0918106e10 0x7f091819dbb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0910007d70 tx=0x7f091000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.117+0000 7f090fe00700 1 -- 172.21.15.99:0/1871882553 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091000ad40 con 0x7f0918106e10 2024-03-30T00:33:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.117+0000 7f091e600700 1 -- 172.21.15.99:0/1871882553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091819e7d0 con 0x7f0918106e10 2024-03-30T00:33:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.117+0000 7f090fe00700 1 -- 172.21.15.99:0/1871882553 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09100128b0 con 0x7f0918106e10 2024-03-30T00:33:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.117+0000 7f090fe00700 1 -- 172.21.15.99:0/1871882553 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09100115c0 con 0x7f0918106e10 2024-03-30T00:33:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.117+0000 7f091e600700 1 -- 172.21.15.99:0/1871882553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09181a6030 con 0x7f0918106e10 2024-03-30T00:33:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.118+0000 7f091e600700 1 -- 172.21.15.99:0/1871882553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09181024c0 con 0x7f0918106e10 2024-03-30T00:33:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.122+0000 7f090fe00700 1 -- 172.21.15.99:0/1871882553 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f0910012a20 con 0x7f0918106e10 2024-03-30T00:33:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.123+0000 7f090fe00700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08fc0a11b0 0x7f08fc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.123+0000 7f090fe00700 1 -- 172.21.15.99:0/1871882553 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f09100ccee0 con 0x7f0918106e10 2024-03-30T00:33:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.123+0000 7f0917400700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08fc0a11b0 0x7f08fc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.123+0000 7f090fe00700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840] conn(0x7f08fc0aa990 0x7f08fc0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.123+0000 7f090fe00700 1 -- 172.21.15.99:0/1871882553 --> [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08fc0ad460 con 0x7f08fc0aa990 2024-03-30T00:33:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.123+0000 7f090fe00700 1 -- 172.21.15.99:0/1871882553 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09100cd380 con 0x7f0918106e10 2024-03-30T00:33:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.124+0000 7f0916a00700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840] conn(0x7f08fc0aa990 0x7f08fc0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.124+0000 7f0917400700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08fc0a11b0 0x7f08fc0a3670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f090800b5c0 tx=0x7f0908005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.124+0000 7f0916a00700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840] conn(0x7f08fc0aa990 0x7f08fc0acdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.126+0000 7f090fe00700 1 -- 172.21.15.99:0/1871882553 <== osd.3 v2:172.21.15.99:6810/1432967840 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f08fc0ad460 con 0x7f08fc0aa990 2024-03-30T00:33:42.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.142+0000 7f091e600700 1 -- 172.21.15.99:0/1871882553 --> [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0918110dd0 con 0x7f08fc0aa990 2024-03-30T00:33:42.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.143+0000 7f090fe00700 1 -- 172.21.15.99:0/1871882553 <== osd.3 v2:172.21.15.99:6810/1432967840 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0918110dd0 con 0x7f08fc0aa990 2024-03-30T00:33:42.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.143+0000 7f091e600700 1 -- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840] conn(0x7f08fc0aa990 msgr2=0x7f08fc0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.143+0000 7f091e600700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840] conn(0x7f08fc0aa990 0x7f08fc0acdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.143+0000 7f091e600700 1 -- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08fc0a11b0 msgr2=0x7f08fc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.143+0000 7f091e600700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08fc0a11b0 0x7f08fc0a3670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f090800b5c0 tx=0x7f0908005fb0 comp rx=0 tx=0).stop 2024-03-30T00:33:42.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.143+0000 7f091e600700 1 -- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0918106e10 msgr2=0x7f091819dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:42.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.143+0000 7f091e600700 1 --2- 172.21.15.99:0/1871882553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0918106e10 0x7f091819dbb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0910007d70 tx=0x7f091000a340 comp rx=0 tx=0).stop 2024-03-30T00:33:42.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.143+0000 7f0917e00700 1 -- 172.21.15.99:0/1871882553 reap_dead start 2024-03-30T00:33:42.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.144+0000 7f091e600700 1 -- 172.21.15.99:0/1871882553 shutdown_connections 2024-03-30T00:33:42.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.144+0000 7f091e600700 1 -- 172.21.15.99:0/1871882553 >> 172.21.15.99:0/1871882553 conn(0x7f0918075d50 msgr2=0x7f09180fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:42.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.144+0000 7f091e600700 1 -- 172.21.15.99:0/1871882553 shutdown_connections 2024-03-30T00:33:42.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:42.144+0000 7f091e600700 1 -- 172.21.15.99:0/1871882553 wait complete. 2024-03-30T00:33:42.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:42 smithi161 ceph-mon[42221]: pgmap v269: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-30T00:33:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:42 smithi142 ceph-mon[41779]: pgmap v269: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-30T00:33:42.709 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:43.521 INFO:teuthology.orchestra.run.smithi099.stdout:34359738433 2024-03-30T00:33:43.521 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-03-30T00:33:43.592 INFO:teuthology.orchestra.run.smithi099.stdout:103079215154 2024-03-30T00:33:43.593 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-03-30T00:33:43.901 INFO:teuthology.orchestra.run.smithi099.stdout:335007449105 2024-03-30T00:33:43.902 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.11 2024-03-30T00:33:44.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.996+0000 7fdf4d200700 1 -- 172.21.15.99:0/4140829458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf48103810 msgr2=0x7fdf48103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.996+0000 7fdf4d200700 1 --2- 172.21.15.99:0/4140829458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf48103810 0x7fdf48103cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000b0a0 tx=0x7fdf4000b3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.996+0000 7fdf4d200700 1 -- 172.21.15.99:0/4140829458 shutdown_connections 2024-03-30T00:33:44.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.996+0000 7fdf4d200700 1 --2- 172.21.15.99:0/4140829458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf48103810 0x7fdf48103cc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.996+0000 7fdf4d200700 1 --2- 172.21.15.99:0/4140829458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf48105cb0 0x7fdf48068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.996+0000 7fdf4d200700 1 --2- 172.21.15.99:0/4140829458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf48068c90 0x7fdf48069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.996+0000 7fdf4d200700 1 -- 172.21.15.99:0/4140829458 >> 172.21.15.99:0/4140829458 conn(0x7fdf48075d50 msgr2=0x7fdf48076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:44.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.996+0000 7fdf4d200700 1 -- 172.21.15.99:0/4140829458 shutdown_connections 2024-03-30T00:33:44.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.996+0000 7fdf4d200700 1 -- 172.21.15.99:0/4140829458 wait complete. 2024-03-30T00:33:44.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.997+0000 7fdf4d200700 1 Processor -- start 2024-03-30T00:33:44.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.998+0000 7fdf4d200700 1 -- start start 2024-03-30T00:33:44.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.998+0000 7fdf4d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf48068c90 0x7fdf4819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:44.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.998+0000 7fdf4d200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf48103810 0x7fdf4819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:44.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.998+0000 7fdf4d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf48105cb0 0x7fdf481a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:44.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.998+0000 7fdf4d200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481a2560 con 0x7fdf48105cb0 2024-03-30T00:33:44.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.998+0000 7fdf4d200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481a26d0 con 0x7fdf48068c90 2024-03-30T00:33:44.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.998+0000 7fdf4d200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481a2890 con 0x7fdf48103810 2024-03-30T00:33:44.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.999+0000 7fdf46800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf48105cb0 0x7fdf481a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:44.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.999+0000 7fdf45e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf48068c90 0x7fdf4819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:44.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.999+0000 7fdf45e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf48068c90 0x7fdf4819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:37570/0 (socket says 172.21.15.99:37570) 2024-03-30T00:33:44.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.999+0000 7fdf45e00700 1 -- 172.21.15.99:0/2696303526 learned_addr learned my addr 172.21.15.99:0/2696303526 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:44.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.999+0000 7fdf45400700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf48103810 0x7fdf4819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:44.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.999+0000 7fdf45e00700 1 -- 172.21.15.99:0/2696303526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf48103810 msgr2=0x7fdf4819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.999+0000 7fdf45e00700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf48103810 0x7fdf4819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.999+0000 7fdf45e00700 1 -- 172.21.15.99:0/2696303526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf48105cb0 msgr2=0x7fdf481a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.999+0000 7fdf45e00700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf48105cb0 0x7fdf481a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.999+0000 7fdf45e00700 1 -- 172.21.15.99:0/2696303526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf40009d00 con 0x7fdf48068c90 2024-03-30T00:33:44.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:43.999+0000 7fdf46800700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf48105cb0 0x7fdf481a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:33:44.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.000+0000 7fdf45e00700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf48068c90 0x7fdf4819d660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800e9a0 tx=0x7fdf3800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:44.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.000+0000 7fdf3ea00700 1 -- 172.21.15.99:0/2696303526 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38009790 con 0x7fdf48068c90 2024-03-30T00:33:44.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.000+0000 7fdf4d200700 1 -- 172.21.15.99:0/2696303526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf481a2b70 con 0x7fdf48068c90 2024-03-30T00:33:44.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.000+0000 7fdf3ea00700 1 -- 172.21.15.99:0/2696303526 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf38010430 con 0x7fdf48068c90 2024-03-30T00:33:44.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.000+0000 7fdf3ea00700 1 -- 172.21.15.99:0/2696303526 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3800cea0 con 0x7fdf48068c90 2024-03-30T00:33:44.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.000+0000 7fdf4d200700 1 -- 172.21.15.99:0/2696303526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf481aa3d0 con 0x7fdf48068c90 2024-03-30T00:33:44.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.001+0000 7fdf4d200700 1 -- 172.21.15.99:0/2696303526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf48117390 con 0x7fdf48068c90 2024-03-30T00:33:44.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.004+0000 7fdf3ea00700 1 -- 172.21.15.99:0/2696303526 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fdf380105a0 con 0x7fdf48068c90 2024-03-30T00:33:44.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.006+0000 7fdf3ea00700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdf2c0a1230 0x7fdf2c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:44.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.006+0000 7fdf3ea00700 1 -- 172.21.15.99:0/2696303526 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdf38014070 con 0x7fdf48068c90 2024-03-30T00:33:44.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.007+0000 7fdf45400700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdf2c0a1230 0x7fdf2c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:44.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.007+0000 7fdf3ea00700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361] conn(0x7fdf2c0aaa10 0x7fdf2c0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:44.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.007+0000 7fdf3ea00700 1 -- 172.21.15.99:0/2696303526 --> [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf2c0ad4e0 con 0x7fdf2c0aaa10 2024-03-30T00:33:44.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.007+0000 7fdf3ea00700 1 -- 172.21.15.99:0/2696303526 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf380ccc20 con 0x7fdf48068c90 2024-03-30T00:33:44.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.008+0000 7fdf45400700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdf2c0a1230 0x7fdf2c0a36f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdf4819ebc0 tx=0x7fdf34009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:44.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.008+0000 7fdf46800700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361] conn(0x7fdf2c0aaa10 0x7fdf2c0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:44.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.008+0000 7fdf46800700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361] conn(0x7fdf2c0aaa10 0x7fdf2c0ace30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:44.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.010+0000 7fdf3ea00700 1 -- 172.21.15.99:0/2696303526 <== osd.1 v2:172.21.15.142:6800/912538361 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fdf2c0ad4e0 con 0x7fdf2c0aaa10 2024-03-30T00:33:44.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.030+0000 7fdf4d200700 1 -- 172.21.15.99:0/2696303526 --> [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf481002f0 con 0x7fdf2c0aaa10 2024-03-30T00:33:44.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.030+0000 7fdf3ea00700 1 -- 172.21.15.99:0/2696303526 <== osd.1 v2:172.21.15.142:6800/912538361 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdf481002f0 con 0x7fdf2c0aaa10 2024-03-30T00:33:44.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.031+0000 7fdf4d200700 1 -- 172.21.15.99:0/2696303526 >> [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361] conn(0x7fdf2c0aaa10 msgr2=0x7fdf2c0ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.031+0000 7fdf4d200700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361] conn(0x7fdf2c0aaa10 0x7fdf2c0ace30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.031+0000 7fdf4d200700 1 -- 172.21.15.99:0/2696303526 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdf2c0a1230 msgr2=0x7fdf2c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.031+0000 7fdf4d200700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdf2c0a1230 0x7fdf2c0a36f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdf4819ebc0 tx=0x7fdf34009260 comp rx=0 tx=0).stop 2024-03-30T00:33:44.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.031+0000 7fdf4d200700 1 -- 172.21.15.99:0/2696303526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf48068c90 msgr2=0x7fdf4819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.031+0000 7fdf4d200700 1 --2- 172.21.15.99:0/2696303526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf48068c90 0x7fdf4819d660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800e9a0 tx=0x7fdf3800ecb0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.031+0000 7fdf46800700 1 -- 172.21.15.99:0/2696303526 reap_dead start 2024-03-30T00:33:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.031+0000 7fdf4d200700 1 -- 172.21.15.99:0/2696303526 shutdown_connections 2024-03-30T00:33:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.031+0000 7fdf4d200700 1 -- 172.21.15.99:0/2696303526 >> 172.21.15.99:0/2696303526 conn(0x7fdf48075d50 msgr2=0x7fdf481007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.031+0000 7fdf4d200700 1 -- 172.21.15.99:0/2696303526 shutdown_connections 2024-03-30T00:33:44.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.031+0000 7fdf4d200700 1 -- 172.21.15.99:0/2696303526 wait complete. 2024-03-30T00:33:44.312 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:44 smithi099 ceph-mon[38079]: pgmap v270: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-30T00:33:44.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:44 smithi161 ceph-mon[42221]: pgmap v270: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-30T00:33:44.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.480+0000 7f35b2800700 1 -- 172.21.15.99:0/2355902019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35ac101850 msgr2=0x7f35ac10dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.480+0000 7f35b2800700 1 --2- 172.21.15.99:0/2355902019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35ac101850 0x7f35ac10dc10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f35a400b0d0 tx=0x7f35a400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.481+0000 7f35b2800700 1 -- 172.21.15.99:0/2355902019 shutdown_connections 2024-03-30T00:33:44.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.481+0000 7f35b2800700 1 --2- 172.21.15.99:0/2355902019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35ac101850 0x7f35ac10dc10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.481+0000 7f35b2800700 1 --2- 172.21.15.99:0/2355902019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35ac106e60 0x7f35ac107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.481+0000 7f35b2800700 1 --2- 172.21.15.99:0/2355902019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ac100e90 0x7f35ac101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.481+0000 7f35b2800700 1 -- 172.21.15.99:0/2355902019 >> 172.21.15.99:0/2355902019 conn(0x7f35ac077ed0 msgr2=0x7f35ac0782e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:44.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.481+0000 7f35b2800700 1 -- 172.21.15.99:0/2355902019 shutdown_connections 2024-03-30T00:33:44.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.481+0000 7f35b2800700 1 -- 172.21.15.99:0/2355902019 wait complete. 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.482+0000 7f35b2800700 1 Processor -- start 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.482+0000 7f35b2800700 1 -- start start 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.482+0000 7f35b2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35ac100e90 0x7f35ac0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.482+0000 7f35b2800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ac101850 0x7f35ac0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.482+0000 7f35b2800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35ac106e60 0x7f35ac073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.482+0000 7f35b2800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac1a5540 con 0x7f35ac106e60 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.482+0000 7f35b2800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac1a56b0 con 0x7f35ac100e90 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.482+0000 7f35b2800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac1a5870 con 0x7f35ac101850 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.482+0000 7f35abe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35ac106e60 0x7f35ac073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35abe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35ac106e60 0x7f35ac073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42832/0 (socket says 172.21.15.99:42832) 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35abe00700 1 -- 172.21.15.99:0/1273011907 learned_addr learned my addr 172.21.15.99:0/1273011907 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35ab400700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35ac100e90 0x7f35ac0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:44.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35abe00700 1 -- 172.21.15.99:0/1273011907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ac101850 msgr2=0x7f35ac0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35abe00700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35ac101850 0x7f35ac0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35abe00700 1 -- 172.21.15.99:0/1273011907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35ac100e90 msgr2=0x7f35ac0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35abe00700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35ac100e90 0x7f35ac0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35abe00700 1 -- 172.21.15.99:0/1273011907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a4009d00 con 0x7f35ac106e60 2024-03-30T00:33:44.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35abe00700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35ac106e60 0x7f35ac073b30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f35a4006070 tx=0x7f35a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:44.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35a3e00700 1 -- 172.21.15.99:0/1273011907 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a400e040 con 0x7f35ac106e60 2024-03-30T00:33:44.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35a3e00700 1 -- 172.21.15.99:0/1273011907 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35a4003e90 con 0x7f35ac106e60 2024-03-30T00:33:44.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35b2800700 1 -- 172.21.15.99:0/1273011907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac1a5af0 con 0x7f35ac106e60 2024-03-30T00:33:44.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.483+0000 7f35a3e00700 1 -- 172.21.15.99:0/1273011907 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4004440 con 0x7f35ac106e60 2024-03-30T00:33:44.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.484+0000 7f35b2800700 1 -- 172.21.15.99:0/1273011907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac1a5fb0 con 0x7f35ac106e60 2024-03-30T00:33:44.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.484+0000 7f35b2800700 1 -- 172.21.15.99:0/1273011907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35ac110e10 con 0x7f35ac106e60 2024-03-30T00:33:44.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.486+0000 7f35a3e00700 1 -- 172.21.15.99:0/1273011907 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f35a4003a00 con 0x7f35ac106e60 2024-03-30T00:33:44.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.488+0000 7f35a3e00700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f35900a1230 0x7f35900a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:44.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.488+0000 7f35a3e00700 1 -- 172.21.15.99:0/1273011907 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f35a40ce5f0 con 0x7f35ac106e60 2024-03-30T00:33:44.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.488+0000 7f35ab400700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f35900a1230 0x7f35900a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:44.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.488+0000 7f35a3e00700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120] conn(0x7f35900aaa10 0x7f35900ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:44.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.488+0000 7f35a3e00700 1 -- 172.21.15.99:0/1273011907 --> [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35900ad4e0 con 0x7f35900aaa10 2024-03-30T00:33:44.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.488+0000 7f35a3e00700 1 -- 172.21.15.99:0/1273011907 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a40ce960 con 0x7f35ac106e60 2024-03-30T00:33:44.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.488+0000 7f35ab400700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f35900a1230 0x7f35900a36f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f359c009be0 tx=0x7f359c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:44.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.488+0000 7f35aaa00700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120] conn(0x7f35900aaa10 0x7f35900ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:44.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.489+0000 7f35aaa00700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120] conn(0x7f35900aaa10 0x7f35900ace30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:44.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.490+0000 7f35a3e00700 1 -- 172.21.15.99:0/1273011907 <== osd.8 v2:172.21.15.161:6816/2321115120 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f35900ad4e0 con 0x7f35900aaa10 2024-03-30T00:33:44.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.506+0000 7f35b2800700 1 -- 172.21.15.99:0/1273011907 --> [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f35ac1024b0 con 0x7f35900aaa10 2024-03-30T00:33:44.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.507+0000 7f35a3e00700 1 -- 172.21.15.99:0/1273011907 <== osd.8 v2:172.21.15.161:6816/2321115120 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f35ac1024b0 con 0x7f35900aaa10 2024-03-30T00:33:44.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.507+0000 7f35b2800700 1 -- 172.21.15.99:0/1273011907 >> [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120] conn(0x7f35900aaa10 msgr2=0x7f35900ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.507+0000 7f35b2800700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120] conn(0x7f35900aaa10 0x7f35900ace30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.508+0000 7f35b2800700 1 -- 172.21.15.99:0/1273011907 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f35900a1230 msgr2=0x7f35900a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.508+0000 7f35b2800700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f35900a1230 0x7f35900a36f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f359c009be0 tx=0x7f359c009260 comp rx=0 tx=0).stop 2024-03-30T00:33:44.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.508+0000 7f35b2800700 1 -- 172.21.15.99:0/1273011907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35ac106e60 msgr2=0x7f35ac073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:44.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.508+0000 7f35b2800700 1 --2- 172.21.15.99:0/1273011907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35ac106e60 0x7f35ac073b30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f35a4006070 tx=0x7f35a400cce0 comp rx=0 tx=0).stop 2024-03-30T00:33:44.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.508+0000 7f35abe00700 1 -- 172.21.15.99:0/1273011907 reap_dead start 2024-03-30T00:33:44.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.508+0000 7f35b2800700 1 -- 172.21.15.99:0/1273011907 shutdown_connections 2024-03-30T00:33:44.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.508+0000 7f35b2800700 1 -- 172.21.15.99:0/1273011907 >> 172.21.15.99:0/1273011907 conn(0x7f35ac077ed0 msgr2=0x7f35ac0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:44.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.508+0000 7f35b2800700 1 -- 172.21.15.99:0/1273011907 shutdown_connections 2024-03-30T00:33:44.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:44.508+0000 7f35b2800700 1 -- 172.21.15.99:0/1273011907 wait complete. 2024-03-30T00:33:44.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:44 smithi142 ceph-mon[41779]: pgmap v270: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-30T00:33:44.579 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:44.603 INFO:teuthology.orchestra.run.smithi099.stdout:55834574909 2024-03-30T00:33:44.603 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-03-30T00:33:45.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.662+0000 7f1113200700 1 -- 172.21.15.99:0/597958579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1114102750 msgr2=0x7f1114106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.662+0000 7f1113200700 1 --2- 172.21.15.99:0/597958579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1114102750 0x7f1114106b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1100009990 tx=0x7f1100009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.662+0000 7f1113200700 1 -- 172.21.15.99:0/597958579 shutdown_connections 2024-03-30T00:33:45.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.662+0000 7f1113200700 1 --2- 172.21.15.99:0/597958579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1114107270 0x7f111410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.662+0000 7f1113200700 1 --2- 172.21.15.99:0/597958579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1114101da0 0x7f1114102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.662+0000 7f1113200700 1 --2- 172.21.15.99:0/597958579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1114102750 0x7f1114106b90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.662+0000 7f1113200700 1 -- 172.21.15.99:0/597958579 >> 172.21.15.99:0/597958579 conn(0x7f11140fb5f0 msgr2=0x7f11140fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:45.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.662+0000 7f1113200700 1 -- 172.21.15.99:0/597958579 shutdown_connections 2024-03-30T00:33:45.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.662+0000 7f1113200700 1 -- 172.21.15.99:0/597958579 wait complete. 2024-03-30T00:33:45.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.663+0000 7f1113200700 1 Processor -- start 2024-03-30T00:33:45.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.664+0000 7f1113200700 1 -- start start 2024-03-30T00:33:45.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.664+0000 7f1113200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1114101da0 0x7f11141970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:45.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.664+0000 7f1113200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1114102750 0x7f1114197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:45.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.664+0000 7f1113200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1114107270 0x7f111419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:45.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.664+0000 7f1113200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111419bf80 con 0x7f1114101da0 2024-03-30T00:33:45.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.664+0000 7f1113200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111419c0f0 con 0x7f1114102750 2024-03-30T00:33:45.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.664+0000 7f1113200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111419c2b0 con 0x7f1114107270 2024-03-30T00:33:45.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.665+0000 7f1111e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1114101da0 0x7f11141970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:45.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.665+0000 7f1111e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1114101da0 0x7f11141970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55662/0 (socket says 172.21.15.99:55662) 2024-03-30T00:33:45.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.665+0000 7f1111e00700 1 -- 172.21.15.99:0/2123805304 learned_addr learned my addr 172.21.15.99:0/2123805304 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:45.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.665+0000 7f1111400700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1114102750 0x7f1114197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:45.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.665+0000 7f1112800700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1114107270 0x7f111419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:45.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.665+0000 7f1111e00700 1 -- 172.21.15.99:0/2123805304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1114107270 msgr2=0x7f111419b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.665+0000 7f1111e00700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1114107270 0x7f111419b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.665+0000 7f1111e00700 1 -- 172.21.15.99:0/2123805304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1114102750 msgr2=0x7f1114197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.665+0000 7f1111e00700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1114102750 0x7f1114197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.665+0000 7f1111e00700 1 -- 172.21.15.99:0/2123805304 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11000095f0 con 0x7f1114101da0 2024-03-30T00:33:45.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.666+0000 7f1111e00700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1114101da0 0x7f11141970d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f110400d980 tx=0x7f110400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:45.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.666+0000 7f110aa00700 1 -- 172.21.15.99:0/2123805304 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104004a30 con 0x7f1114101da0 2024-03-30T00:33:45.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.666+0000 7f1113200700 1 -- 172.21.15.99:0/2123805304 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111419c590 con 0x7f1114101da0 2024-03-30T00:33:45.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.666+0000 7f110aa00700 1 -- 172.21.15.99:0/2123805304 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11040054b0 con 0x7f1114101da0 2024-03-30T00:33:45.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.666+0000 7f110aa00700 1 -- 172.21.15.99:0/2123805304 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104009bf0 con 0x7f1114101da0 2024-03-30T00:33:45.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.666+0000 7f1113200700 1 -- 172.21.15.99:0/2123805304 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11141a3df0 con 0x7f1114101da0 2024-03-30T00:33:45.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.667+0000 7f1113200700 1 -- 172.21.15.99:0/2123805304 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1114110e00 con 0x7f1114101da0 2024-03-30T00:33:45.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.671+0000 7f110aa00700 1 -- 172.21.15.99:0/2123805304 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1104009d50 con 0x7f1114101da0 2024-03-30T00:33:45.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.673+0000 7f110aa00700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f10f80a1230 0x7f10f80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:45.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.673+0000 7f110aa00700 1 -- 172.21.15.99:0/2123805304 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f11040cc6f0 con 0x7f1114101da0 2024-03-30T00:33:45.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.673+0000 7f1111400700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f10f80a1230 0x7f10f80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:45.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.674+0000 7f110aa00700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396] conn(0x7f10f80aaa10 0x7f10f80ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:45.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.674+0000 7f110aa00700 1 -- 172.21.15.99:0/2123805304 --> [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f10f80ad4e0 con 0x7f10f80aaa10 2024-03-30T00:33:45.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.674+0000 7f110aa00700 1 -- 172.21.15.99:0/2123805304 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11040cee00 con 0x7f1114101da0 2024-03-30T00:33:45.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.674+0000 7f1112800700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396] conn(0x7f10f80aaa10 0x7f10f80ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:45.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.675+0000 7f1111400700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f10f80a1230 0x7f10f80a36f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f110000b5c0 tx=0x7f1100005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:45.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.675+0000 7f1112800700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396] conn(0x7f10f80aaa10 0x7f10f80ace30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:45.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.676+0000 7f110aa00700 1 -- 172.21.15.99:0/2123805304 <== osd.4 v2:172.21.15.142:6808/1812504396 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f10f80ad4e0 con 0x7f10f80aaa10 2024-03-30T00:33:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.107+0000 7f7996000700 1 -- 172.21.15.99:0/3113252749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7990106df0 msgr2=0x7f79901071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.107+0000 7f7996000700 1 --2- 172.21.15.99:0/3113252749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7990106df0 0x7f79901071d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f79800099c0 tx=0x7f7980009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.107+0000 7f7996000700 1 -- 172.21.15.99:0/3113252749 shutdown_connections 2024-03-30T00:33:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.107+0000 7f7996000700 1 --2- 172.21.15.99:0/3113252749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79900699f0 0x7f799010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.107+0000 7f7996000700 1 --2- 172.21.15.99:0/3113252749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7990106df0 0x7f79901071d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.107+0000 7f7996000700 1 --2- 172.21.15.99:0/3113252749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7990069030 0x7f79900694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.107+0000 7f7996000700 1 -- 172.21.15.99:0/3113252749 >> 172.21.15.99:0/3113252749 conn(0x7f7990076c80 msgr2=0x7f7990077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.107+0000 7f7996000700 1 -- 172.21.15.99:0/3113252749 shutdown_connections 2024-03-30T00:33:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.107+0000 7f7996000700 1 -- 172.21.15.99:0/3113252749 wait complete. 2024-03-30T00:33:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.108+0000 7f7996000700 1 Processor -- start 2024-03-30T00:33:45.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.108+0000 7f7996000700 1 -- start start 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.108+0000 7f7996000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7990069030 0x7f7990197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.108+0000 7f7996000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79900699f0 0x7f79901975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.108+0000 7f7996000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7990106df0 0x7f799019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.108+0000 7f7996000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799019bf90 con 0x7f7990106df0 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.108+0000 7f7996000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799019c100 con 0x7f79900699f0 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.108+0000 7f7996000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799019c2c0 con 0x7f7990069030 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f798fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7990106df0 0x7f799019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f798fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7990106df0 0x7f799019b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55644/0 (socket says 172.21.15.99:55644) 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f798fe00700 1 -- 172.21.15.99:0/2086295973 learned_addr learned my addr 172.21.15.99:0/2086295973 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f798f400700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7990069030 0x7f7990197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f798ea00700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79900699f0 0x7f79901975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f798fe00700 1 -- 172.21.15.99:0/2086295973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7990069030 msgr2=0x7f7990197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f798fe00700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7990069030 0x7f7990197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f798fe00700 1 -- 172.21.15.99:0/2086295973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79900699f0 msgr2=0x7f79901975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f798fe00700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79900699f0 0x7f79901975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f798fe00700 1 -- 172.21.15.99:0/2086295973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79800095f0 con 0x7f7990106df0 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f798fe00700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7990106df0 0x7f799019b980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7988007ee0 tx=0x7f798800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f7987e00700 1 -- 172.21.15.99:0/2086295973 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798800dd20 con 0x7f7990106df0 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f7987e00700 1 -- 172.21.15.99:0/2086295973 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f798800f040 con 0x7f7990106df0 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.109+0000 7f7996000700 1 -- 172.21.15.99:0/2086295973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799019c5a0 con 0x7f7990106df0 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.110+0000 7f7987e00700 1 -- 172.21.15.99:0/2086295973 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988004750 con 0x7f7990106df0 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.110+0000 7f7996000700 1 -- 172.21.15.99:0/2086295973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79901a3e00 con 0x7f7990106df0 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.111+0000 7f7996000700 1 -- 172.21.15.99:0/2086295973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7990110e80 con 0x7f7990106df0 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.112+0000 7f7987e00700 1 -- 172.21.15.99:0/2086295973 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f7988003a00 con 0x7f7990106df0 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.114+0000 7f7987e00700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f79740a11b0 0x7f79740a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:45.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.114+0000 7f7987e00700 1 -- 172.21.15.99:0/2086295973 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f79880cc230 con 0x7f7990106df0 2024-03-30T00:33:45.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.114+0000 7f798f400700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f79740a11b0 0x7f79740a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:45.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.114+0000 7f7987e00700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760] conn(0x7f79740aa990 0x7f79740acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:45.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.114+0000 7f7987e00700 1 -- 172.21.15.99:0/2086295973 --> [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79740ad460 con 0x7f79740aa990 2024-03-30T00:33:45.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.114+0000 7f7987e00700 1 -- 172.21.15.99:0/2086295973 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79880ce810 con 0x7f7990106df0 2024-03-30T00:33:45.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.115+0000 7f798f400700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f79740a11b0 0x7f79740a3670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7980006010 tx=0x7f7980019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.115+0000 7f798ea00700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760] conn(0x7f79740aa990 0x7f79740acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.115+0000 7f798ea00700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760] conn(0x7f79740aa990 0x7f79740acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.117+0000 7f7987e00700 1 -- 172.21.15.99:0/2086295973 <== osd.10 v2:172.21.15.142:6824/3224161760 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f79740ad460 con 0x7f79740aa990 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.133+0000 7f7996000700 1 -- 172.21.15.99:0/2086295973 --> [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f79901024a0 con 0x7f79740aa990 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f7987e00700 1 -- 172.21.15.99:0/2086295973 <== osd.10 v2:172.21.15.142:6824/3224161760 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f79901024a0 con 0x7f79740aa990 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f7996000700 1 -- 172.21.15.99:0/2086295973 >> [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760] conn(0x7f79740aa990 msgr2=0x7f79740acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f7996000700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760] conn(0x7f79740aa990 0x7f79740acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f7996000700 1 -- 172.21.15.99:0/2086295973 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f79740a11b0 msgr2=0x7f79740a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f7996000700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f79740a11b0 0x7f79740a3670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7980006010 tx=0x7f7980019040 comp rx=0 tx=0).stop 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f7996000700 1 -- 172.21.15.99:0/2086295973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7990106df0 msgr2=0x7f799019b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f7996000700 1 --2- 172.21.15.99:0/2086295973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7990106df0 0x7f799019b980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7988007ee0 tx=0x7f798800d3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f798fe00700 1 -- 172.21.15.99:0/2086295973 reap_dead start 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f7996000700 1 -- 172.21.15.99:0/2086295973 shutdown_connections 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f7996000700 1 -- 172.21.15.99:0/2086295973 >> 172.21.15.99:0/2086295973 conn(0x7f7990076c80 msgr2=0x7f79900feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:45.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f7996000700 1 -- 172.21.15.99:0/2086295973 shutdown_connections 2024-03-30T00:33:45.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.134+0000 7f7996000700 1 -- 172.21.15.99:0/2086295973 wait complete. 2024-03-30T00:33:45.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.697+0000 7f1113200700 1 -- 172.21.15.99:0/2123805304 --> [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1114068ff0 con 0x7f10f80aaa10 2024-03-30T00:33:45.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.697+0000 7f110aa00700 1 -- 172.21.15.99:0/2123805304 <== osd.4 v2:172.21.15.142:6808/1812504396 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1114068ff0 con 0x7f10f80aaa10 2024-03-30T00:33:45.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.698+0000 7f1113200700 1 -- 172.21.15.99:0/2123805304 >> [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396] conn(0x7f10f80aaa10 msgr2=0x7f10f80ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.698+0000 7f1113200700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396] conn(0x7f10f80aaa10 0x7f10f80ace30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.698+0000 7f1113200700 1 -- 172.21.15.99:0/2123805304 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f10f80a1230 msgr2=0x7f10f80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.698+0000 7f1113200700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f10f80a1230 0x7f10f80a36f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f110000b5c0 tx=0x7f1100005fb0 comp rx=0 tx=0).stop 2024-03-30T00:33:45.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.698+0000 7f1113200700 1 -- 172.21.15.99:0/2123805304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1114101da0 msgr2=0x7f11141970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:45.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.698+0000 7f1113200700 1 --2- 172.21.15.99:0/2123805304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1114101da0 0x7f11141970d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f110400d980 tx=0x7f110400dc90 comp rx=0 tx=0).stop 2024-03-30T00:33:45.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.698+0000 7f1112800700 1 -- 172.21.15.99:0/2123805304 reap_dead start 2024-03-30T00:33:45.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.698+0000 7f1113200700 1 -- 172.21.15.99:0/2123805304 shutdown_connections 2024-03-30T00:33:45.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.698+0000 7f1113200700 1 -- 172.21.15.99:0/2123805304 >> 172.21.15.99:0/2123805304 conn(0x7f11140fb5f0 msgr2=0x7f11141053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:45.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.698+0000 7f1113200700 1 -- 172.21.15.99:0/2123805304 shutdown_connections 2024-03-30T00:33:45.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:45.699+0000 7f1113200700 1 -- 172.21.15.99:0/2123805304 wait complete. 2024-03-30T00:33:46.140 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:46.200 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:46 smithi099 ceph-mon[38079]: pgmap v271: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-30T00:33:46.202 INFO:teuthology.orchestra.run.smithi099.stdout:249108103198 2024-03-30T00:33:46.203 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.8 2024-03-30T00:33:46.447 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:46.447 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:46 smithi161 ceph-mon[42221]: pgmap v271: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-30T00:33:46.508 INFO:teuthology.orchestra.run.smithi099.stdout:133143986224 2024-03-30T00:33:46.508 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-03-30T00:33:46.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:46 smithi142 ceph-mon[41779]: pgmap v271: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-30T00:33:46.831 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:46.958 INFO:teuthology.orchestra.run.smithi099.stdout:304942678038 2024-03-30T00:33:46.958 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.10 2024-03-30T00:33:46.997 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:47.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.398+0000 7f292e000700 1 -- 172.21.15.99:0/1458362789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2928101820 msgr2=0x7f292810dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:47.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.398+0000 7f292e000700 1 --2- 172.21.15.99:0/1458362789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2928101820 0x7f292810dc10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f292000b0d0 tx=0x7f292000b3e0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.398+0000 7f292e000700 1 -- 172.21.15.99:0/1458362789 shutdown_connections 2024-03-30T00:33:47.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.398+0000 7f292e000700 1 --2- 172.21.15.99:0/1458362789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2928101820 0x7f292810dc10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.398+0000 7f292e000700 1 --2- 172.21.15.99:0/1458362789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928106ef0 0x7f29281072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.398+0000 7f292e000700 1 --2- 172.21.15.99:0/1458362789 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2928100e60 0x7f29281012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.398+0000 7f292e000700 1 -- 172.21.15.99:0/1458362789 >> 172.21.15.99:0/1458362789 conn(0x7f2928077fa0 msgr2=0x7f29280783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:47.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.398+0000 7f292e000700 1 -- 172.21.15.99:0/1458362789 shutdown_connections 2024-03-30T00:33:47.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.398+0000 7f292e000700 1 -- 172.21.15.99:0/1458362789 wait complete. 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.400+0000 7f292e000700 1 Processor -- start 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.400+0000 7f292e000700 1 -- start start 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.400+0000 7f292e000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2928100e60 0x7f29281972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.400+0000 7f292e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928101820 0x7f2928197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.400+0000 7f292e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2928106ef0 0x7f292819bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.400+0000 7f292e000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292819c1a0 con 0x7f2928106ef0 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.400+0000 7f292e000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292819c310 con 0x7f2928101820 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.400+0000 7f292e000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292819c4d0 con 0x7f2928100e60 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.401+0000 7f2927400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2928100e60 0x7f29281972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.401+0000 7f2927400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2928100e60 0x7f29281972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:56008/0 (socket says 172.21.15.99:56008) 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.401+0000 7f2927400700 1 -- 172.21.15.99:0/1654324788 learned_addr learned my addr 172.21.15.99:0/1654324788 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:47.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.401+0000 7f2927e00700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2928106ef0 0x7f292819bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:47.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.401+0000 7f2926a00700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928101820 0x7f2928197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.401+0000 7f2927400700 1 -- 172.21.15.99:0/1654324788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928101820 msgr2=0x7f2928197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.401+0000 7f2927400700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928101820 0x7f2928197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.401+0000 7f2927400700 1 -- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2928106ef0 msgr2=0x7f292819bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.401+0000 7f2927400700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2928106ef0 0x7f292819bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.401+0000 7f2927400700 1 -- 172.21.15.99:0/1654324788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920009d00 con 0x7f2928100e60 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.401+0000 7f2927e00700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2928106ef0 0x7f292819bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.402+0000 7f2927400700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2928100e60 0x7f29281972f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f291800d980 tx=0x7f291800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.402+0000 7f291fe00700 1 -- 172.21.15.99:0/1654324788 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918004220 con 0x7f2928100e60 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.402+0000 7f291fe00700 1 -- 172.21.15.99:0/1654324788 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2918007640 con 0x7f2928100e60 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.402+0000 7f292e000700 1 -- 172.21.15.99:0/1654324788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292819c7b0 con 0x7f2928100e60 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.402+0000 7f292e000700 1 -- 172.21.15.99:0/1654324788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29280731b0 con 0x7f2928100e60 2024-03-30T00:33:47.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.402+0000 7f291fe00700 1 -- 172.21.15.99:0/1654324788 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29180055e0 con 0x7f2928100e60 2024-03-30T00:33:47.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.403+0000 7f292e000700 1 -- 172.21.15.99:0/1654324788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2908000fc0 con 0x7f2928100e60 2024-03-30T00:33:47.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.405+0000 7f291fe00700 1 -- 172.21.15.99:0/1654324788 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f2918004830 con 0x7f2928100e60 2024-03-30T00:33:47.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.408+0000 7f291fe00700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f290c0a1230 0x7f290c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:47.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.408+0000 7f291fe00700 1 -- 172.21.15.99:0/1654324788 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f29180cc540 con 0x7f2928100e60 2024-03-30T00:33:47.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.408+0000 7f2926a00700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f290c0a1230 0x7f290c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:47.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.408+0000 7f291fe00700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896] conn(0x7f290c0aaa10 0x7f290c0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:47.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.408+0000 7f291fe00700 1 -- 172.21.15.99:0/1654324788 --> [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290c0ad4e0 con 0x7f290c0aaa10 2024-03-30T00:33:47.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.408+0000 7f291fe00700 1 -- 172.21.15.99:0/1654324788 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29180ceb20 con 0x7f2928100e60 2024-03-30T00:33:47.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.409+0000 7f2927e00700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896] conn(0x7f290c0aaa10 0x7f290c0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:47.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.409+0000 7f2926a00700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f290c0a1230 0x7f290c0a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2928198800 tx=0x7f29140094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:47.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.409+0000 7f2927e00700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896] conn(0x7f290c0aaa10 0x7f290c0ace30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:47.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.410+0000 7f291fe00700 1 -- 172.21.15.99:0/1654324788 <== osd.9 v2:172.21.15.99:6826/3086664896 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f290c0ad4e0 con 0x7f290c0aaa10 2024-03-30T00:33:47.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.427+0000 7f292e000700 1 -- 172.21.15.99:0/1654324788 --> [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2908002d40 con 0x7f290c0aaa10 2024-03-30T00:33:47.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.427+0000 7f291fe00700 1 -- 172.21.15.99:0/1654324788 <== osd.9 v2:172.21.15.99:6826/3086664896 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2908002d40 con 0x7f290c0aaa10 2024-03-30T00:33:47.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.428+0000 7f292e000700 1 -- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896] conn(0x7f290c0aaa10 msgr2=0x7f290c0ace30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:47.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.428+0000 7f292e000700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896] conn(0x7f290c0aaa10 0x7f290c0ace30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.428+0000 7f292e000700 1 -- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f290c0a1230 msgr2=0x7f290c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:47.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.428+0000 7f292e000700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f290c0a1230 0x7f290c0a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2928198800 tx=0x7f29140094f0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.428+0000 7f292e000700 1 -- 172.21.15.99:0/1654324788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2928100e60 msgr2=0x7f29281972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:47.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.428+0000 7f292e000700 1 --2- 172.21.15.99:0/1654324788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2928100e60 0x7f29281972f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f291800d980 tx=0x7f291800dc90 comp rx=0 tx=0).stop 2024-03-30T00:33:47.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.428+0000 7f2927e00700 1 -- 172.21.15.99:0/1654324788 reap_dead start 2024-03-30T00:33:47.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.428+0000 7f292e000700 1 -- 172.21.15.99:0/1654324788 shutdown_connections 2024-03-30T00:33:47.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.428+0000 7f292e000700 1 -- 172.21.15.99:0/1654324788 >> 172.21.15.99:0/1654324788 conn(0x7f2928077fa0 msgr2=0x7f29280ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:47.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.429+0000 7f292e000700 1 -- 172.21.15.99:0/1654324788 shutdown_connections 2024-03-30T00:33:47.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.429+0000 7f292e000700 1 -- 172.21.15.99:0/1654324788 wait complete. 2024-03-30T00:33:47.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.814+0000 7f931f400700 1 -- 172.21.15.99:0/2212857443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9318072fe0 msgr2=0x7f93180733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:47.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.814+0000 7f931f400700 1 --2- 172.21.15.99:0/2212857443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9318072fe0 0x7f93180733c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f930c009990 tx=0x7f930c009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.814+0000 7f931f400700 1 -- 172.21.15.99:0/2212857443 shutdown_connections 2024-03-30T00:33:47.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.814+0000 7f931f400700 1 --2- 172.21.15.99:0/2212857443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f931810d810 0x7f931810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.814+0000 7f931f400700 1 --2- 172.21.15.99:0/2212857443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9318072fe0 0x7f93180733c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.814+0000 7f931f400700 1 --2- 172.21.15.99:0/2212857443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9318073900 0x7f931810d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.814+0000 7f931f400700 1 -- 172.21.15.99:0/2212857443 >> 172.21.15.99:0/2212857443 conn(0x7f9318077fa0 msgr2=0x7f93180783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:47.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.814+0000 7f931f400700 1 -- 172.21.15.99:0/2212857443 shutdown_connections 2024-03-30T00:33:47.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.814+0000 7f931f400700 1 -- 172.21.15.99:0/2212857443 wait complete. 2024-03-30T00:33:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.815+0000 7f931f400700 1 Processor -- start 2024-03-30T00:33:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.815+0000 7f931f400700 1 -- start start 2024-03-30T00:33:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.815+0000 7f931f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9318072fe0 0x7f9318199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.815+0000 7f931f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9318073900 0x7f9318199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.815+0000 7f931f400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f931810d810 0x7f931819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f931f400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931819e340 con 0x7f9318073900 2024-03-30T00:33:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f931f400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931819e4b0 con 0x7f9318072fe0 2024-03-30T00:33:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f931f400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931819e670 con 0x7f931810d810 2024-03-30T00:33:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f931ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9318072fe0 0x7f9318199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:47.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f931ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9318072fe0 0x7f9318199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:51012/0 (socket says 172.21.15.99:51012) 2024-03-30T00:33:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f931ca00700 1 -- 172.21.15.99:0/1885563294 learned_addr learned my addr 172.21.15.99:0/1885563294 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f9317e00700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9318073900 0x7f9318199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f931d400700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f931810d810 0x7f931819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f9317e00700 1 -- 172.21.15.99:0/1885563294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f931810d810 msgr2=0x7f931819dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f9317e00700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f931810d810 0x7f931819dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f9317e00700 1 -- 172.21.15.99:0/1885563294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9318072fe0 msgr2=0x7f9318199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f9317e00700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9318072fe0 0x7f9318199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f9317e00700 1 -- 172.21.15.99:0/1885563294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c0095f0 con 0x7f9318073900 2024-03-30T00:33:47.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.816+0000 7f931ca00700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9318072fe0 0x7f9318199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:33:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.817+0000 7f9317e00700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9318073900 0x7f9318199980 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f930400d950 tx=0x7f930400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.817+0000 7f9315600700 1 -- 172.21.15.99:0/1885563294 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9304009930 con 0x7f9318073900 2024-03-30T00:33:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.817+0000 7f9315600700 1 -- 172.21.15.99:0/1885563294 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9304010460 con 0x7f9318073900 2024-03-30T00:33:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.817+0000 7f931f400700 1 -- 172.21.15.99:0/1885563294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931819e950 con 0x7f9318073900 2024-03-30T00:33:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.817+0000 7f9315600700 1 -- 172.21.15.99:0/1885563294 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930400b5d0 con 0x7f9318073900 2024-03-30T00:33:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.817+0000 7f931f400700 1 -- 172.21.15.99:0/1885563294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93181a61b0 con 0x7f9318073900 2024-03-30T00:33:47.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.818+0000 7f931f400700 1 -- 172.21.15.99:0/1885563294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9318108a20 con 0x7f9318073900 2024-03-30T00:33:47.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.820+0000 7f9315600700 1 -- 172.21.15.99:0/1885563294 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f9304009a90 con 0x7f9318073900 2024-03-30T00:33:47.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.821+0000 7f9315600700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93000a1230 0x7f93000a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:47.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.821+0000 7f9315600700 1 -- 172.21.15.99:0/1885563294 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f93040cc4d0 con 0x7f9318073900 2024-03-30T00:33:47.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.822+0000 7f931ca00700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93000a1230 0x7f93000a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:47.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.822+0000 7f931ca00700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93000a1230 0x7f93000a36f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f930c006010 tx=0x7f930c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:47.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:47.826+0000 7f9315600700 1 -- 172.21.15.99:0/1885563294 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9304095bb0 con 0x7f9318073900 2024-03-30T00:33:48.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.055+0000 7f931f400700 1 -- 172.21.15.99:0/1885563294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9318074670 con 0x7f9318073900 2024-03-30T00:33:48.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.056+0000 7f9315600700 1 -- 172.21.15.99:0/1885563294 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93040b7e60 con 0x7f9318073900 2024-03-30T00:33:48.064 INFO:teuthology.orchestra.run.smithi099.stdout:193273528359 2024-03-30T00:33:48.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 -- 172.21.15.99:0/1885563294 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93000a1230 msgr2=0x7f93000a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:48.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93000a1230 0x7f93000a36f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f930c006010 tx=0x7f930c00b540 comp rx=0 tx=0).stop 2024-03-30T00:33:48.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 -- 172.21.15.99:0/1885563294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9318073900 msgr2=0x7f9318199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:48.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9318073900 0x7f9318199980 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f930400d950 tx=0x7f930400dc60 comp rx=0 tx=0).stop 2024-03-30T00:33:48.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 -- 172.21.15.99:0/1885563294 shutdown_connections 2024-03-30T00:33:48.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f931810d810 0x7f931819dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:48.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93000a1230 0x7f93000a36f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:48.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9318072fe0 0x7f9318199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:48.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 --2- 172.21.15.99:0/1885563294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9318073900 0x7f9318199980 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:48.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 -- 172.21.15.99:0/1885563294 >> 172.21.15.99:0/1885563294 conn(0x7f9318077fa0 msgr2=0x7f93180fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:48.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 -- 172.21.15.99:0/1885563294 shutdown_connections 2024-03-30T00:33:48.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:48.058+0000 7f931f400700 1 -- 172.21.15.99:0/1885563294 wait complete. 2024-03-30T00:33:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:48 smithi161 ceph-mon[42221]: pgmap v272: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-30T00:33:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:48 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1885563294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-30T00:33:48.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:48 smithi099 ceph-mon[38079]: pgmap v272: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-30T00:33:48.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:48 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1885563294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-30T00:33:48.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:48 smithi142 ceph-mon[41779]: pgmap v272: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-30T00:33:48.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:48 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1885563294' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-30T00:33:48.571 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528358 got 193273528359 for osd.6 2024-03-30T00:33:48.571 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:48.767 INFO:teuthology.orchestra.run.smithi099.stdout:283467841561 2024-03-30T00:33:48.767 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.9 2024-03-30T00:33:49.380 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:50.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:50 smithi099 ceph-mon[38079]: pgmap v273: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-30T00:33:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.209+0000 7f5de9600700 1 -- 172.21.15.99:0/190000631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5de4100e60 msgr2=0x7f5de41012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.209+0000 7f5de9600700 1 --2- 172.21.15.99:0/190000631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5de4100e60 0x7f5de41012e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00099c0 tx=0x7f5dd0009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.209+0000 7f5de9600700 1 -- 172.21.15.99:0/190000631 shutdown_connections 2024-03-30T00:33:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.209+0000 7f5de9600700 1 --2- 172.21.15.99:0/190000631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5de4101820 0x7f5de410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.209+0000 7f5de9600700 1 --2- 172.21.15.99:0/190000631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5de4106ef0 0x7f5de41072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.209+0000 7f5de9600700 1 --2- 172.21.15.99:0/190000631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5de4100e60 0x7f5de41012e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.209+0000 7f5de9600700 1 -- 172.21.15.99:0/190000631 >> 172.21.15.99:0/190000631 conn(0x7f5de4077fa0 msgr2=0x7f5de40783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:50.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.209+0000 7f5de9600700 1 -- 172.21.15.99:0/190000631 shutdown_connections 2024-03-30T00:33:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.210+0000 7f5de9600700 1 -- 172.21.15.99:0/190000631 wait complete. 2024-03-30T00:33:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.210+0000 7f5de9600700 1 Processor -- start 2024-03-30T00:33:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.211+0000 7f5de9600700 1 -- start start 2024-03-30T00:33:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.211+0000 7f5de9600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5de4100e60 0x7f5de4197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.211+0000 7f5de9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5de4101820 0x7f5de4197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.211+0000 7f5de9600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5de4106ef0 0x7f5de419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.211+0000 7f5de9600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de419c030 con 0x7f5de4100e60 2024-03-30T00:33:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.211+0000 7f5de9600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de419c1a0 con 0x7f5de4101820 2024-03-30T00:33:50.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.211+0000 7f5de9600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de419c360 con 0x7f5de4106ef0 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.211+0000 7f5de2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5de4100e60 0x7f5de4197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.211+0000 7f5de2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5de4100e60 0x7f5de4197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55742/0 (socket says 172.21.15.99:55742) 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.211+0000 7f5de2a00700 1 -- 172.21.15.99:0/3205772138 learned_addr learned my addr 172.21.15.99:0/3205772138 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.211+0000 7f5de2000700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5de4101820 0x7f5de4197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.212+0000 7f5de2a00700 1 -- 172.21.15.99:0/3205772138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5de4106ef0 msgr2=0x7f5de419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.212+0000 7f5de2a00700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5de4106ef0 0x7f5de419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.212+0000 7f5de2a00700 1 -- 172.21.15.99:0/3205772138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5de4101820 msgr2=0x7f5de4197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.212+0000 7f5de2a00700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5de4101820 0x7f5de4197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.212+0000 7f5de2a00700 1 -- 172.21.15.99:0/3205772138 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd00095f0 con 0x7f5de4100e60 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.212+0000 7f5de2a00700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5de4100e60 0x7f5de4197130 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400e9a0 tx=0x7f5dd400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.212+0000 7f5ddb400700 1 -- 172.21.15.99:0/3205772138 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4009790 con 0x7f5de4100e60 2024-03-30T00:33:50.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.212+0000 7f5ddb400700 1 -- 172.21.15.99:0/3205772138 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5dd400cbd0 con 0x7f5de4100e60 2024-03-30T00:33:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.212+0000 7f5de9600700 1 -- 172.21.15.99:0/3205772138 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de419c640 con 0x7f5de4100e60 2024-03-30T00:33:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.212+0000 7f5ddb400700 1 -- 172.21.15.99:0/3205772138 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4018680 con 0x7f5de4100e60 2024-03-30T00:33:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.212+0000 7f5de9600700 1 -- 172.21.15.99:0/3205772138 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de41a3ea0 con 0x7f5de4100e60 2024-03-30T00:33:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.213+0000 7f5de9600700 1 -- 172.21.15.99:0/3205772138 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de41092f0 con 0x7f5de4100e60 2024-03-30T00:33:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.215+0000 7f5ddb400700 1 -- 172.21.15.99:0/3205772138 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f5dd400cd40 con 0x7f5de4100e60 2024-03-30T00:33:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.217+0000 7f5ddb400700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5dc80a1230 0x7f5dc80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.217+0000 7f5ddb400700 1 -- 172.21.15.99:0/3205772138 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5dd4014070 con 0x7f5de4100e60 2024-03-30T00:33:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.217+0000 7f5de2000700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5dc80a1230 0x7f5dc80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.218+0000 7f5de2000700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5dc80a1230 0x7f5dc80a36f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009990 tx=0x7f5dd0009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:50.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.222+0000 7f5ddb400700 1 -- 172.21.15.99:0/3205772138 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5dd40529f0 con 0x7f5de4100e60 2024-03-30T00:33:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.992+0000 7fa4c6600700 1 -- 172.21.15.99:0/660536211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4c0103810 msgr2=0x7fa4c0103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.992+0000 7fa4c6600700 1 --2- 172.21.15.99:0/660536211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4c0103810 0x7fa4c0103cc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009f90 tx=0x7fa4b800b330 comp rx=0 tx=0).stop 2024-03-30T00:33:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.992+0000 7fa4c6600700 1 -- 172.21.15.99:0/660536211 shutdown_connections 2024-03-30T00:33:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.992+0000 7fa4c6600700 1 --2- 172.21.15.99:0/660536211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4c0103810 0x7fa4c0103cc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.992+0000 7fa4c6600700 1 --2- 172.21.15.99:0/660536211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c0105cb0 0x7fa4c0068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.992+0000 7fa4c6600700 1 --2- 172.21.15.99:0/660536211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4c0068c90 0x7fa4c0069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.992+0000 7fa4c6600700 1 -- 172.21.15.99:0/660536211 >> 172.21.15.99:0/660536211 conn(0x7fa4c0075d50 msgr2=0x7fa4c0076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.992+0000 7fa4c6600700 1 -- 172.21.15.99:0/660536211 shutdown_connections 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.992+0000 7fa4c6600700 1 -- 172.21.15.99:0/660536211 wait complete. 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.994+0000 7fa4c6600700 1 Processor -- start 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.994+0000 7fa4c6600700 1 -- start start 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.995+0000 7fa4c6600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4c0068c90 0x7fa4c01a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.995+0000 7fa4c6600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4c0103810 0x7fa4c01a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.995+0000 7fa4c6600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c0105cb0 0x7fa4c019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.995+0000 7fa4c6600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c019dce0 con 0x7fa4c0105cb0 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.995+0000 7fa4c6600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c019de50 con 0x7fa4c0068c90 2024-03-30T00:33:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.995+0000 7fa4c6600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c019e010 con 0x7fa4c0103810 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.995+0000 7fa4bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4c0068c90 0x7fa4c01a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.995+0000 7fa4bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4c0068c90 0x7fa4c01a3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:51020/0 (socket says 172.21.15.99:51020) 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.995+0000 7fa4bf400700 1 -- 172.21.15.99:0/2185063174 learned_addr learned my addr 172.21.15.99:0/2185063174 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.995+0000 7fa4bfe00700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c0105cb0 0x7fa4c019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.995+0000 7fa4bea00700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4c0103810 0x7fa4c01a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.996+0000 7fa4bf400700 1 -- 172.21.15.99:0/2185063174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4c0103810 msgr2=0x7fa4c01a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.996+0000 7fa4bf400700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4c0103810 0x7fa4c01a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.996+0000 7fa4bf400700 1 -- 172.21.15.99:0/2185063174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c0105cb0 msgr2=0x7fa4c019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.996+0000 7fa4bf400700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c0105cb0 0x7fa4c019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.996+0000 7fa4bf400700 1 -- 172.21.15.99:0/2185063174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b00095f0 con 0x7fa4c0068c90 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.996+0000 7fa4bea00700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4c0103810 0x7fa4c01a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.996+0000 7fa4bf400700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4c0068c90 0x7fa4c01a3540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000ec10 tx=0x7fa4b000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.996+0000 7fa4b7e00700 1 -- 172.21.15.99:0/2185063174 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b000cd40 con 0x7fa4c0068c90 2024-03-30T00:33:50.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.996+0000 7fa4c6600700 1 -- 172.21.15.99:0/2185063174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b8009c40 con 0x7fa4c0068c90 2024-03-30T00:33:50.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.997+0000 7fa4b7e00700 1 -- 172.21.15.99:0/2185063174 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4b00044b0 con 0x7fa4c0068c90 2024-03-30T00:33:50.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.997+0000 7fa4b7e00700 1 -- 172.21.15.99:0/2185063174 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0010430 con 0x7fa4c0068c90 2024-03-30T00:33:50.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.997+0000 7fa4c6600700 1 -- 172.21.15.99:0/2185063174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c019e4e0 con 0x7fa4c0068c90 2024-03-30T00:33:50.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:49.999+0000 7fa4c6600700 1 -- 172.21.15.99:0/2185063174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c01002f0 con 0x7fa4c0068c90 2024-03-30T00:33:50.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.000+0000 7fa4b7e00700 1 -- 172.21.15.99:0/2185063174 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa4b0004000 con 0x7fa4c0068c90 2024-03-30T00:33:50.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.001+0000 7fa4b7e00700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa4a40a1230 0x7fa4a40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.415 INFO:teuthology.orchestra.run.smithi099.stdout:34359738435 2024-03-30T00:33:50.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.001+0000 7fa4b7e00700 1 -- 172.21.15.99:0/2185063174 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa4b0014070 con 0x7fa4c0068c90 2024-03-30T00:33:50.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.002+0000 7fa4bea00700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa4a40a1230 0x7fa4a40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:50.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.003+0000 7fa4bea00700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa4a40a1230 0x7fa4a40a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa4c019ed90 tx=0x7fa4ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:50.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.006+0000 7fa4b7e00700 1 -- 172.21.15.99:0/2185063174 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa4b0096490 con 0x7fa4c0068c90 2024-03-30T00:33:50.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.235+0000 7fa4c6600700 1 -- 172.21.15.99:0/2185063174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa4c019ebd0 con 0x7fa4c0068c90 2024-03-30T00:33:50.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.236+0000 7fa4b7e00700 1 -- 172.21.15.99:0/2185063174 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa4c019ebd0 con 0x7fa4c0068c90 2024-03-30T00:33:50.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.238+0000 7fa4c6600700 1 -- 172.21.15.99:0/2185063174 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa4a40a1230 msgr2=0x7fa4a40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.238+0000 7fa4c6600700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa4a40a1230 0x7fa4a40a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa4c019ed90 tx=0x7fa4ac009260 comp rx=0 tx=0).stop 2024-03-30T00:33:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.238+0000 7fa4c6600700 1 -- 172.21.15.99:0/2185063174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4c0068c90 msgr2=0x7fa4c01a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.238+0000 7fa4c6600700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4c0068c90 0x7fa4c01a3540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000ec10 tx=0x7fa4b000ef20 comp rx=0 tx=0).stop 2024-03-30T00:33:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.238+0000 7fa4c6600700 1 -- 172.21.15.99:0/2185063174 shutdown_connections 2024-03-30T00:33:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.238+0000 7fa4c6600700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c0105cb0 0x7fa4c019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.238+0000 7fa4c6600700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa4a40a1230 0x7fa4a40a36f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.238+0000 7fa4c6600700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4c0068c90 0x7fa4c01a3540 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.238+0000 7fa4c6600700 1 --2- 172.21.15.99:0/2185063174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4c0103810 0x7fa4c01a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.238+0000 7fa4c6600700 1 -- 172.21.15.99:0/2185063174 >> 172.21.15.99:0/2185063174 conn(0x7fa4c0075d50 msgr2=0x7fa4c01027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.239+0000 7fa4c6600700 1 -- 172.21.15.99:0/2185063174 shutdown_connections 2024-03-30T00:33:50.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.239+0000 7fa4c6600700 1 -- 172.21.15.99:0/2185063174 wait complete. 2024-03-30T00:33:50.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.459+0000 7f5de9600700 1 -- 172.21.15.99:0/3205772138 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5de4198440 con 0x7f5de4100e60 2024-03-30T00:33:50.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.460+0000 7f5ddb400700 1 -- 172.21.15.99:0/3205772138 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5dd40961d0 con 0x7f5de4100e60 2024-03-30T00:33:50.467 INFO:teuthology.orchestra.run.smithi099.stdout:158913789997 2024-03-30T00:33:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:50 smithi161 ceph-mon[42221]: pgmap v273: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-30T00:33:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.462+0000 7f5de9600700 1 -- 172.21.15.99:0/3205772138 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5dc80a1230 msgr2=0x7f5dc80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.462+0000 7f5de9600700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5dc80a1230 0x7f5dc80a36f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009990 tx=0x7f5dd0009550 comp rx=0 tx=0).stop 2024-03-30T00:33:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.462+0000 7f5de9600700 1 -- 172.21.15.99:0/3205772138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5de4100e60 msgr2=0x7f5de4197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.462+0000 7f5de9600700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5de4100e60 0x7f5de4197130 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400e9a0 tx=0x7f5dd400ecb0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.462+0000 7f5de9600700 1 -- 172.21.15.99:0/3205772138 shutdown_connections 2024-03-30T00:33:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.462+0000 7f5de9600700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5de4106ef0 0x7f5de419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.462+0000 7f5de9600700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5de4100e60 0x7f5de4197130 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.462+0000 7f5de9600700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5dc80a1230 0x7f5dc80a36f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.462+0000 7f5de9600700 1 --2- 172.21.15.99:0/3205772138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5de4101820 0x7f5de4197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.463+0000 7f5de9600700 1 -- 172.21.15.99:0/3205772138 >> 172.21.15.99:0/3205772138 conn(0x7f5de4077fa0 msgr2=0x7f5de40ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.463+0000 7f5de9600700 1 -- 172.21.15.99:0/3205772138 shutdown_connections 2024-03-30T00:33:50.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.463+0000 7f5de9600700 1 -- 172.21.15.99:0/3205772138 wait complete. 2024-03-30T00:33:50.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:50 smithi142 ceph-mon[41779]: pgmap v273: 97 pgs: 97 active+clean; 638 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-30T00:33:50.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.771+0000 7feafbc00700 1 -- 172.21.15.99:0/3380354436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf4106ef0 msgr2=0x7feaf41072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.771+0000 7feafbc00700 1 --2- 172.21.15.99:0/3380354436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf4106ef0 0x7feaf41072d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feae80099c0 tx=0x7feae8009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.771+0000 7feafbc00700 1 -- 172.21.15.99:0/3380354436 shutdown_connections 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.771+0000 7feafbc00700 1 --2- 172.21.15.99:0/3380354436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4101820 0x7feaf410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.771+0000 7feafbc00700 1 --2- 172.21.15.99:0/3380354436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf4106ef0 0x7feaf41072d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.771+0000 7feafbc00700 1 --2- 172.21.15.99:0/3380354436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feaf4100e60 0x7feaf41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.771+0000 7feafbc00700 1 -- 172.21.15.99:0/3380354436 >> 172.21.15.99:0/3380354436 conn(0x7feaf4077fa0 msgr2=0x7feaf40783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.771+0000 7feafbc00700 1 -- 172.21.15.99:0/3380354436 shutdown_connections 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.771+0000 7feafbc00700 1 -- 172.21.15.99:0/3380354436 wait complete. 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.772+0000 7feafbc00700 1 Processor -- start 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.772+0000 7feafbc00700 1 -- start start 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feafbc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4100e60 0x7feaf4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feafbc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feaf4101820 0x7feaf4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feafbc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf4106ef0 0x7feaf419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feafbc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf419e210 con 0x7feaf4106ef0 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feafbc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf419e380 con 0x7feaf4100e60 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feafbc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf419e540 con 0x7feaf4101820 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf9c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf4106ef0 0x7feaf419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf9c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf4106ef0 0x7feaf419dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55762/0 (socket says 172.21.15.99:55762) 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf9c00700 1 -- 172.21.15.99:0/935296321 learned_addr learned my addr 172.21.15.99:0/935296321 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf9200700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4100e60 0x7feaf4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf3e00700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feaf4101820 0x7feaf4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf9c00700 1 -- 172.21.15.99:0/935296321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feaf4101820 msgr2=0x7feaf4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf9c00700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feaf4101820 0x7feaf4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf9c00700 1 -- 172.21.15.99:0/935296321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4100e60 msgr2=0x7feaf4199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:50.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf9c00700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4100e60 0x7feaf4199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf9c00700 1 -- 172.21.15.99:0/935296321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae80095f0 con 0x7feaf4106ef0 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf3e00700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feaf4101820 0x7feaf4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.773+0000 7feaf9200700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4100e60 0x7feaf4199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.774+0000 7feaf9c00700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf4106ef0 0x7feaf419dc00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feaec00ab50 tx=0x7feaec00af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.774+0000 7feaf1600700 1 -- 172.21.15.99:0/935296321 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec00f070 con 0x7feaf4106ef0 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.774+0000 7feaf1600700 1 -- 172.21.15.99:0/935296321 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feaec007a80 con 0x7feaf4106ef0 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.774+0000 7feafbc00700 1 -- 172.21.15.99:0/935296321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf419e820 con 0x7feaf4106ef0 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.774+0000 7feaf1600700 1 -- 172.21.15.99:0/935296321 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec005280 con 0x7feaf4106ef0 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.774+0000 7feafbc00700 1 -- 172.21.15.99:0/935296321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf41a6080 con 0x7feaf4106ef0 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.775+0000 7feafbc00700 1 -- 172.21.15.99:0/935296321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf41024f0 con 0x7feaf4106ef0 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.777+0000 7feaf1600700 1 -- 172.21.15.99:0/935296321 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7feaec004ad0 con 0x7feaf4106ef0 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.778+0000 7feaf1600700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feadc0a11b0 0x7feadc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.778+0000 7feaf1600700 1 -- 172.21.15.99:0/935296321 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7feaec0cbf00 con 0x7feaf4106ef0 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.778+0000 7feaf9200700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feadc0a11b0 0x7feadc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:50.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.779+0000 7feaf9200700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feadc0a11b0 0x7feadc0a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feae80036c0 tx=0x7feae8005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:50.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.782+0000 7feaf1600700 1 -- 172.21.15.99:0/935296321 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7feaec095670 con 0x7feaf4106ef0 2024-03-30T00:33:50.976 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:51.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.998+0000 7feafbc00700 1 -- 172.21.15.99:0/935296321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7feaf419a6c0 con 0x7feaf4106ef0 2024-03-30T00:33:51.007 INFO:teuthology.orchestra.run.smithi099.stdout:214748364835 2024-03-30T00:33:51.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:50.998+0000 7feaf1600700 1 -- 172.21.15.99:0/935296321 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feaec098c90 con 0x7feaf4106ef0 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 -- 172.21.15.99:0/935296321 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feadc0a11b0 msgr2=0x7feadc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feadc0a11b0 0x7feadc0a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7feae80036c0 tx=0x7feae8005e80 comp rx=0 tx=0).stop 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 -- 172.21.15.99:0/935296321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf4106ef0 msgr2=0x7feaf419dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf4106ef0 0x7feaf419dc00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feaec00ab50 tx=0x7feaec00af10 comp rx=0 tx=0).stop 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 -- 172.21.15.99:0/935296321 shutdown_connections 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaf4106ef0 0x7feaf419dc00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feadc0a11b0 0x7feadc0a3670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feaf4100e60 0x7feaf4199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 --2- 172.21.15.99:0/935296321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feaf4101820 0x7feaf4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 -- 172.21.15.99:0/935296321 >> 172.21.15.99:0/935296321 conn(0x7feaf4077fa0 msgr2=0x7feaf40ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 -- 172.21.15.99:0/935296321 shutdown_connections 2024-03-30T00:33:51.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.001+0000 7feafbc00700 1 -- 172.21.15.99:0/935296321 wait complete. 2024-03-30T00:33:51.411 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:51 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2185063174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-30T00:33:51.411 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:51 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3205772138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-30T00:33:51.412 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:51 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/935296321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-30T00:33:51.439 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.194+0000 7f5927200700 1 -- 172.21.15.99:0/402928535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5920100e60 msgr2=0x7f59201012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:51.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.194+0000 7f5927200700 1 --2- 172.21.15.99:0/402928535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5920100e60 0x7f59201012e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f590c0099c0 tx=0x7f590c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.195+0000 7f5927200700 1 -- 172.21.15.99:0/402928535 shutdown_connections 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.195+0000 7f5927200700 1 --2- 172.21.15.99:0/402928535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5920101820 0x7f592010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.195+0000 7f5927200700 1 --2- 172.21.15.99:0/402928535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5920106ef0 0x7f59201072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.195+0000 7f5927200700 1 --2- 172.21.15.99:0/402928535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5920100e60 0x7f59201012e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.195+0000 7f5927200700 1 -- 172.21.15.99:0/402928535 >> 172.21.15.99:0/402928535 conn(0x7f5920077fa0 msgr2=0x7f59200783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.195+0000 7f5927200700 1 -- 172.21.15.99:0/402928535 shutdown_connections 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.195+0000 7f5927200700 1 -- 172.21.15.99:0/402928535 wait complete. 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.196+0000 7f5927200700 1 Processor -- start 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.196+0000 7f5927200700 1 -- start start 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.196+0000 7f5927200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5920100e60 0x7f59200730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.196+0000 7f5927200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5920101820 0x7f59200735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.196+0000 7f5927200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5920106ef0 0x7f5920073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:51.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.196+0000 7f5927200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59201a5570 con 0x7f5920100e60 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.196+0000 7f5927200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59201a56e0 con 0x7f5920106ef0 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.196+0000 7f5927200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59201a58a0 con 0x7f5920101820 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.196+0000 7f591fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5920100e60 0x7f59200730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.196+0000 7f591fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5920100e60 0x7f59200730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55788/0 (socket says 172.21.15.99:55788) 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591fe00700 1 -- 172.21.15.99:0/4244750091 learned_addr learned my addr 172.21.15.99:0/4244750091 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f5925200700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5920106ef0 0x7f5920073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591f400700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5920101820 0x7f59200735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591fe00700 1 -- 172.21.15.99:0/4244750091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5920101820 msgr2=0x7f59200735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591fe00700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5920101820 0x7f59200735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591fe00700 1 -- 172.21.15.99:0/4244750091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5920106ef0 msgr2=0x7f5920073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591fe00700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5920106ef0 0x7f5920073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591fe00700 1 -- 172.21.15.99:0/4244750091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c0095f0 con 0x7f5920100e60 2024-03-30T00:33:51.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591f400700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5920101820 0x7f59200735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591fe00700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5920100e60 0x7f59200730b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f591000e9a0 tx=0x7f591000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591cc00700 1 -- 172.21.15.99:0/4244750091 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910009790 con 0x7f5920100e60 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f5927200700 1 -- 172.21.15.99:0/4244750091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59201a5b80 con 0x7f5920100e60 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591cc00700 1 -- 172.21.15.99:0/4244750091 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f591000cbd0 con 0x7f5920100e60 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.197+0000 7f591cc00700 1 -- 172.21.15.99:0/4244750091 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910018680 con 0x7f5920100e60 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.198+0000 7f5927200700 1 -- 172.21.15.99:0/4244750091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59201a6080 con 0x7f5920100e60 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.199+0000 7f5927200700 1 -- 172.21.15.99:0/4244750091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59201092f0 con 0x7f5920100e60 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.200+0000 7f591cc00700 1 -- 172.21.15.99:0/4244750091 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f591000cd40 con 0x7f5920100e60 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.202+0000 7f591cc00700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f59040a11b0 0x7f59040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.202+0000 7f591cc00700 1 -- 172.21.15.99:0/4244750091 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5910014070 con 0x7f5920100e60 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.202+0000 7f591f400700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f59040a11b0 0x7f59040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.202+0000 7f591f400700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f59040a11b0 0x7f59040a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f590c006010 tx=0x7f590c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.205+0000 7f591cc00700 1 -- 172.21.15.99:0/4244750091 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f59100529f0 con 0x7f5920100e60 2024-03-30T00:33:51.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.450+0000 7f5927200700 1 -- 172.21.15.99:0/4244750091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5920074590 con 0x7f5920100e60 2024-03-30T00:33:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.451+0000 7f591cc00700 1 -- 172.21.15.99:0/4244750091 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f59100962a0 con 0x7f5920100e60 2024-03-30T00:33:51.466 INFO:teuthology.orchestra.run.smithi099.stdout:77309411386 2024-03-30T00:33:51.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.454+0000 7f5927200700 1 -- 172.21.15.99:0/4244750091 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f59040a11b0 msgr2=0x7f59040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.454+0000 7f5927200700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f59040a11b0 0x7f59040a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f590c006010 tx=0x7f590c000bc0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.454+0000 7f5927200700 1 -- 172.21.15.99:0/4244750091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5920100e60 msgr2=0x7f59200730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.454+0000 7f5927200700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5920100e60 0x7f59200730b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f591000e9a0 tx=0x7f591000ecb0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.454+0000 7f5927200700 1 -- 172.21.15.99:0/4244750091 shutdown_connections 2024-03-30T00:33:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.454+0000 7f5927200700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f59040a11b0 0x7f59040a3670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.454+0000 7f5927200700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5920106ef0 0x7f5920073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.455+0000 7f5927200700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5920100e60 0x7f59200730b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.455+0000 7f5927200700 1 --2- 172.21.15.99:0/4244750091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5920101820 0x7f59200735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.455+0000 7f5927200700 1 -- 172.21.15.99:0/4244750091 >> 172.21.15.99:0/4244750091 conn(0x7f5920077fa0 msgr2=0x7f59200ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.455+0000 7f5927200700 1 -- 172.21.15.99:0/4244750091 shutdown_connections 2024-03-30T00:33:51.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.455+0000 7f5927200700 1 -- 172.21.15.99:0/4244750091 wait complete. 2024-03-30T00:33:51.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:51 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2185063174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-30T00:33:51.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:51 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3205772138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-30T00:33:51.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:51 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/935296321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-30T00:33:51.482 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789995 got 158913789997 for osd.5 2024-03-30T00:33:51.482 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:51.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:51 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2185063174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-30T00:33:51.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:51 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3205772138' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-30T00:33:51.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:51 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/935296321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-30T00:33:52.083 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364834 got 214748364835 for osd.7 2024-03-30T00:33:52.084 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:52.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.096+0000 7fc3fc400700 1 -- 172.21.15.99:0/3951247564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4100e60 msgr2=0x7fc3f41012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:52.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.096+0000 7fc3fc400700 1 --2- 172.21.15.99:0/3951247564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4100e60 0x7fc3f41012e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009920 tx=0x7fc3e0009c30 comp rx=0 tx=0).stop 2024-03-30T00:33:52.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.096+0000 7fc3fc400700 1 -- 172.21.15.99:0/3951247564 shutdown_connections 2024-03-30T00:33:52.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.096+0000 7fc3fc400700 1 --2- 172.21.15.99:0/3951247564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f4101820 0x7fc3f410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.096+0000 7fc3fc400700 1 --2- 172.21.15.99:0/3951247564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f4106ef0 0x7fc3f41072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.096+0000 7fc3fc400700 1 --2- 172.21.15.99:0/3951247564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4100e60 0x7fc3f41012e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.096+0000 7fc3fc400700 1 -- 172.21.15.99:0/3951247564 >> 172.21.15.99:0/3951247564 conn(0x7fc3f4077fa0 msgr2=0x7fc3f40783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:52.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.096+0000 7fc3fc400700 1 -- 172.21.15.99:0/3951247564 shutdown_connections 2024-03-30T00:33:52.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.096+0000 7fc3fc400700 1 -- 172.21.15.99:0/3951247564 wait complete. 2024-03-30T00:33:52.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.097+0000 7fc3fc400700 1 Processor -- start 2024-03-30T00:33:52.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.098+0000 7fc3fc400700 1 -- start start 2024-03-30T00:33:52.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.098+0000 7fc3fc400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f4100e60 0x7fc3f4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:52.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.098+0000 7fc3fc400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4101820 0x7fc3f4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:52.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.098+0000 7fc3fc400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f4106ef0 0x7fc3f419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:52.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.098+0000 7fc3fc400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f419c150 con 0x7fc3f4106ef0 2024-03-30T00:33:52.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.098+0000 7fc3fc400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f419c2c0 con 0x7fc3f4101820 2024-03-30T00:33:52.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.098+0000 7fc3fc400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f419c480 con 0x7fc3f4100e60 2024-03-30T00:33:52.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.098+0000 7fc3fa400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f4106ef0 0x7fc3f419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:52.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.098+0000 7fc3fa400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f4106ef0 0x7fc3f419bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55820/0 (socket says 172.21.15.99:55820) 2024-03-30T00:33:52.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.098+0000 7fc3f9a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f4100e60 0x7fc3f4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:52.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.098+0000 7fc3fa400700 1 -- 172.21.15.99:0/1029863374 learned_addr learned my addr 172.21.15.99:0/1029863374 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:52.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.099+0000 7fc3f9000700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4101820 0x7fc3f4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:52.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.099+0000 7fc3fa400700 1 -- 172.21.15.99:0/1029863374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f4100e60 msgr2=0x7fc3f4197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:52.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.099+0000 7fc3fa400700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f4100e60 0x7fc3f4197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.099+0000 7fc3fa400700 1 -- 172.21.15.99:0/1029863374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4101820 msgr2=0x7fc3f4197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.099+0000 7fc3fa400700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4101820 0x7fc3f4197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.099+0000 7fc3fa400700 1 -- 172.21.15.99:0/1029863374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e0009520 con 0x7fc3f4106ef0 2024-03-30T00:33:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.099+0000 7fc3f9000700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4101820 0x7fc3f4197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:33:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.099+0000 7fc3fa400700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f4106ef0 0x7fc3f419bb40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00efc0 tx=0x7fc3ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.099+0000 7fc3f2000700 1 -- 172.21.15.99:0/1029863374 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec018070 con 0x7fc3f4106ef0 2024-03-30T00:33:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.099+0000 7fc3f2000700 1 -- 172.21.15.99:0/1029863374 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3ec00f040 con 0x7fc3f4106ef0 2024-03-30T00:33:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.099+0000 7fc3fc400700 1 -- 172.21.15.99:0/1029863374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f419c700 con 0x7fc3f4106ef0 2024-03-30T00:33:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.100+0000 7fc3f2000700 1 -- 172.21.15.99:0/1029863374 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00cb90 con 0x7fc3f4106ef0 2024-03-30T00:33:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.100+0000 7fc3fc400700 1 -- 172.21.15.99:0/1029863374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f41a3fc0 con 0x7fc3f4106ef0 2024-03-30T00:33:52.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.101+0000 7fc3fc400700 1 -- 172.21.15.99:0/1029863374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f41092f0 con 0x7fc3f4106ef0 2024-03-30T00:33:52.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.105+0000 7fc3f2000700 1 -- 172.21.15.99:0/1029863374 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc3ec00ccf0 con 0x7fc3f4106ef0 2024-03-30T00:33:52.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.107+0000 7fc3f2000700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3dc0a11b0 0x7fc3dc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:52.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.107+0000 7fc3f2000700 1 -- 172.21.15.99:0/1029863374 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3ec0cbf90 con 0x7fc3f4106ef0 2024-03-30T00:33:52.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.107+0000 7fc3f9a00700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3dc0a11b0 0x7fc3dc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:52.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.108+0000 7fc3f9a00700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3dc0a11b0 0x7fc3dc0a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8005fd0 tx=0x7fc3e8009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:52.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.114+0000 7fc3f2000700 1 -- 172.21.15.99:0/1029863374 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc3ec095700 con 0x7fc3f4106ef0 2024-03-30T00:33:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.888+0000 7f71ee600700 1 -- 172.21.15.99:0/2401178918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e8105ef0 msgr2=0x7f71e810dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.888+0000 7f71ee600700 1 --2- 172.21.15.99:0/2401178918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e8105ef0 0x7f71e810dac0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009f90 tx=0x7f71e000b330 comp rx=0 tx=0).stop 2024-03-30T00:33:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.888+0000 7f71ee600700 1 -- 172.21.15.99:0/2401178918 shutdown_connections 2024-03-30T00:33:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.888+0000 7f71ee600700 1 --2- 172.21.15.99:0/2401178918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e8105ef0 0x7f71e810dac0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.888+0000 7f71ee600700 1 --2- 172.21.15.99:0/2401178918 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8100e10 0x7f71e81011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.888+0000 7f71ee600700 1 --2- 172.21.15.99:0/2401178918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71e81017c0 0x7f71e8105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.888+0000 7f71ee600700 1 -- 172.21.15.99:0/2401178918 >> 172.21.15.99:0/2401178918 conn(0x7f71e80fc6e0 msgr2=0x7f71e80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.888+0000 7f71ee600700 1 -- 172.21.15.99:0/2401178918 shutdown_connections 2024-03-30T00:33:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.888+0000 7f71ee600700 1 -- 172.21.15.99:0/2401178918 wait complete. 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.889+0000 7f71ee600700 1 Processor -- start 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.889+0000 7f71ee600700 1 -- start start 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71ee600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71e8100e10 0x7f71e80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71ee600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81017c0 0x7f71e80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71ee600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8105ef0 0x7f71e8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71ee600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81a5470 con 0x7f71e81017c0 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71ee600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81a55e0 con 0x7f71e8100e10 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71ee600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81a57a0 con 0x7f71e8105ef0 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81017c0 0x7f71e80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81017c0 0x7f71e80735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55806/0 (socket says 172.21.15.99:55806) 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71e7e00700 1 -- 172.21.15.99:0/965679022 learned_addr learned my addr 172.21.15.99:0/965679022 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71edc00700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8105ef0 0x7f71e8073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71ed200700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71e8100e10 0x7f71e80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:52.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71e7e00700 1 -- 172.21.15.99:0/965679022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8105ef0 msgr2=0x7f71e8073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71e7e00700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8105ef0 0x7f71e8073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71e7e00700 1 -- 172.21.15.99:0/965679022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71e8100e10 msgr2=0x7f71e80730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71e7e00700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71e8100e10 0x7f71e80730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.890+0000 7f71e7e00700 1 -- 172.21.15.99:0/965679022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d4009520 con 0x7f71e81017c0 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.891+0000 7f71ed200700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71e8100e10 0x7f71e80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.891+0000 7f71e7e00700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81017c0 0x7f71e80735f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f71d400ea30 tx=0x7f71d400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.891+0000 7f71e5600700 1 -- 172.21.15.99:0/965679022 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4009820 con 0x7f71e81017c0 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.891+0000 7f71ee600700 1 -- 172.21.15.99:0/965679022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e0009c40 con 0x7f71e81017c0 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.891+0000 7f71e5600700 1 -- 172.21.15.99:0/965679022 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71d40044b0 con 0x7f71e81017c0 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.891+0000 7f71e5600700 1 -- 172.21.15.99:0/965679022 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4003ce0 con 0x7f71e81017c0 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.891+0000 7f71ee600700 1 -- 172.21.15.99:0/965679022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e81a5e40 con 0x7f71e81017c0 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.892+0000 7f71ee600700 1 -- 172.21.15.99:0/965679022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e8109210 con 0x7f71e81017c0 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.894+0000 7f71e5600700 1 -- 172.21.15.99:0/965679022 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f71d4003ea0 con 0x7f71e81017c0 2024-03-30T00:33:52.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.895+0000 7f71e5600700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71d00a1230 0x7f71d00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:52.340 INFO:teuthology.orchestra.run.smithi099.stdout:335007449107 2024-03-30T00:33:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.895+0000 7f71e5600700 1 -- 172.21.15.99:0/965679022 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f71d4014070 con 0x7f71e81017c0 2024-03-30T00:33:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.896+0000 7f71ed200700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71d00a1230 0x7f71d00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.896+0000 7f71ed200700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71d00a1230 0x7f71d00a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f71dc005fd0 tx=0x7f71dc009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:51.899+0000 7f71e5600700 1 -- 172.21.15.99:0/965679022 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f71d40962b0 con 0x7f71e81017c0 2024-03-30T00:33:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.144+0000 7f71ee600700 1 -- 172.21.15.99:0/965679022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f71e81022f0 con 0x7f71e81017c0 2024-03-30T00:33:52.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.144+0000 7f71e5600700 1 -- 172.21.15.99:0/965679022 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f71d40998d0 con 0x7f71e81017c0 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 -- 172.21.15.99:0/965679022 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71d00a1230 msgr2=0x7f71d00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71d00a1230 0x7f71d00a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f71dc005fd0 tx=0x7f71dc009310 comp rx=0 tx=0).stop 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 -- 172.21.15.99:0/965679022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81017c0 msgr2=0x7f71e80735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81017c0 0x7f71e80735f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f71d400ea30 tx=0x7f71d400ed40 comp rx=0 tx=0).stop 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 -- 172.21.15.99:0/965679022 shutdown_connections 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e8105ef0 0x7f71e8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71d00a1230 0x7f71d00a36f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71e8100e10 0x7f71e80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 --2- 172.21.15.99:0/965679022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81017c0 0x7f71e80735f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 -- 172.21.15.99:0/965679022 >> 172.21.15.99:0/965679022 conn(0x7f71e80fc6e0 msgr2=0x7f71e80fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 -- 172.21.15.99:0/965679022 shutdown_connections 2024-03-30T00:33:52.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.147+0000 7f71ee600700 1 -- 172.21.15.99:0/965679022 wait complete. 2024-03-30T00:33:52.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.346+0000 7fc3fc400700 1 -- 172.21.15.99:0/1029863374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc3f404eab0 con 0x7fc3f4106ef0 2024-03-30T00:33:52.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.346+0000 7fc3f2000700 1 -- 172.21.15.99:0/1029863374 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc3ec098d20 con 0x7fc3f4106ef0 2024-03-30T00:33:52.353 INFO:teuthology.orchestra.run.smithi099.stdout:103079215156 2024-03-30T00:33:52.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.348+0000 7fc3fc400700 1 -- 172.21.15.99:0/1029863374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3dc0a11b0 msgr2=0x7fc3dc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:52.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.348+0000 7fc3fc400700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3dc0a11b0 0x7fc3dc0a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8005fd0 tx=0x7fc3e8009310 comp rx=0 tx=0).stop 2024-03-30T00:33:52.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.348+0000 7fc3fc400700 1 -- 172.21.15.99:0/1029863374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f4106ef0 msgr2=0x7fc3f419bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.348+0000 7fc3fc400700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f4106ef0 0x7fc3f419bb40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00efc0 tx=0x7fc3ec00c3a0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.348+0000 7fc3fc400700 1 -- 172.21.15.99:0/1029863374 shutdown_connections 2024-03-30T00:33:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.348+0000 7fc3fc400700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3f4106ef0 0x7fc3f419bb40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.348+0000 7fc3fc400700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3f4100e60 0x7fc3f4197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.348+0000 7fc3fc400700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3dc0a11b0 0x7fc3dc0a3670 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.349+0000 7fc3fc400700 1 --2- 172.21.15.99:0/1029863374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f4101820 0x7fc3f4197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.349+0000 7fc3fc400700 1 -- 172.21.15.99:0/1029863374 >> 172.21.15.99:0/1029863374 conn(0x7fc3f4077fa0 msgr2=0x7fc3f40ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.349+0000 7fc3fc400700 1 -- 172.21.15.99:0/1029863374 shutdown_connections 2024-03-30T00:33:52.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:52.349+0000 7fc3fc400700 1 -- 172.21.15.99:0/1029863374 wait complete. 2024-03-30T00:33:52.378 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:52 smithi099 ceph-mon[38079]: pgmap v274: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-03-30T00:33:52.378 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:52 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4244750091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-30T00:33:52.378 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:52 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/965679022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-30T00:33:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:52 smithi161 ceph-mon[42221]: pgmap v274: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-03-30T00:33:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:52 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4244750091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-30T00:33:52.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:52 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/965679022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-30T00:33:52.514 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738433 got 34359738435 for osd.0 2024-03-30T00:33:52.514 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:52.561 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:52.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:52 smithi142 ceph-mon[41779]: pgmap v274: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 7 op/s 2024-03-30T00:33:52.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:52 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4244750091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-30T00:33:52.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:52 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/965679022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-30T00:33:52.903 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411384 got 77309411386 for osd.2 2024-03-30T00:33:52.903 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:53.389 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:53 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1029863374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-30T00:33:53.429 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:53.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:53 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1029863374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-30T00:33:53.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:53 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1029863374' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-30T00:33:53.588 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215154 got 103079215156 for osd.3 2024-03-30T00:33:53.589 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:53.606 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:53.759 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449105 got 335007449107 for osd.11 2024-03-30T00:33:53.759 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:54.257 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:54 smithi099 ceph-mon[38079]: pgmap v275: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-30T00:33:54.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.343+0000 7f32cfc00700 1 -- 172.21.15.99:0/2652554497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8072fe0 msgr2=0x7f32c80733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:54.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.343+0000 7f32cfc00700 1 --2- 172.21.15.99:0/2652554497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8072fe0 0x7f32c80733c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009990 tx=0x7f32bc009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.343+0000 7f32cfc00700 1 -- 172.21.15.99:0/2652554497 shutdown_connections 2024-03-30T00:33:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.343+0000 7f32cfc00700 1 --2- 172.21.15.99:0/2652554497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32c810d810 0x7f32c810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.343+0000 7f32cfc00700 1 --2- 172.21.15.99:0/2652554497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8072fe0 0x7f32c80733c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.343+0000 7f32cfc00700 1 --2- 172.21.15.99:0/2652554497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c8073900 0x7f32c810d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.343+0000 7f32cfc00700 1 -- 172.21.15.99:0/2652554497 >> 172.21.15.99:0/2652554497 conn(0x7f32c8077fa0 msgr2=0x7f32c80783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.343+0000 7f32cfc00700 1 -- 172.21.15.99:0/2652554497 shutdown_connections 2024-03-30T00:33:54.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.343+0000 7f32cfc00700 1 -- 172.21.15.99:0/2652554497 wait complete. 2024-03-30T00:33:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.344+0000 7f32cfc00700 1 Processor -- start 2024-03-30T00:33:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.345+0000 7f32cfc00700 1 -- start start 2024-03-30T00:33:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.345+0000 7f32cfc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c8072fe0 0x7f32c8199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.345+0000 7f32cfc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8073900 0x7f32c8199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.345+0000 7f32cfc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32c810d810 0x7f32c819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.345+0000 7f32cfc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c819e340 con 0x7f32c8073900 2024-03-30T00:33:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.345+0000 7f32cfc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c819e4b0 con 0x7f32c810d810 2024-03-30T00:33:54.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.345+0000 7f32cfc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c819e670 con 0x7f32c8072fe0 2024-03-30T00:33:54.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.345+0000 7f32c7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8073900 0x7f32c8199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:54.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.345+0000 7f32c7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8073900 0x7f32c8199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55834/0 (socket says 172.21.15.99:55834) 2024-03-30T00:33:54.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.345+0000 7f32c7e00700 1 -- 172.21.15.99:0/72822799 learned_addr learned my addr 172.21.15.99:0/72822799 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:54.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.345+0000 7f32cdc00700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32c810d810 0x7f32c819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:54.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32cd200700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c8072fe0 0x7f32c8199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:54.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32c7e00700 1 -- 172.21.15.99:0/72822799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c8072fe0 msgr2=0x7f32c8199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:54.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32c7e00700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c8072fe0 0x7f32c8199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32c7e00700 1 -- 172.21.15.99:0/72822799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32c810d810 msgr2=0x7f32c819dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:54.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32c7e00700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32c810d810 0x7f32c819dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32c7e00700 1 -- 172.21.15.99:0/72822799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0095f0 con 0x7f32c8073900 2024-03-30T00:33:54.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32cd200700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c8072fe0 0x7f32c8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:54.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32c7e00700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8073900 0x7f32c8199980 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f32b400d950 tx=0x7f32b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:54.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32c5600700 1 -- 172.21.15.99:0/72822799 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4009940 con 0x7f32c8073900 2024-03-30T00:33:54.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32c5600700 1 -- 172.21.15.99:0/72822799 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32b4010460 con 0x7f32c8073900 2024-03-30T00:33:54.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32c5600700 1 -- 172.21.15.99:0/72822799 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400f620 con 0x7f32c8073900 2024-03-30T00:33:54.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32cfc00700 1 -- 172.21.15.99:0/72822799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c819e950 con 0x7f32c8073900 2024-03-30T00:33:54.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.346+0000 7f32cfc00700 1 -- 172.21.15.99:0/72822799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c81a61b0 con 0x7f32c8073900 2024-03-30T00:33:54.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.347+0000 7f32cfc00700 1 -- 172.21.15.99:0/72822799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c8108a20 con 0x7f32c8073900 2024-03-30T00:33:54.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.349+0000 7f32c5600700 1 -- 172.21.15.99:0/72822799 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f32b40105d0 con 0x7f32c8073900 2024-03-30T00:33:54.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.351+0000 7f32c5600700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f32b00a1230 0x7f32b00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:54.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.352+0000 7f32cd200700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f32b00a1230 0x7f32b00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:54.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.352+0000 7f32c5600700 1 -- 172.21.15.99:0/72822799 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f32b40cc7a0 con 0x7f32c8073900 2024-03-30T00:33:54.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.352+0000 7f32cd200700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f32b00a1230 0x7f32b00a36f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00b5c0 tx=0x7f32bc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:54.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.356+0000 7f32c5600700 1 -- 172.21.15.99:0/72822799 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f32b4095e80 con 0x7f32c8073900 2024-03-30T00:33:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:54 smithi161 ceph-mon[42221]: pgmap v275: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-30T00:33:54.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:54 smithi142 ceph-mon[41779]: pgmap v275: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-30T00:33:54.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.573+0000 7f32cfc00700 1 -- 172.21.15.99:0/72822799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f32c819eb40 con 0x7f32c8073900 2024-03-30T00:33:54.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.574+0000 7f32c5600700 1 -- 172.21.15.99:0/72822799 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f32b40994a0 con 0x7f32c8073900 2024-03-30T00:33:54.580 INFO:teuthology.orchestra.run.smithi099.stdout:249108103200 2024-03-30T00:33:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 -- 172.21.15.99:0/72822799 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f32b00a1230 msgr2=0x7f32b00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f32b00a1230 0x7f32b00a36f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00b5c0 tx=0x7f32bc005fb0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 -- 172.21.15.99:0/72822799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8073900 msgr2=0x7f32c8199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8073900 0x7f32c8199980 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f32b400d950 tx=0x7f32b400dc60 comp rx=0 tx=0).stop 2024-03-30T00:33:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 -- 172.21.15.99:0/72822799 shutdown_connections 2024-03-30T00:33:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f32b00a1230 0x7f32b00a36f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32c810d810 0x7f32c819dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32c8072fe0 0x7f32c8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 --2- 172.21.15.99:0/72822799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8073900 0x7f32c8199980 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 -- 172.21.15.99:0/72822799 >> 172.21.15.99:0/72822799 conn(0x7f32c8077fa0 msgr2=0x7f32c80fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:54.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 -- 172.21.15.99:0/72822799 shutdown_connections 2024-03-30T00:33:54.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.576+0000 7f32cfc00700 1 -- 172.21.15.99:0/72822799 wait complete. 2024-03-30T00:33:54.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.841+0000 7f3854800700 1 -- 172.21.15.99:0/1865834510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f384c101820 msgr2=0x7f384c10dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:54.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.841+0000 7f3854800700 1 --2- 172.21.15.99:0/1865834510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f384c101820 0x7f384c10dc10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f384800b0d0 tx=0x7f384800b3e0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.841+0000 7f3854800700 1 -- 172.21.15.99:0/1865834510 shutdown_connections 2024-03-30T00:33:54.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.841+0000 7f3854800700 1 --2- 172.21.15.99:0/1865834510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f384c101820 0x7f384c10dc10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.841+0000 7f3854800700 1 --2- 172.21.15.99:0/1865834510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f384c106ef0 0x7f384c1072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.841+0000 7f3854800700 1 --2- 172.21.15.99:0/1865834510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c100e60 0x7f384c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.841+0000 7f3854800700 1 -- 172.21.15.99:0/1865834510 >> 172.21.15.99:0/1865834510 conn(0x7f384c077fa0 msgr2=0x7f384c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:54.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.841+0000 7f3854800700 1 -- 172.21.15.99:0/1865834510 shutdown_connections 2024-03-30T00:33:54.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.841+0000 7f3854800700 1 -- 172.21.15.99:0/1865834510 wait complete. 2024-03-30T00:33:54.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.842+0000 7f3854800700 1 Processor -- start 2024-03-30T00:33:54.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.842+0000 7f3854800700 1 -- start start 2024-03-30T00:33:54.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.842+0000 7f3854800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c100e60 0x7f384c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.842+0000 7f3854800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f384c101820 0x7f384c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.842+0000 7f3854800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f384c106ef0 0x7f384c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.842+0000 7f3854800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c19c130 con 0x7f384c106ef0 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.842+0000 7f3854800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c19c2a0 con 0x7f384c101820 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.842+0000 7f3854800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c19c460 con 0x7f384c100e60 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.843+0000 7f3852800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f384c106ef0 0x7f384c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.843+0000 7f3852800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f384c106ef0 0x7f384c19bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55858/0 (socket says 172.21.15.99:55858) 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.843+0000 7f3852800700 1 -- 172.21.15.99:0/1149137241 learned_addr learned my addr 172.21.15.99:0/1149137241 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.843+0000 7f3851e00700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c100e60 0x7f384c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.843+0000 7f3852800700 1 -- 172.21.15.99:0/1149137241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c100e60 msgr2=0x7f384c197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.843+0000 7f3852800700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c100e60 0x7f384c197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.843+0000 7f3852800700 1 -- 172.21.15.99:0/1149137241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f384c101820 msgr2=0x7f384c197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:54.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.843+0000 7f3852800700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f384c101820 0x7f384c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:54.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.843+0000 7f3852800700 1 -- 172.21.15.99:0/1149137241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848009d00 con 0x7f384c106ef0 2024-03-30T00:33:54.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.843+0000 7f3852800700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f384c106ef0 0x7f384c19bb20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3848007fd0 tx=0x7f38480077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:54.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.844+0000 7f3846a00700 1 -- 172.21.15.99:0/1149137241 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384800e040 con 0x7f384c106ef0 2024-03-30T00:33:54.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.844+0000 7f3846a00700 1 -- 172.21.15.99:0/1149137241 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3848003e90 con 0x7f384c106ef0 2024-03-30T00:33:54.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.844+0000 7f3846a00700 1 -- 172.21.15.99:0/1149137241 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384801b470 con 0x7f384c106ef0 2024-03-30T00:33:54.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.844+0000 7f3854800700 1 -- 172.21.15.99:0/1149137241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384c19c6e0 con 0x7f384c106ef0 2024-03-30T00:33:54.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.844+0000 7f3854800700 1 -- 172.21.15.99:0/1149137241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c1a3dd0 con 0x7f384c106ef0 2024-03-30T00:33:54.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.845+0000 7f3854800700 1 -- 172.21.15.99:0/1149137241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384c1092f0 con 0x7f384c106ef0 2024-03-30T00:33:54.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.847+0000 7f3846a00700 1 -- 172.21.15.99:0/1149137241 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3848003a00 con 0x7f384c106ef0 2024-03-30T00:33:54.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.848+0000 7f3846a00700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f38340a1230 0x7f38340a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:54.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.848+0000 7f3846a00700 1 -- 172.21.15.99:0/1149137241 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f38480cdb60 con 0x7f384c106ef0 2024-03-30T00:33:54.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.848+0000 7f3851e00700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f38340a1230 0x7f38340a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:54.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.849+0000 7f3851e00700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f38340a1230 0x7f38340a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3840007b60 tx=0x7f38400073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:54.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:54.852+0000 7f3846a00700 1 -- 172.21.15.99:0/1149137241 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f38480972d0 con 0x7f384c106ef0 2024-03-30T00:33:55.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.110+0000 7f3854800700 1 -- 172.21.15.99:0/1149137241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f384c1024f0 con 0x7f384c106ef0 2024-03-30T00:33:55.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.111+0000 7f3846a00700 1 -- 172.21.15.99:0/1149137241 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f384809a8f0 con 0x7f384c106ef0 2024-03-30T00:33:55.118 INFO:teuthology.orchestra.run.smithi099.stdout:55834574911 2024-03-30T00:33:55.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.114+0000 7f3854800700 1 -- 172.21.15.99:0/1149137241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f38340a1230 msgr2=0x7f38340a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.114+0000 7f3854800700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f38340a1230 0x7f38340a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3840007b60 tx=0x7f38400073d0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.114+0000 7f3854800700 1 -- 172.21.15.99:0/1149137241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f384c106ef0 msgr2=0x7f384c19bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.114+0000 7f3854800700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f384c106ef0 0x7f384c19bb20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3848007fd0 tx=0x7f38480077f0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.114+0000 7f3854800700 1 -- 172.21.15.99:0/1149137241 shutdown_connections 2024-03-30T00:33:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.114+0000 7f3854800700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f384c106ef0 0x7f384c19bb20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.114+0000 7f3854800700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f384c100e60 0x7f384c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.114+0000 7f3854800700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f38340a1230 0x7f38340a36f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.114+0000 7f3854800700 1 --2- 172.21.15.99:0/1149137241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f384c101820 0x7f384c197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.114+0000 7f3854800700 1 -- 172.21.15.99:0/1149137241 >> 172.21.15.99:0/1149137241 conn(0x7f384c077fa0 msgr2=0x7f384c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:55.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.115+0000 7f3854800700 1 -- 172.21.15.99:0/1149137241 shutdown_connections 2024-03-30T00:33:55.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.115+0000 7f3854800700 1 -- 172.21.15.99:0/1149137241 wait complete. 2024-03-30T00:33:55.311 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:55 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/72822799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-30T00:33:55.311 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:55 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1149137241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-30T00:33:55.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:55 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/72822799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-30T00:33:55.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:55 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1149137241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-30T00:33:55.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.494+0000 7fdc9fc00700 1 -- 172.21.15.99:0/4058923857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc98068750 msgr2=0x7fdc98068bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:55.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.494+0000 7fdc9fc00700 1 --2- 172.21.15.99:0/4058923857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc98068750 0x7fdc98068bb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009990 tx=0x7fdc84009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.494+0000 7fdc9fc00700 1 -- 172.21.15.99:0/4058923857 shutdown_connections 2024-03-30T00:33:55.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.494+0000 7fdc9fc00700 1 --2- 172.21.15.99:0/4058923857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc980690f0 0x7fdc9810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.494+0000 7fdc9fc00700 1 --2- 172.21.15.99:0/4058923857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc98106df0 0x7fdc981071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.494+0000 7fdc9fc00700 1 --2- 172.21.15.99:0/4058923857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc98068750 0x7fdc98068bb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.494+0000 7fdc9fc00700 1 -- 172.21.15.99:0/4058923857 >> 172.21.15.99:0/4058923857 conn(0x7fdc98075d50 msgr2=0x7fdc98076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:55.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.494+0000 7fdc9fc00700 1 -- 172.21.15.99:0/4058923857 shutdown_connections 2024-03-30T00:33:55.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.494+0000 7fdc9fc00700 1 -- 172.21.15.99:0/4058923857 wait complete. 2024-03-30T00:33:55.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.495+0000 7fdc9fc00700 1 Processor -- start 2024-03-30T00:33:55.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc9fc00700 1 -- start start 2024-03-30T00:33:55.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc9fc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98068750 0x7fdc98199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:55.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc9fc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc980690f0 0x7fdc98199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:55.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc9fc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc98106df0 0x7fdc9819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:55.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc9fc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9819e210 con 0x7fdc98106df0 2024-03-30T00:33:55.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc9fc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9819e380 con 0x7fdc980690f0 2024-03-30T00:33:55.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc9fc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9819e540 con 0x7fdc98068750 2024-03-30T00:33:55.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc9dc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc98106df0 0x7fdc9819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:55.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc9d200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98068750 0x7fdc98199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:55.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc9dc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc98106df0 0x7fdc9819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38768/0 (socket says 172.21.15.99:38768) 2024-03-30T00:33:55.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc9dc00700 1 -- 172.21.15.99:0/3005429445 learned_addr learned my addr 172.21.15.99:0/3005429445 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:55.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.496+0000 7fdc97e00700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc980690f0 0x7fdc98199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:55.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc9dc00700 1 -- 172.21.15.99:0/3005429445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98068750 msgr2=0x7fdc98199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc9dc00700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98068750 0x7fdc98199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc9dc00700 1 -- 172.21.15.99:0/3005429445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc980690f0 msgr2=0x7fdc98199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc9dc00700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc980690f0 0x7fdc98199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc9dc00700 1 -- 172.21.15.99:0/3005429445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc840095f0 con 0x7fdc98106df0 2024-03-30T00:33:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc97e00700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc980690f0 0x7fdc98199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc9dc00700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc98106df0 0x7fdc9819dc00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdc90007ee0 tx=0x7fdc9000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc95600700 1 -- 172.21.15.99:0/3005429445 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9000ddd0 con 0x7fdc98106df0 2024-03-30T00:33:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc95600700 1 -- 172.21.15.99:0/3005429445 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc9000f040 con 0x7fdc98106df0 2024-03-30T00:33:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc9fc00700 1 -- 172.21.15.99:0/3005429445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9819e820 con 0x7fdc98106df0 2024-03-30T00:33:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc95600700 1 -- 172.21.15.99:0/3005429445 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900047d0 con 0x7fdc98106df0 2024-03-30T00:33:55.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.497+0000 7fdc9fc00700 1 -- 172.21.15.99:0/3005429445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc981a6080 con 0x7fdc98106df0 2024-03-30T00:33:55.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.498+0000 7fdc9fc00700 1 -- 172.21.15.99:0/3005429445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc981024a0 con 0x7fdc98106df0 2024-03-30T00:33:55.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.500+0000 7fdc95600700 1 -- 172.21.15.99:0/3005429445 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fdc90003a00 con 0x7fdc98106df0 2024-03-30T00:33:55.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.501+0000 7fdc95600700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdc800a1230 0x7fdc800a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:55.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.501+0000 7fdc95600700 1 -- 172.21.15.99:0/3005429445 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdc900cc250 con 0x7fdc98106df0 2024-03-30T00:33:55.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.502+0000 7fdc9d200700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdc800a1230 0x7fdc800a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:55.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.502+0000 7fdc9d200700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdc800a1230 0x7fdc800a36f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c005950 tx=0x7fdc8c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:55.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.506+0000 7fdc95600700 1 -- 172.21.15.99:0/3005429445 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdc90095930 con 0x7fdc98106df0 2024-03-30T00:33:55.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:55 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/72822799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-30T00:33:55.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:55 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1149137241' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-30T00:33:55.657 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103198 got 249108103200 for osd.8 2024-03-30T00:33:55.658 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:55.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.739+0000 7fdc9fc00700 1 -- 172.21.15.99:0/3005429445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdc98110b60 con 0x7fdc98106df0 2024-03-30T00:33:55.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.739+0000 7fdc95600700 1 -- 172.21.15.99:0/3005429445 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc90098f50 con 0x7fdc98106df0 2024-03-30T00:33:55.747 INFO:teuthology.orchestra.run.smithi099.stdout:133143986226 2024-03-30T00:33:55.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 -- 172.21.15.99:0/3005429445 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdc800a1230 msgr2=0x7fdc800a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:55.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdc800a1230 0x7fdc800a36f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c005950 tx=0x7fdc8c00b410 comp rx=0 tx=0).stop 2024-03-30T00:33:55.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 -- 172.21.15.99:0/3005429445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc98106df0 msgr2=0x7fdc9819dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:55.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc98106df0 0x7fdc9819dc00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdc90007ee0 tx=0x7fdc9000d3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 -- 172.21.15.99:0/3005429445 shutdown_connections 2024-03-30T00:33:55.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc98106df0 0x7fdc9819dc00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdc98068750 0x7fdc98199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fdc800a1230 0x7fdc800a36f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 --2- 172.21.15.99:0/3005429445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc980690f0 0x7fdc98199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 -- 172.21.15.99:0/3005429445 >> 172.21.15.99:0/3005429445 conn(0x7fdc98075d50 msgr2=0x7fdc9810bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:55.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 -- 172.21.15.99:0/3005429445 shutdown_connections 2024-03-30T00:33:55.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.742+0000 7fdc9fc00700 1 -- 172.21.15.99:0/3005429445 wait complete. 2024-03-30T00:33:55.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.851+0000 7fc3f1a00700 1 -- 172.21.15.99:0/1860718634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ec072fe0 msgr2=0x7fc3ec0733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:55.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.851+0000 7fc3f1a00700 1 --2- 172.21.15.99:0/1860718634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ec072fe0 0x7fc3ec0733c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009990 tx=0x7fc3dc009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.851+0000 7fc3f1a00700 1 -- 172.21.15.99:0/1860718634 shutdown_connections 2024-03-30T00:33:55.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.851+0000 7fc3f1a00700 1 --2- 172.21.15.99:0/1860718634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3ec10d810 0x7fc3ec10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.851+0000 7fc3f1a00700 1 --2- 172.21.15.99:0/1860718634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ec072fe0 0x7fc3ec0733c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.851+0000 7fc3f1a00700 1 --2- 172.21.15.99:0/1860718634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3ec073900 0x7fc3ec10d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.851+0000 7fc3f1a00700 1 -- 172.21.15.99:0/1860718634 >> 172.21.15.99:0/1860718634 conn(0x7fc3ec077fa0 msgr2=0x7fc3ec0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:55.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.851+0000 7fc3f1a00700 1 -- 172.21.15.99:0/1860718634 shutdown_connections 2024-03-30T00:33:55.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.851+0000 7fc3f1a00700 1 -- 172.21.15.99:0/1860718634 wait complete. 2024-03-30T00:33:55.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.852+0000 7fc3f1a00700 1 Processor -- start 2024-03-30T00:33:55.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.852+0000 7fc3f1a00700 1 -- start start 2024-03-30T00:33:55.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.852+0000 7fc3f1a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3ec072fe0 0x7fc3ec199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:55.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.852+0000 7fc3f1a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3ec073900 0x7fc3ec1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:55.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3f1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ec10d810 0x7fc3ec19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:55.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3f1a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec19e360 con 0x7fc3ec10d810 2024-03-30T00:33:55.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3f1a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec19e4d0 con 0x7fc3ec072fe0 2024-03-30T00:33:55.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3f1a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec19e690 con 0x7fc3ec073900 2024-03-30T00:33:55.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3eb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ec10d810 0x7fc3ec19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:55.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3eb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ec10d810 0x7fc3ec19dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38778/0 (socket says 172.21.15.99:38778) 2024-03-30T00:33:55.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3eb400700 1 -- 172.21.15.99:0/3583366062 learned_addr learned my addr 172.21.15.99:0/3583366062 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:55.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3ea000700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3ec073900 0x7fc3ec1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:55.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3eaa00700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3ec072fe0 0x7fc3ec199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:55.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3eb400700 1 -- 172.21.15.99:0/3583366062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3ec073900 msgr2=0x7fc3ec1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:55.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3eb400700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3ec073900 0x7fc3ec1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3eb400700 1 -- 172.21.15.99:0/3583366062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3ec072fe0 msgr2=0x7fc3ec199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:55.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3eb400700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3ec072fe0 0x7fc3ec199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:55.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3eb400700 1 -- 172.21.15.99:0/3583366062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc0095f0 con 0x7fc3ec10d810 2024-03-30T00:33:55.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3eaa00700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3ec072fe0 0x7fc3ec199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:55.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.853+0000 7fc3eb400700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ec10d810 0x7fc3ec19dd50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4007ee0 tx=0x7fc3e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:55.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.854+0000 7fc3e3400700 1 -- 172.21.15.99:0/3583366062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e400ddd0 con 0x7fc3ec10d810 2024-03-30T00:33:55.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.854+0000 7fc3e3400700 1 -- 172.21.15.99:0/3583366062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3e400f040 con 0x7fc3ec10d810 2024-03-30T00:33:55.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.854+0000 7fc3f1a00700 1 -- 172.21.15.99:0/3583366062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec19e970 con 0x7fc3ec10d810 2024-03-30T00:33:55.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.854+0000 7fc3e3400700 1 -- 172.21.15.99:0/3583366062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4005970 con 0x7fc3ec10d810 2024-03-30T00:33:55.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.854+0000 7fc3f1a00700 1 -- 172.21.15.99:0/3583366062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec1a61d0 con 0x7fc3ec10d810 2024-03-30T00:33:55.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.855+0000 7fc3f1a00700 1 -- 172.21.15.99:0/3583366062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ec108a20 con 0x7fc3ec10d810 2024-03-30T00:33:55.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.857+0000 7fc3e3400700 1 -- 172.21.15.99:0/3583366062 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc3e4004fb0 con 0x7fc3ec10d810 2024-03-30T00:33:55.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.858+0000 7fc3e3400700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3d00a1230 0x7fc3d00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:55.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.858+0000 7fc3e3400700 1 -- 172.21.15.99:0/3583366062 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3e40cc030 con 0x7fc3ec10d810 2024-03-30T00:33:55.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.858+0000 7fc3eaa00700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3d00a1230 0x7fc3d00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:55.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.859+0000 7fc3eaa00700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3d00a1230 0x7fc3d00a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b5c0 tx=0x7fc3dc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:55.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:55.862+0000 7fc3e3400700 1 -- 172.21.15.99:0/3583366062 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc3e4095810 con 0x7fc3ec10d810 2024-03-30T00:33:55.973 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574909 got 55834574911 for osd.1 2024-03-30T00:33:55.973 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:56.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.105+0000 7fc3f1a00700 1 -- 172.21.15.99:0/3583366062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fc3ec19a770 con 0x7fc3ec10d810 2024-03-30T00:33:56.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.106+0000 7fc3e3400700 1 -- 172.21.15.99:0/3583366062 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc3e4098e30 con 0x7fc3ec10d810 2024-03-30T00:33:56.113 INFO:teuthology.orchestra.run.smithi099.stdout:304942678040 2024-03-30T00:33:56.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.108+0000 7fc3f1a00700 1 -- 172.21.15.99:0/3583366062 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3d00a1230 msgr2=0x7fc3d00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:56.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.108+0000 7fc3f1a00700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3d00a1230 0x7fc3d00a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00b5c0 tx=0x7fc3dc005fb0 comp rx=0 tx=0).stop 2024-03-30T00:33:56.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.108+0000 7fc3f1a00700 1 -- 172.21.15.99:0/3583366062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ec10d810 msgr2=0x7fc3ec19dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:56.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.108+0000 7fc3f1a00700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ec10d810 0x7fc3ec19dd50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4007ee0 tx=0x7fc3e400d3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:56.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.108+0000 7fc3f1a00700 1 -- 172.21.15.99:0/3583366062 shutdown_connections 2024-03-30T00:33:56.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.108+0000 7fc3f1a00700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3ec10d810 0x7fc3ec19dd50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:56.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.108+0000 7fc3f1a00700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3d00a1230 0x7fc3d00a36f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:56.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.108+0000 7fc3f1a00700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3ec072fe0 0x7fc3ec199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:56.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.108+0000 7fc3f1a00700 1 --2- 172.21.15.99:0/3583366062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3ec073900 0x7fc3ec1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:56.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.108+0000 7fc3f1a00700 1 -- 172.21.15.99:0/3583366062 >> 172.21.15.99:0/3583366062 conn(0x7fc3ec077fa0 msgr2=0x7fc3ec0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:56.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.108+0000 7fc3f1a00700 1 -- 172.21.15.99:0/3583366062 shutdown_connections 2024-03-30T00:33:56.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:56.109+0000 7fc3f1a00700 1 -- 172.21.15.99:0/3583366062 wait complete. 2024-03-30T00:33:56.294 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986224 got 133143986226 for osd.4 2024-03-30T00:33:56.294 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:56 smithi161 ceph-mon[42221]: pgmap v276: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-30T00:33:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:56 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3005429445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-30T00:33:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:56 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3583366062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-30T00:33:56.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:56 smithi099 ceph-mon[38079]: pgmap v276: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-30T00:33:56.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:56 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3005429445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-30T00:33:56.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:56 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3583366062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-30T00:33:56.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:56 smithi142 ceph-mon[41779]: pgmap v276: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-30T00:33:56.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:56 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3005429445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-30T00:33:56.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:56 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3583366062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-30T00:33:56.687 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678038 got 304942678040 for osd.10 2024-03-30T00:33:56.688 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:57.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.027+0000 7fe932400700 1 -- 172.21.15.99:0/1501163083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe92c101920 msgr2=0x7fe92c105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:57.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.027+0000 7fe932400700 1 --2- 172.21.15.99:0/1501163083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe92c101920 0x7fe92c105970 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe914009990 tx=0x7fe914009ca0 comp rx=0 tx=0).stop 2024-03-30T00:33:57.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.027+0000 7fe932400700 1 -- 172.21.15.99:0/1501163083 shutdown_connections 2024-03-30T00:33:57.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.027+0000 7fe932400700 1 --2- 172.21.15.99:0/1501163083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c106050 0x7fe92c10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:57.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.027+0000 7fe932400700 1 --2- 172.21.15.99:0/1501163083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe92c100f70 0x7fe92c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:57.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.027+0000 7fe932400700 1 --2- 172.21.15.99:0/1501163083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe92c101920 0x7fe92c105970 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:57.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.027+0000 7fe932400700 1 -- 172.21.15.99:0/1501163083 >> 172.21.15.99:0/1501163083 conn(0x7fe92c0fc7c0 msgr2=0x7fe92c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:57.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.027+0000 7fe932400700 1 -- 172.21.15.99:0/1501163083 shutdown_connections 2024-03-30T00:33:57.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.027+0000 7fe932400700 1 -- 172.21.15.99:0/1501163083 wait complete. 2024-03-30T00:33:57.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.028+0000 7fe932400700 1 Processor -- start 2024-03-30T00:33:57.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.028+0000 7fe932400700 1 -- start start 2024-03-30T00:33:57.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.028+0000 7fe932400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe92c100f70 0x7fe92c197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:57.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.028+0000 7fe932400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe92c101920 0x7fe92c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:57.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.028+0000 7fe932400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c106050 0x7fe92c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:57.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.028+0000 7fe932400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c19c040 con 0x7fe92c101920 2024-03-30T00:33:57.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.028+0000 7fe932400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c19c1b0 con 0x7fe92c106050 2024-03-30T00:33:57.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.028+0000 7fe932400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c19c370 con 0x7fe92c100f70 2024-03-30T00:33:57.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe923e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe92c101920 0x7fe92c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe923e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe92c101920 0x7fe92c197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38796/0 (socket says 172.21.15.99:38796) 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe923e00700 1 -- 172.21.15.99:0/3834927917 learned_addr learned my addr 172.21.15.99:0/3834927917 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe92be00700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c106050 0x7fe92c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe92b400700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe92c100f70 0x7fe92c197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe923e00700 1 -- 172.21.15.99:0/3834927917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe92c100f70 msgr2=0x7fe92c197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe923e00700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe92c100f70 0x7fe92c197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe923e00700 1 -- 172.21.15.99:0/3834927917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c106050 msgr2=0x7fe92c19ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe923e00700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c106050 0x7fe92c19ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe923e00700 1 -- 172.21.15.99:0/3834927917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9140095f0 con 0x7fe92c101920 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe92b400700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe92c100f70 0x7fe92c197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.029+0000 7fe923e00700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe92c101920 0x7fe92c197680 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe914006010 tx=0x7fe9140057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.030+0000 7fe928c00700 1 -- 172.21.15.99:0/3834927917 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91401d070 con 0x7fe92c101920 2024-03-30T00:33:57.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.030+0000 7fe928c00700 1 -- 172.21.15.99:0/3834927917 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe91400fcc0 con 0x7fe92c101920 2024-03-30T00:33:57.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.030+0000 7fe932400700 1 -- 172.21.15.99:0/3834927917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c19c5f0 con 0x7fe92c101920 2024-03-30T00:33:57.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.030+0000 7fe928c00700 1 -- 172.21.15.99:0/3834927917 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe914022c00 con 0x7fe92c101920 2024-03-30T00:33:57.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.030+0000 7fe932400700 1 -- 172.21.15.99:0/3834927917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c0731b0 con 0x7fe92c101920 2024-03-30T00:33:57.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.031+0000 7fe932400700 1 -- 172.21.15.99:0/3834927917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c109370 con 0x7fe92c101920 2024-03-30T00:33:57.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.033+0000 7fe928c00700 1 -- 172.21.15.99:0/3834927917 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fe91400f830 con 0x7fe92c101920 2024-03-30T00:33:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.034+0000 7fe928c00700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9100a11b0 0x7fe9100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.034+0000 7fe928c00700 1 -- 172.21.15.99:0/3834927917 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe9140cde20 con 0x7fe92c101920 2024-03-30T00:33:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.034+0000 7fe92b400700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9100a11b0 0x7fe9100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:57.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.035+0000 7fe92b400700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9100a11b0 0x7fe9100a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe91c007900 tx=0x7fe91c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:57.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.038+0000 7fe928c00700 1 -- 172.21.15.99:0/3834927917 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe9140977b0 con 0x7fe92c101920 2024-03-30T00:33:57.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.241+0000 7fe932400700 1 -- 172.21.15.99:0/3834927917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fe92c04ea80 con 0x7fe92c101920 2024-03-30T00:33:57.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.241+0000 7fe928c00700 1 -- 172.21.15.99:0/3834927917 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe914027020 con 0x7fe92c101920 2024-03-30T00:33:57.248 INFO:teuthology.orchestra.run.smithi099.stdout:283467841563 2024-03-30T00:33:57.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.243+0000 7fe932400700 1 -- 172.21.15.99:0/3834927917 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9100a11b0 msgr2=0x7fe9100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:57.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.243+0000 7fe932400700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9100a11b0 0x7fe9100a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe91c007900 tx=0x7fe91c008040 comp rx=0 tx=0).stop 2024-03-30T00:33:57.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.243+0000 7fe932400700 1 -- 172.21.15.99:0/3834927917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe92c101920 msgr2=0x7fe92c197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:57.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.243+0000 7fe932400700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe92c101920 0x7fe92c197680 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe914006010 tx=0x7fe9140057d0 comp rx=0 tx=0).stop 2024-03-30T00:33:57.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.244+0000 7fe932400700 1 -- 172.21.15.99:0/3834927917 shutdown_connections 2024-03-30T00:33:57.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.244+0000 7fe932400700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe9100a11b0 0x7fe9100a3670 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:57.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.244+0000 7fe932400700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe92c106050 0x7fe92c19ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:57.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.244+0000 7fe932400700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe92c100f70 0x7fe92c197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:57.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.244+0000 7fe932400700 1 --2- 172.21.15.99:0/3834927917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe92c101920 0x7fe92c197680 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:57.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.244+0000 7fe932400700 1 -- 172.21.15.99:0/3834927917 >> 172.21.15.99:0/3834927917 conn(0x7fe92c0fc7c0 msgr2=0x7fe92c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:57.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.244+0000 7fe932400700 1 -- 172.21.15.99:0/3834927917 shutdown_connections 2024-03-30T00:33:57.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:57.244+0000 7fe932400700 1 -- 172.21.15.99:0/3834927917 wait complete. 2024-03-30T00:33:57.625 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841561 got 283467841563 for osd.9 2024-03-30T00:33:57.625 DEBUG:teuthology.parallel:result is None 2024-03-30T00:33:57.625 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-30T00:33:57.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-03-30T00:33:57.985 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:33:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:58 smithi161 ceph-mon[42221]: pgmap v277: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-30T00:33:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:33:58 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3834927917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-30T00:33:58.479 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:58 smithi099 ceph-mon[38079]: pgmap v277: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-30T00:33:58.479 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:33:58 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3834927917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-30T00:33:58.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:58 smithi142 ceph-mon[41779]: pgmap v277: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-30T00:33:58.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:33:58 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3834927917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-30T00:33:59.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.097+0000 7f5f68c00700 1 -- 172.21.15.99:0/2538079169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f641017c0 msgr2=0x7f5f6410dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:59.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.097+0000 7f5f68c00700 1 --2- 172.21.15.99:0/2538079169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f641017c0 0x7f5f6410dbf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00b0a0 tx=0x7f5f5c00b3b0 comp rx=0 tx=0).stop 2024-03-30T00:33:59.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.097+0000 7f5f68c00700 1 -- 172.21.15.99:0/2538079169 shutdown_connections 2024-03-30T00:33:59.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.097+0000 7f5f68c00700 1 --2- 172.21.15.99:0/2538079169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f641017c0 0x7f5f6410dbf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:59.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.097+0000 7f5f68c00700 1 --2- 172.21.15.99:0/2538079169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f64106e40 0x7f5f64107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:59.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.097+0000 7f5f68c00700 1 --2- 172.21.15.99:0/2538079169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f64100e00 0x7f5f64101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:59.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.097+0000 7f5f68c00700 1 -- 172.21.15.99:0/2538079169 >> 172.21.15.99:0/2538079169 conn(0x7f5f64077f10 msgr2=0x7f5f64078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:59.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.097+0000 7f5f68c00700 1 -- 172.21.15.99:0/2538079169 shutdown_connections 2024-03-30T00:33:59.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.097+0000 7f5f68c00700 1 -- 172.21.15.99:0/2538079169 wait complete. 2024-03-30T00:33:59.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.098+0000 7f5f68c00700 1 Processor -- start 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.098+0000 7f5f68c00700 1 -- start start 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.098+0000 7f5f68c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f64100e00 0x7f5f64199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.098+0000 7f5f68c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f641017c0 0x7f5f64199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.098+0000 7f5f68c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f64106e40 0x7f5f6419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.098+0000 7f5f68c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6419e210 con 0x7f5f641017c0 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.098+0000 7f5f68c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6419e380 con 0x7f5f64100e00 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.098+0000 7f5f68c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6419e540 con 0x7f5f64106e40 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f5be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f641017c0 0x7f5f64199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f5be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f641017c0 0x7f5f64199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38822/0 (socket says 172.21.15.99:38822) 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f61e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f64100e00 0x7f5f64199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f5be00700 1 -- 172.21.15.99:0/3635032630 learned_addr learned my addr 172.21.15.99:0/3635032630 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f62800700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f64106e40 0x7f5f6419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:59.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f5be00700 1 -- 172.21.15.99:0/3635032630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f64106e40 msgr2=0x7f5f6419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:59.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f5be00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f64106e40 0x7f5f6419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:59.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f5be00700 1 -- 172.21.15.99:0/3635032630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f64100e00 msgr2=0x7f5f64199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:59.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f5be00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f64100e00 0x7f5f64199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:59.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f5be00700 1 -- 172.21.15.99:0/3635032630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c009d00 con 0x7f5f641017c0 2024-03-30T00:33:59.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f62800700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f64106e40 0x7f5f6419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:33:59.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f5be00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f641017c0 0x7f5f64199850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00d950 tx=0x7f5f4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:59.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.099+0000 7f5f5aa00700 1 -- 172.21.15.99:0/3635032630 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c009940 con 0x7f5f641017c0 2024-03-30T00:33:59.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.100+0000 7f5f5aa00700 1 -- 172.21.15.99:0/3635032630 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f4c010460 con 0x7f5f641017c0 2024-03-30T00:33:59.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.100+0000 7f5f5aa00700 1 -- 172.21.15.99:0/3635032630 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00f620 con 0x7f5f641017c0 2024-03-30T00:33:59.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.100+0000 7f5f68c00700 1 -- 172.21.15.99:0/3635032630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6419e820 con 0x7f5f641017c0 2024-03-30T00:33:59.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.100+0000 7f5f68c00700 1 -- 172.21.15.99:0/3635032630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f641a6080 con 0x7f5f641017c0 2024-03-30T00:33:59.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.101+0000 7f5f68c00700 1 -- 172.21.15.99:0/3635032630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f64104110 con 0x7f5f641017c0 2024-03-30T00:33:59.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.103+0000 7f5f5aa00700 1 -- 172.21.15.99:0/3635032630 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f5f4c0105d0 con 0x7f5f641017c0 2024-03-30T00:33:59.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.104+0000 7f5f5aa00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5f480a11b0 0x7f5f480a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:33:59.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.104+0000 7f5f5aa00700 1 -- 172.21.15.99:0/3635032630 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5f4c0cc810 con 0x7f5f641017c0 2024-03-30T00:33:59.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.104+0000 7f5f61e00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5f480a11b0 0x7f5f480a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:33:59.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.105+0000 7f5f61e00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5f480a11b0 0x7f5f480a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009c60 tx=0x7f5f54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:33:59.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.106+0000 7f5f5aa00700 1 -- 172.21.15.99:0/3635032630 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5f4c095ef0 con 0x7f5f641017c0 2024-03-30T00:33:59.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.307+0000 7f5f68c00700 1 -- 172.21.15.99:0/3635032630 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f6419ea10 con 0x7f5f480a11b0 2024-03-30T00:33:59.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.317+0000 7f5f5aa00700 1 -- 172.21.15.99:0/3635032630 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340241 (secure 0 0 0) 0x7f5f6419ea10 con 0x7f5f480a11b0 2024-03-30T00:33:59.329 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:33:59.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.322+0000 7f5f68c00700 1 -- 172.21.15.99:0/3635032630 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5f480a11b0 msgr2=0x7f5f480a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:59.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.322+0000 7f5f68c00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5f480a11b0 0x7f5f480a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009c60 tx=0x7f5f54009260 comp rx=0 tx=0).stop 2024-03-30T00:33:59.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.322+0000 7f5f68c00700 1 -- 172.21.15.99:0/3635032630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f641017c0 msgr2=0x7f5f64199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:33:59.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.322+0000 7f5f68c00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f641017c0 0x7f5f64199850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00d950 tx=0x7f5f4c00dc60 comp rx=0 tx=0).stop 2024-03-30T00:33:59.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.322+0000 7f5f68c00700 1 -- 172.21.15.99:0/3635032630 shutdown_connections 2024-03-30T00:33:59.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.322+0000 7f5f68c00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f64106e40 0x7f5f6419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:59.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.322+0000 7f5f68c00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f5f480a11b0 0x7f5f480a3670 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:59.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.322+0000 7f5f68c00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f64100e00 0x7f5f64199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:59.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.322+0000 7f5f68c00700 1 --2- 172.21.15.99:0/3635032630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f641017c0 0x7f5f64199850 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:33:59.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.322+0000 7f5f68c00700 1 -- 172.21.15.99:0/3635032630 >> 172.21.15.99:0/3635032630 conn(0x7f5f64077f10 msgr2=0x7f5f6410c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:33:59.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.323+0000 7f5f68c00700 1 -- 172.21.15.99:0/3635032630 shutdown_connections 2024-03-30T00:33:59.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:33:59.323+0000 7f5f68c00700 1 -- 172.21.15.99:0/3635032630 wait complete. 2024-03-30T00:33:59.333 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-30T00:33:59.690 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-03-30T00:33:59.018491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656735,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":369,"num_read_kb":301,"num_write":339,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377840,"kb_used_data":12688,"kb_used_omap":18,"kb_used_meta":365037,"kb_avail":1124563984,"statfs":{"total":1151940427776,"available":1151553519616,"internally_reserved":0,"allocated":12992512,"data_stored":7117317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":373798214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3564,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":39,"num_write":3,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003053"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290853+0000","last_change":"2024-03-30T00:33:15.686814+0000","last_active":"2024-03-30T00:33:17.290853+0000","last_peered":"2024-03-30T00:33:17.290853+0000","last_clean":"2024-03-30T00:33:17.290853+0000","last_became_active":"2024-03-30T00:33:15.686432+0000","last_became_peered":"2024-03-30T00:33:15.686432+0000","last_unstale":"2024-03-30T00:33:17.290853+0000","last_undegraded":"2024-03-30T00:33:17.290853+0000","last_fullsized":"2024-03-30T00:33:17.290853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:35:05.898973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002227+0000","last_change":"2024-03-30T00:33:15.678089+0000","last_active":"2024-03-30T00:33:17.002227+0000","last_peered":"2024-03-30T00:33:17.002227+0000","last_clean":"2024-03-30T00:33:17.002227+0000","last_became_active":"2024-03-30T00:33:15.677254+0000","last_became_peered":"2024-03-30T00:33:15.677254+0000","last_unstale":"2024-03-30T00:33:17.002227+0000","last_undegraded":"2024-03-30T00:33:17.002227+0000","last_fullsized":"2024-03-30T00:33:17.002227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:45.922652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002176+0000","last_change":"2024-03-30T00:33:15.686777+0000","last_active":"2024-03-30T00:33:17.002176+0000","last_peered":"2024-03-30T00:33:17.002176+0000","last_clean":"2024-03-30T00:33:17.002176+0000","last_became_active":"2024-03-30T00:33:15.686461+0000","last_became_peered":"2024-03-30T00:33:15.686461+0000","last_unstale":"2024-03-30T00:33:17.002176+0000","last_undegraded":"2024-03-30T00:33:17.002176+0000","last_fullsized":"2024-03-30T00:33:17.002176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:33:27.073089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002223+0000","last_change":"2024-03-30T00:33:15.681237+0000","last_active":"2024-03-30T00:33:17.002223+0000","last_peered":"2024-03-30T00:33:17.002223+0000","last_clean":"2024-03-30T00:33:17.002223+0000","last_became_active":"2024-03-30T00:33:15.680669+0000","last_became_peered":"2024-03-30T00:33:15.680669+0000","last_unstale":"2024-03-30T00:33:17.002223+0000","last_undegraded":"2024-03-30T00:33:17.002223+0000","last_fullsized":"2024-03-30T00:33:17.002223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:59.300262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.682286+0000","last_change":"2024-03-30T00:33:15.678825+0000","last_active":"2024-03-30T00:33:15.682286+0000","last_peered":"2024-03-30T00:33:15.682286+0000","last_clean":"2024-03-30T00:33:15.682286+0000","last_became_active":"2024-03-30T00:33:15.676864+0000","last_became_peered":"2024-03-30T00:33:15.676864+0000","last_unstale":"2024-03-30T00:33:15.682286+0000","last_undegraded":"2024-03-30T00:33:15.682286+0000","last_fullsized":"2024-03-30T00:33:15.682286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:12:09.724441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.688863+0000","last_change":"2024-03-30T00:33:15.681866+0000","last_active":"2024-03-30T00:33:15.688863+0000","last_peered":"2024-03-30T00:33:15.688863+0000","last_clean":"2024-03-30T00:33:15.688863+0000","last_became_active":"2024-03-30T00:33:15.678118+0000","last_became_peered":"2024-03-30T00:33:15.678118+0000","last_unstale":"2024-03-30T00:33:15.688863+0000","last_undegraded":"2024-03-30T00:33:15.688863+0000","last_fullsized":"2024-03-30T00:33:15.688863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:24.433027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001193+0000","last_change":"2024-03-30T00:33:15.675999+0000","last_active":"2024-03-30T00:33:17.001193+0000","last_peered":"2024-03-30T00:33:17.001193+0000","last_clean":"2024-03-30T00:33:17.001193+0000","last_became_active":"2024-03-30T00:33:15.675319+0000","last_became_peered":"2024-03-30T00:33:15.675319+0000","last_unstale":"2024-03-30T00:33:17.001193+0000","last_undegraded":"2024-03-30T00:33:17.001193+0000","last_fullsized":"2024-03-30T00:33:17.001193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:42.668747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000423+0000","last_change":"2024-03-30T00:33:15.679211+0000","last_active":"2024-03-30T00:33:17.000423+0000","last_peered":"2024-03-30T00:33:17.000423+0000","last_clean":"2024-03-30T00:33:17.000423+0000","last_became_active":"2024-03-30T00:33:15.675061+0000","last_became_peered":"2024-03-30T00:33:15.675061+0000","last_unstale":"2024-03-30T00:33:17.000423+0000","last_undegraded":"2024-03-30T00:33:17.000423+0000","last_fullsized":"2024-03-30T00:33:17.000423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:43.973542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.692417+0000","last_change":"2024-03-30T00:33:15.680842+0000","last_active":"2024-03-30T00:33:15.692417+0000","last_peered":"2024-03-30T00:33:15.692417+0000","last_clean":"2024-03-30T00:33:15.692417+0000","last_became_active":"2024-03-30T00:33:15.680307+0000","last_became_peered":"2024-03-30T00:33:15.680307+0000","last_unstale":"2024-03-30T00:33:15.692417+0000","last_undegraded":"2024-03-30T00:33:15.692417+0000","last_fullsized":"2024-03-30T00:33:15.692417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:59:40.331779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693556+0000","last_change":"2024-03-30T00:33:15.692397+0000","last_active":"2024-03-30T00:33:15.693556+0000","last_peered":"2024-03-30T00:33:15.693556+0000","last_clean":"2024-03-30T00:33:15.693556+0000","last_became_active":"2024-03-30T00:33:15.691226+0000","last_became_peered":"2024-03-30T00:33:15.691226+0000","last_unstale":"2024-03-30T00:33:15.693556+0000","last_undegraded":"2024-03-30T00:33:15.693556+0000","last_fullsized":"2024-03-30T00:33:15.693556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:48.816644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001314+0000","last_change":"2024-03-30T00:33:15.688417+0000","last_active":"2024-03-30T00:33:17.001314+0000","last_peered":"2024-03-30T00:33:17.001314+0000","last_clean":"2024-03-30T00:33:17.001314+0000","last_became_active":"2024-03-30T00:33:15.687755+0000","last_became_peered":"2024-03-30T00:33:15.687755+0000","last_unstale":"2024-03-30T00:33:17.001314+0000","last_undegraded":"2024-03-30T00:33:17.001314+0000","last_fullsized":"2024-03-30T00:33:17.001314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:30:19.782280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002119+0000","last_change":"2024-03-30T00:33:15.681942+0000","last_active":"2024-03-30T00:33:17.002119+0000","last_peered":"2024-03-30T00:33:17.002119+0000","last_clean":"2024-03-30T00:33:17.002119+0000","last_became_active":"2024-03-30T00:33:15.679854+0000","last_became_peered":"2024-03-30T00:33:15.679854+0000","last_unstale":"2024-03-30T00:33:17.002119+0000","last_undegraded":"2024-03-30T00:33:17.002119+0000","last_fullsized":"2024-03-30T00:33:17.002119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:19.771959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001108+0000","last_change":"2024-03-30T00:33:15.682968+0000","last_active":"2024-03-30T00:33:17.001108+0000","last_peered":"2024-03-30T00:33:17.001108+0000","last_clean":"2024-03-30T00:33:17.001108+0000","last_became_active":"2024-03-30T00:33:15.681216+0000","last_became_peered":"2024-03-30T00:33:15.681216+0000","last_unstale":"2024-03-30T00:33:17.001108+0000","last_undegraded":"2024-03-30T00:33:17.001108+0000","last_fullsized":"2024-03-30T00:33:17.001108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:50.038843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001697+0000","last_change":"2024-03-30T00:33:15.677727+0000","last_active":"2024-03-30T00:33:17.001697+0000","last_peered":"2024-03-30T00:33:17.001697+0000","last_clean":"2024-03-30T00:33:17.001697+0000","last_became_active":"2024-03-30T00:33:15.676758+0000","last_became_peered":"2024-03-30T00:33:15.676758+0000","last_unstale":"2024-03-30T00:33:17.001697+0000","last_undegraded":"2024-03-30T00:33:17.001697+0000","last_fullsized":"2024-03-30T00:33:17.001697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:34.745920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001691+0000","last_change":"2024-03-30T00:33:15.684545+0000","last_active":"2024-03-30T00:33:17.001691+0000","last_peered":"2024-03-30T00:33:17.001691+0000","last_clean":"2024-03-30T00:33:17.001691+0000","last_became_active":"2024-03-30T00:33:15.682989+0000","last_became_peered":"2024-03-30T00:33:15.682989+0000","last_unstale":"2024-03-30T00:33:17.001691+0000","last_undegraded":"2024-03-30T00:33:17.001691+0000","last_fullsized":"2024-03-30T00:33:17.001691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:11.120981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.686822+0000","last_change":"2024-03-30T00:33:15.678122+0000","last_active":"2024-03-30T00:33:15.686822+0000","last_peered":"2024-03-30T00:33:15.686822+0000","last_clean":"2024-03-30T00:33:15.686822+0000","last_became_active":"2024-03-30T00:33:15.677681+0000","last_became_peered":"2024-03-30T00:33:15.677681+0000","last_unstale":"2024-03-30T00:33:15.686822+0000","last_undegraded":"2024-03-30T00:33:15.686822+0000","last_fullsized":"2024-03-30T00:33:15.686822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:50.454125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290980+0000","last_change":"2024-03-30T00:33:15.687689+0000","last_active":"2024-03-30T00:33:17.290980+0000","last_peered":"2024-03-30T00:33:17.290980+0000","last_clean":"2024-03-30T00:33:17.290980+0000","last_became_active":"2024-03-30T00:33:15.686009+0000","last_became_peered":"2024-03-30T00:33:15.686009+0000","last_unstale":"2024-03-30T00:33:17.290980+0000","last_undegraded":"2024-03-30T00:33:17.290980+0000","last_fullsized":"2024-03-30T00:33:17.290980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:17.234749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659641+0000","last_change":"2024-03-30T00:33:15.686053+0000","last_active":"2024-03-30T00:33:16.659641+0000","last_peered":"2024-03-30T00:33:16.659641+0000","last_clean":"2024-03-30T00:33:16.659641+0000","last_became_active":"2024-03-30T00:33:15.685723+0000","last_became_peered":"2024-03-30T00:33:15.685723+0000","last_unstale":"2024-03-30T00:33:16.659641+0000","last_undegraded":"2024-03-30T00:33:16.659641+0000","last_fullsized":"2024-03-30T00:33:16.659641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:27.322834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660740+0000","last_change":"2024-03-30T00:33:15.674783+0000","last_active":"2024-03-30T00:33:16.660740+0000","last_peered":"2024-03-30T00:33:16.660740+0000","last_clean":"2024-03-30T00:33:16.660740+0000","last_became_active":"2024-03-30T00:33:15.674152+0000","last_became_peered":"2024-03-30T00:33:15.674152+0000","last_unstale":"2024-03-30T00:33:16.660740+0000","last_undegraded":"2024-03-30T00:33:16.660740+0000","last_fullsized":"2024-03-30T00:33:16.660740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:39.517240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.119665+0000","last_change":"2024-03-30T00:33:15.172217+0000","last_active":"2024-03-30T00:33:20.119665+0000","last_peered":"2024-03-30T00:33:20.119665+0000","last_clean":"2024-03-30T00:33:20.119665+0000","last_became_active":"2024-03-30T00:33:15.171718+0000","last_became_peered":"2024-03-30T00:33:15.171718+0000","last_unstale":"2024-03-30T00:33:20.119665+0000","last_undegraded":"2024-03-30T00:33:20.119665+0000","last_fullsized":"2024-03-30T00:33:20.119665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:11.691572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001530+0000","last_change":"2024-03-30T00:33:15.669937+0000","last_active":"2024-03-30T00:33:17.001530+0000","last_peered":"2024-03-30T00:33:17.001530+0000","last_clean":"2024-03-30T00:33:17.001530+0000","last_became_active":"2024-03-30T00:33:15.668704+0000","last_became_peered":"2024-03-30T00:33:15.668704+0000","last_unstale":"2024-03-30T00:33:17.001530+0000","last_undegraded":"2024-03-30T00:33:17.001530+0000","last_fullsized":"2024-03-30T00:33:17.001530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:23.914755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:18.428835+0000","last_change":"2024-03-30T00:33:15.173663+0000","last_active":"2024-03-30T00:33:18.428835+0000","last_peered":"2024-03-30T00:33:18.428835+0000","last_clean":"2024-03-30T00:33:18.428835+0000","last_became_active":"2024-03-30T00:33:15.173288+0000","last_became_peered":"2024-03-30T00:33:15.173288+0000","last_unstale":"2024-03-30T00:33:18.428835+0000","last_undegraded":"2024-03-30T00:33:18.428835+0000","last_fullsized":"2024-03-30T00:33:18.428835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.435142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002383+0000","last_change":"2024-03-30T00:33:15.686757+0000","last_active":"2024-03-30T00:33:17.002383+0000","last_peered":"2024-03-30T00:33:17.002383+0000","last_clean":"2024-03-30T00:33:17.002383+0000","last_became_active":"2024-03-30T00:33:15.686306+0000","last_became_peered":"2024-03-30T00:33:15.686306+0000","last_unstale":"2024-03-30T00:33:17.002383+0000","last_undegraded":"2024-03-30T00:33:17.002383+0000","last_fullsized":"2024-03-30T00:33:17.002383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:50:21.048212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.117995+0000","last_change":"2024-03-30T00:33:15.173402+0000","last_active":"2024-03-30T00:33:20.117995+0000","last_peered":"2024-03-30T00:33:20.117995+0000","last_clean":"2024-03-30T00:33:20.117995+0000","last_became_active":"2024-03-30T00:33:15.172995+0000","last_became_peered":"2024-03-30T00:33:15.172995+0000","last_unstale":"2024-03-30T00:33:20.117995+0000","last_undegraded":"2024-03-30T00:33:20.117995+0000","last_fullsized":"2024-03-30T00:33:20.117995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:25.694792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.691152+0000","last_change":"2024-03-30T00:33:15.688292+0000","last_active":"2024-03-30T00:33:15.691152+0000","last_peered":"2024-03-30T00:33:15.691152+0000","last_clean":"2024-03-30T00:33:15.691152+0000","last_became_active":"2024-03-30T00:33:15.682722+0000","last_became_peered":"2024-03-30T00:33:15.682722+0000","last_unstale":"2024-03-30T00:33:15.691152+0000","last_undegraded":"2024-03-30T00:33:15.691152+0000","last_fullsized":"2024-03-30T00:33:15.691152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:52:10.074656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.692186+0000","last_change":"2024-03-30T00:33:15.162316+0000","last_active":"2024-03-30T00:33:15.692186+0000","last_peered":"2024-03-30T00:33:15.692186+0000","last_clean":"2024-03-30T00:33:15.692186+0000","last_became_active":"2024-03-30T00:33:15.161805+0000","last_became_peered":"2024-03-30T00:33:15.161805+0000","last_unstale":"2024-03-30T00:33:15.692186+0000","last_undegraded":"2024-03-30T00:33:15.692186+0000","last_fullsized":"2024-03-30T00:33:15.692186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:26:16.470245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.532008+0000","last_change":"2024-03-30T00:33:15.675442+0000","last_active":"2024-03-30T00:33:16.532008+0000","last_peered":"2024-03-30T00:33:16.532008+0000","last_clean":"2024-03-30T00:33:16.532008+0000","last_became_active":"2024-03-30T00:33:15.675068+0000","last_became_peered":"2024-03-30T00:33:15.675068+0000","last_unstale":"2024-03-30T00:33:16.532008+0000","last_undegraded":"2024-03-30T00:33:16.532008+0000","last_fullsized":"2024-03-30T00:33:16.532008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:36:31.353262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:55.169272+0000","last_change":"2024-03-30T00:33:15.172087+0000","last_active":"2024-03-30T00:33:55.169272+0000","last_peered":"2024-03-30T00:33:55.169272+0000","last_clean":"2024-03-30T00:33:55.169272+0000","last_became_active":"2024-03-30T00:33:15.171768+0000","last_became_peered":"2024-03-30T00:33:15.171768+0000","last_unstale":"2024-03-30T00:33:55.169272+0000","last_undegraded":"2024-03-30T00:33:55.169272+0000","last_fullsized":"2024-03-30T00:33:55.169272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:17.198995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.668850+0000","last_change":"2024-03-30T00:33:15.665219+0000","last_active":"2024-03-30T00:33:15.668850+0000","last_peered":"2024-03-30T00:33:15.668850+0000","last_clean":"2024-03-30T00:33:15.668850+0000","last_became_active":"2024-03-30T00:33:15.664935+0000","last_became_peered":"2024-03-30T00:33:15.664935+0000","last_unstale":"2024-03-30T00:33:15.668850+0000","last_undegraded":"2024-03-30T00:33:15.668850+0000","last_fullsized":"2024-03-30T00:33:15.668850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:37.123937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.116508+0000","last_change":"2024-03-30T00:33:15.165668+0000","last_active":"2024-03-30T00:33:20.116508+0000","last_peered":"2024-03-30T00:33:20.116508+0000","last_clean":"2024-03-30T00:33:20.116508+0000","last_became_active":"2024-03-30T00:33:15.164637+0000","last_became_peered":"2024-03-30T00:33:15.164637+0000","last_unstale":"2024-03-30T00:33:20.116508+0000","last_undegraded":"2024-03-30T00:33:20.116508+0000","last_fullsized":"2024-03-30T00:33:20.116508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:47:59.389414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531981+0000","last_change":"2024-03-30T00:33:15.662357+0000","last_active":"2024-03-30T00:33:16.531981+0000","last_peered":"2024-03-30T00:33:16.531981+0000","last_clean":"2024-03-30T00:33:16.531981+0000","last_became_active":"2024-03-30T00:33:15.662011+0000","last_became_peered":"2024-03-30T00:33:15.662011+0000","last_unstale":"2024-03-30T00:33:16.531981+0000","last_undegraded":"2024-03-30T00:33:16.531981+0000","last_fullsized":"2024-03-30T00:33:16.531981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:18.386054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118436+0000","last_change":"2024-03-30T00:33:15.177269+0000","last_active":"2024-03-30T00:33:20.118436+0000","last_peered":"2024-03-30T00:33:20.118436+0000","last_clean":"2024-03-30T00:33:20.118436+0000","last_became_active":"2024-03-30T00:33:15.176937+0000","last_became_peered":"2024-03-30T00:33:15.176937+0000","last_unstale":"2024-03-30T00:33:20.118436+0000","last_undegraded":"2024-03-30T00:33:20.118436+0000","last_fullsized":"2024-03-30T00:33:20.118436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:44.658775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000691+0000","last_change":"2024-03-30T00:33:15.675832+0000","last_active":"2024-03-30T00:33:17.000691+0000","last_peered":"2024-03-30T00:33:17.000691+0000","last_clean":"2024-03-30T00:33:17.000691+0000","last_became_active":"2024-03-30T00:33:15.675481+0000","last_became_peered":"2024-03-30T00:33:15.675481+0000","last_unstale":"2024-03-30T00:33:17.000691+0000","last_undegraded":"2024-03-30T00:33:17.000691+0000","last_fullsized":"2024-03-30T00:33:17.000691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:11.638056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001044+0000","last_change":"2024-03-30T00:33:15.171238+0000","last_active":"2024-03-30T00:33:17.001044+0000","last_peered":"2024-03-30T00:33:17.001044+0000","last_clean":"2024-03-30T00:33:17.001044+0000","last_became_active":"2024-03-30T00:33:15.170931+0000","last_became_peered":"2024-03-30T00:33:15.170931+0000","last_unstale":"2024-03-30T00:33:17.001044+0000","last_undegraded":"2024-03-30T00:33:17.001044+0000","last_fullsized":"2024-03-30T00:33:17.001044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:43:50.178363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659336+0000","last_change":"2024-03-30T00:33:15.676014+0000","last_active":"2024-03-30T00:33:16.659336+0000","last_peered":"2024-03-30T00:33:16.659336+0000","last_clean":"2024-03-30T00:33:16.659336+0000","last_became_active":"2024-03-30T00:33:15.675553+0000","last_became_peered":"2024-03-30T00:33:15.675553+0000","last_unstale":"2024-03-30T00:33:16.659336+0000","last_undegraded":"2024-03-30T00:33:16.659336+0000","last_fullsized":"2024-03-30T00:33:16.659336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:17.248242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":51,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:54.166852+0000","last_change":"2024-03-30T00:33:15.177033+0000","last_active":"2024-03-30T00:33:54.166852+0000","last_peered":"2024-03-30T00:33:54.166852+0000","last_clean":"2024-03-30T00:33:54.166852+0000","last_became_active":"2024-03-30T00:33:15.176602+0000","last_became_peered":"2024-03-30T00:33:15.176602+0000","last_unstale":"2024-03-30T00:33:54.166852+0000","last_undegraded":"2024-03-30T00:33:54.166852+0000","last_fullsized":"2024-03-30T00:33:54.166852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:10.804704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.691801+0000","last_change":"2024-03-30T00:33:15.683388+0000","last_active":"2024-03-30T00:33:15.691801+0000","last_peered":"2024-03-30T00:33:15.691801+0000","last_clean":"2024-03-30T00:33:15.691801+0000","last_became_active":"2024-03-30T00:33:15.682976+0000","last_became_peered":"2024-03-30T00:33:15.682976+0000","last_unstale":"2024-03-30T00:33:15.691801+0000","last_undegraded":"2024-03-30T00:33:15.691801+0000","last_fullsized":"2024-03-30T00:33:15.691801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:50:34.907265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131200+0000","last_change":"2024-03-30T00:33:15.167819+0000","last_active":"2024-03-30T00:33:19.131200+0000","last_peered":"2024-03-30T00:33:19.131200+0000","last_clean":"2024-03-30T00:33:19.131200+0000","last_became_active":"2024-03-30T00:33:15.167405+0000","last_became_peered":"2024-03-30T00:33:15.167405+0000","last_unstale":"2024-03-30T00:33:19.131200+0000","last_undegraded":"2024-03-30T00:33:19.131200+0000","last_fullsized":"2024-03-30T00:33:19.131200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:37.793792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002345+0000","last_change":"2024-03-30T00:33:15.690907+0000","last_active":"2024-03-30T00:33:17.002345+0000","last_peered":"2024-03-30T00:33:17.002345+0000","last_clean":"2024-03-30T00:33:17.002345+0000","last_became_active":"2024-03-30T00:33:15.690046+0000","last_became_peered":"2024-03-30T00:33:15.690046+0000","last_unstale":"2024-03-30T00:33:17.002345+0000","last_undegraded":"2024-03-30T00:33:17.002345+0000","last_fullsized":"2024-03-30T00:33:17.002345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:24.319685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.132456+0000","last_change":"2024-03-30T00:33:15.166633+0000","last_active":"2024-03-30T00:33:19.132456+0000","last_peered":"2024-03-30T00:33:19.132456+0000","last_clean":"2024-03-30T00:33:19.132456+0000","last_became_active":"2024-03-30T00:33:15.166308+0000","last_became_peered":"2024-03-30T00:33:15.166308+0000","last_unstale":"2024-03-30T00:33:19.132456+0000","last_undegraded":"2024-03-30T00:33:19.132456+0000","last_fullsized":"2024-03-30T00:33:19.132456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:53.102872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693949+0000","last_change":"2024-03-30T00:33:15.691880+0000","last_active":"2024-03-30T00:33:15.693949+0000","last_peered":"2024-03-30T00:33:15.693949+0000","last_clean":"2024-03-30T00:33:15.693949+0000","last_became_active":"2024-03-30T00:33:15.690780+0000","last_became_peered":"2024-03-30T00:33:15.690780+0000","last_unstale":"2024-03-30T00:33:15.693949+0000","last_undegraded":"2024-03-30T00:33:15.693949+0000","last_fullsized":"2024-03-30T00:33:15.693949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:06.532709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'32","reported_seq":125,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:55.180654+0000","last_change":"2024-03-30T00:33:15.164287+0000","last_active":"2024-03-30T00:33:55.180654+0000","last_peered":"2024-03-30T00:33:55.180654+0000","last_clean":"2024-03-30T00:33:55.180654+0000","last_became_active":"2024-03-30T00:33:15.163557+0000","last_became_peered":"2024-03-30T00:33:15.163557+0000","last_unstale":"2024-03-30T00:33:55.180654+0000","last_undegraded":"2024-03-30T00:33:55.180654+0000","last_fullsized":"2024-03-30T00:33:55.180654+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:41.117936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62619,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":81,"num_read_kb":101,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.683827+0000","last_change":"2024-03-30T00:33:15.682841+0000","last_active":"2024-03-30T00:33:15.683827+0000","last_peered":"2024-03-30T00:33:15.683827+0000","last_clean":"2024-03-30T00:33:15.683827+0000","last_became_active":"2024-03-30T00:33:15.682237+0000","last_became_peered":"2024-03-30T00:33:15.682237+0000","last_unstale":"2024-03-30T00:33:15.683827+0000","last_undegraded":"2024-03-30T00:33:15.683827+0000","last_fullsized":"2024-03-30T00:33:15.683827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:59:05.719216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118639+0000","last_change":"2024-03-30T00:33:15.165193+0000","last_active":"2024-03-30T00:33:20.118639+0000","last_peered":"2024-03-30T00:33:20.118639+0000","last_clean":"2024-03-30T00:33:20.118639+0000","last_became_active":"2024-03-30T00:33:15.161741+0000","last_became_peered":"2024-03-30T00:33:15.161741+0000","last_unstale":"2024-03-30T00:33:20.118639+0000","last_undegraded":"2024-03-30T00:33:20.118639+0000","last_fullsized":"2024-03-30T00:33:20.118639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:22:11.891064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001296+0000","last_change":"2024-03-30T00:33:15.680929+0000","last_active":"2024-03-30T00:33:17.001296+0000","last_peered":"2024-03-30T00:33:17.001296+0000","last_clean":"2024-03-30T00:33:17.001296+0000","last_became_active":"2024-03-30T00:33:15.676671+0000","last_became_peered":"2024-03-30T00:33:15.676671+0000","last_unstale":"2024-03-30T00:33:17.001296+0000","last_undegraded":"2024-03-30T00:33:17.001296+0000","last_fullsized":"2024-03-30T00:33:17.001296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:28:33.819135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.132787+0000","last_change":"2024-03-30T00:33:15.168029+0000","last_active":"2024-03-30T00:33:19.132787+0000","last_peered":"2024-03-30T00:33:19.132787+0000","last_clean":"2024-03-30T00:33:19.132787+0000","last_became_active":"2024-03-30T00:33:15.167636+0000","last_became_peered":"2024-03-30T00:33:15.167636+0000","last_unstale":"2024-03-30T00:33:19.132787+0000","last_undegraded":"2024-03-30T00:33:19.132787+0000","last_fullsized":"2024-03-30T00:33:19.132787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:25:47.834696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001171+0000","last_change":"2024-03-30T00:33:15.687977+0000","last_active":"2024-03-30T00:33:17.001171+0000","last_peered":"2024-03-30T00:33:17.001171+0000","last_clean":"2024-03-30T00:33:17.001171+0000","last_became_active":"2024-03-30T00:33:15.687491+0000","last_became_peered":"2024-03-30T00:33:15.687491+0000","last_unstale":"2024-03-30T00:33:17.001171+0000","last_undegraded":"2024-03-30T00:33:17.001171+0000","last_fullsized":"2024-03-30T00:33:17.001171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:08:55.568039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":51,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:53.186679+0000","last_change":"2024-03-30T00:33:15.171826+0000","last_active":"2024-03-30T00:33:53.186679+0000","last_peered":"2024-03-30T00:33:53.186679+0000","last_clean":"2024-03-30T00:33:53.186679+0000","last_became_active":"2024-03-30T00:33:15.170749+0000","last_became_peered":"2024-03-30T00:33:15.170749+0000","last_unstale":"2024-03-30T00:33:53.186679+0000","last_undegraded":"2024-03-30T00:33:53.186679+0000","last_fullsized":"2024-03-30T00:33:53.186679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:57:52.034530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000714+0000","last_change":"2024-03-30T00:33:15.681668+0000","last_active":"2024-03-30T00:33:17.000714+0000","last_peered":"2024-03-30T00:33:17.000714+0000","last_clean":"2024-03-30T00:33:17.000714+0000","last_became_active":"2024-03-30T00:33:15.681129+0000","last_became_peered":"2024-03-30T00:33:15.681129+0000","last_unstale":"2024-03-30T00:33:17.000714+0000","last_undegraded":"2024-03-30T00:33:17.000714+0000","last_fullsized":"2024-03-30T00:33:17.000714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:51:19.552243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118932+0000","last_change":"2024-03-30T00:33:15.171016+0000","last_active":"2024-03-30T00:33:20.118932+0000","last_peered":"2024-03-30T00:33:20.118932+0000","last_clean":"2024-03-30T00:33:20.118932+0000","last_became_active":"2024-03-30T00:33:15.170058+0000","last_became_peered":"2024-03-30T00:33:15.170058+0000","last_unstale":"2024-03-30T00:33:20.118932+0000","last_undegraded":"2024-03-30T00:33:20.118932+0000","last_fullsized":"2024-03-30T00:33:20.118932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:07:39.910286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531944+0000","last_change":"2024-03-30T00:33:15.687012+0000","last_active":"2024-03-30T00:33:16.531944+0000","last_peered":"2024-03-30T00:33:16.531944+0000","last_clean":"2024-03-30T00:33:16.531944+0000","last_became_active":"2024-03-30T00:33:15.684030+0000","last_became_peered":"2024-03-30T00:33:15.684030+0000","last_unstale":"2024-03-30T00:33:16.531944+0000","last_undegraded":"2024-03-30T00:33:16.531944+0000","last_fullsized":"2024-03-30T00:33:16.531944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:32.447119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.103201+0000","last_change":"2024-03-30T00:33:15.163633+0000","last_active":"2024-03-30T00:33:20.103201+0000","last_peered":"2024-03-30T00:33:20.103201+0000","last_clean":"2024-03-30T00:33:20.103201+0000","last_became_active":"2024-03-30T00:33:15.161339+0000","last_became_peered":"2024-03-30T00:33:15.161339+0000","last_unstale":"2024-03-30T00:33:20.103201+0000","last_undegraded":"2024-03-30T00:33:20.103201+0000","last_fullsized":"2024-03-30T00:33:20.103201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:15:25.194650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290780+0000","last_change":"2024-03-30T00:33:15.680617+0000","last_active":"2024-03-30T00:33:17.290780+0000","last_peered":"2024-03-30T00:33:17.290780+0000","last_clean":"2024-03-30T00:33:17.290780+0000","last_became_active":"2024-03-30T00:33:15.679137+0000","last_became_peered":"2024-03-30T00:33:15.679137+0000","last_unstale":"2024-03-30T00:33:17.290780+0000","last_undegraded":"2024-03-30T00:33:17.290780+0000","last_fullsized":"2024-03-30T00:33:17.290780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:34.449218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659518+0000","last_change":"2024-03-30T00:33:15.177273+0000","last_active":"2024-03-30T00:33:16.659518+0000","last_peered":"2024-03-30T00:33:16.659518+0000","last_clean":"2024-03-30T00:33:16.659518+0000","last_became_active":"2024-03-30T00:33:15.175333+0000","last_became_peered":"2024-03-30T00:33:15.175333+0000","last_unstale":"2024-03-30T00:33:16.659518+0000","last_undegraded":"2024-03-30T00:33:16.659518+0000","last_fullsized":"2024-03-30T00:33:16.659518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:27:03.305671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002883+0000","last_change":"2024-03-30T00:33:15.674746+0000","last_active":"2024-03-30T00:33:17.002883+0000","last_peered":"2024-03-30T00:33:17.002883+0000","last_clean":"2024-03-30T00:33:17.002883+0000","last_became_active":"2024-03-30T00:33:15.674388+0000","last_became_peered":"2024-03-30T00:33:15.674388+0000","last_unstale":"2024-03-30T00:33:17.002883+0000","last_undegraded":"2024-03-30T00:33:17.002883+0000","last_fullsized":"2024-03-30T00:33:17.002883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:21:15.024181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.054408+0000","last_change":"2024-03-30T00:33:15.173640+0000","last_active":"2024-03-30T00:33:19.054408+0000","last_peered":"2024-03-30T00:33:19.054408+0000","last_clean":"2024-03-30T00:33:19.054408+0000","last_became_active":"2024-03-30T00:33:15.173104+0000","last_became_peered":"2024-03-30T00:33:15.173104+0000","last_unstale":"2024-03-30T00:33:19.054408+0000","last_undegraded":"2024-03-30T00:33:19.054408+0000","last_fullsized":"2024-03-30T00:33:19.054408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:20:14.880730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.683575+0000","last_change":"2024-03-30T00:33:15.677824+0000","last_active":"2024-03-30T00:33:15.683575+0000","last_peered":"2024-03-30T00:33:15.683575+0000","last_clean":"2024-03-30T00:33:15.683575+0000","last_became_active":"2024-03-30T00:33:15.677435+0000","last_became_peered":"2024-03-30T00:33:15.677435+0000","last_unstale":"2024-03-30T00:33:15.683575+0000","last_undegraded":"2024-03-30T00:33:15.683575+0000","last_fullsized":"2024-03-30T00:33:15.683575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:22.992989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531931+0000","last_change":"2024-03-30T00:33:15.667428+0000","last_active":"2024-03-30T00:33:16.531931+0000","last_peered":"2024-03-30T00:33:16.531931+0000","last_clean":"2024-03-30T00:33:16.531931+0000","last_became_active":"2024-03-30T00:33:15.666634+0000","last_became_peered":"2024-03-30T00:33:15.666634+0000","last_unstale":"2024-03-30T00:33:16.531931+0000","last_undegraded":"2024-03-30T00:33:16.531931+0000","last_fullsized":"2024-03-30T00:33:16.531931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:56:14.710178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:56.191040+0000","last_change":"2024-03-30T00:33:15.173264+0000","last_active":"2024-03-30T00:33:56.191040+0000","last_peered":"2024-03-30T00:33:56.191040+0000","last_clean":"2024-03-30T00:33:56.191040+0000","last_became_active":"2024-03-30T00:33:15.172932+0000","last_became_peered":"2024-03-30T00:33:15.172932+0000","last_unstale":"2024-03-30T00:33:56.191040+0000","last_undegraded":"2024-03-30T00:33:56.191040+0000","last_fullsized":"2024-03-30T00:33:56.191040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:18:25.079284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000756+0000","last_change":"2024-03-30T00:33:15.686606+0000","last_active":"2024-03-30T00:33:17.000756+0000","last_peered":"2024-03-30T00:33:17.000756+0000","last_clean":"2024-03-30T00:33:17.000756+0000","last_became_active":"2024-03-30T00:33:15.686076+0000","last_became_peered":"2024-03-30T00:33:15.686076+0000","last_unstale":"2024-03-30T00:33:17.000756+0000","last_undegraded":"2024-03-30T00:33:17.000756+0000","last_fullsized":"2024-03-30T00:33:17.000756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:46.962659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.119040+0000","last_change":"2024-03-30T00:33:15.177263+0000","last_active":"2024-03-30T00:33:20.119040+0000","last_peered":"2024-03-30T00:33:20.119040+0000","last_clean":"2024-03-30T00:33:20.119040+0000","last_became_active":"2024-03-30T00:33:15.176743+0000","last_became_peered":"2024-03-30T00:33:15.176743+0000","last_unstale":"2024-03-30T00:33:20.119040+0000","last_undegraded":"2024-03-30T00:33:20.119040+0000","last_fullsized":"2024-03-30T00:33:20.119040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:56:47.491371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":322,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.669557+0000","last_change":"2024-03-30T00:32:12.678730+0000","last_active":"2024-03-30T00:33:15.669557+0000","last_peered":"2024-03-30T00:33:15.669557+0000","last_clean":"2024-03-30T00:33:15.669557+0000","last_became_active":"2024-03-30T00:32:12.678256+0000","last_became_peered":"2024-03-30T00:32:12.678256+0000","last_unstale":"2024-03-30T00:33:15.669557+0000","last_undegraded":"2024-03-30T00:33:15.669557+0000","last_fullsized":"2024-03-30T00:33:15.669557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_clean_scrub_stamp":"2024-03-30T00:29:17.768535+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:16.229072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000775+0000","last_change":"2024-03-30T00:33:15.685230+0000","last_active":"2024-03-30T00:33:17.000775+0000","last_peered":"2024-03-30T00:33:17.000775+0000","last_clean":"2024-03-30T00:33:17.000775+0000","last_became_active":"2024-03-30T00:33:15.684263+0000","last_became_peered":"2024-03-30T00:33:15.684263+0000","last_unstale":"2024-03-30T00:33:17.000775+0000","last_undegraded":"2024-03-30T00:33:17.000775+0000","last_fullsized":"2024-03-30T00:33:17.000775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:22:51.980318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.115562+0000","last_change":"2024-03-30T00:33:15.167490+0000","last_active":"2024-03-30T00:33:20.115562+0000","last_peered":"2024-03-30T00:33:20.115562+0000","last_clean":"2024-03-30T00:33:20.115562+0000","last_became_active":"2024-03-30T00:33:15.166888+0000","last_became_peered":"2024-03-30T00:33:15.166888+0000","last_unstale":"2024-03-30T00:33:20.115562+0000","last_undegraded":"2024-03-30T00:33:20.115562+0000","last_fullsized":"2024-03-30T00:33:20.115562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:00:32.571281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131306+0000","last_change":"2024-03-30T00:33:15.178388+0000","last_active":"2024-03-30T00:33:19.131306+0000","last_peered":"2024-03-30T00:33:19.131306+0000","last_clean":"2024-03-30T00:33:19.131306+0000","last_became_active":"2024-03-30T00:33:15.178160+0000","last_became_peered":"2024-03-30T00:33:15.178160+0000","last_unstale":"2024-03-30T00:33:19.131306+0000","last_undegraded":"2024-03-30T00:33:19.131306+0000","last_fullsized":"2024-03-30T00:33:19.131306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:28:51.567598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001200+0000","last_change":"2024-03-30T00:33:15.675426+0000","last_active":"2024-03-30T00:33:17.001200+0000","last_peered":"2024-03-30T00:33:17.001200+0000","last_clean":"2024-03-30T00:33:17.001200+0000","last_became_active":"2024-03-30T00:33:15.675083+0000","last_became_peered":"2024-03-30T00:33:15.675083+0000","last_unstale":"2024-03-30T00:33:17.001200+0000","last_undegraded":"2024-03-30T00:33:17.001200+0000","last_fullsized":"2024-03-30T00:33:17.001200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:37:03.086331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'6","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:28.599500+0000","last_change":"2024-03-30T00:33:15.171537+0000","last_active":"2024-03-30T00:33:28.599500+0000","last_peered":"2024-03-30T00:33:28.599500+0000","last_clean":"2024-03-30T00:33:28.599500+0000","last_became_active":"2024-03-30T00:33:15.171128+0000","last_became_peered":"2024-03-30T00:33:15.171128+0000","last_unstale":"2024-03-30T00:33:28.599500+0000","last_undegraded":"2024-03-30T00:33:28.599500+0000","last_fullsized":"2024-03-30T00:33:28.599500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:52.062144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001730+0000","last_change":"2024-03-30T00:33:15.674075+0000","last_active":"2024-03-30T00:33:17.001730+0000","last_peered":"2024-03-30T00:33:17.001730+0000","last_clean":"2024-03-30T00:33:17.001730+0000","last_became_active":"2024-03-30T00:33:15.672400+0000","last_became_peered":"2024-03-30T00:33:15.672400+0000","last_unstale":"2024-03-30T00:33:17.001730+0000","last_undegraded":"2024-03-30T00:33:17.001730+0000","last_fullsized":"2024-03-30T00:33:17.001730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:59.309835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131339+0000","last_change":"2024-03-30T00:33:15.168245+0000","last_active":"2024-03-30T00:33:19.131339+0000","last_peered":"2024-03-30T00:33:19.131339+0000","last_clean":"2024-03-30T00:33:19.131339+0000","last_became_active":"2024-03-30T00:33:15.167852+0000","last_became_peered":"2024-03-30T00:33:15.167852+0000","last_unstale":"2024-03-30T00:33:19.131339+0000","last_undegraded":"2024-03-30T00:33:19.131339+0000","last_fullsized":"2024-03-30T00:33:19.131339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:33:28.780614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002270+0000","last_change":"2024-03-30T00:33:15.679802+0000","last_active":"2024-03-30T00:33:17.002270+0000","last_peered":"2024-03-30T00:33:17.002270+0000","last_clean":"2024-03-30T00:33:17.002270+0000","last_became_active":"2024-03-30T00:33:15.679468+0000","last_became_peered":"2024-03-30T00:33:15.679468+0000","last_unstale":"2024-03-30T00:33:17.002270+0000","last_undegraded":"2024-03-30T00:33:17.002270+0000","last_fullsized":"2024-03-30T00:33:17.002270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:03:03.223713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131563+0000","last_change":"2024-03-30T00:33:15.170052+0000","last_active":"2024-03-30T00:33:19.131563+0000","last_peered":"2024-03-30T00:33:19.131563+0000","last_clean":"2024-03-30T00:33:19.131563+0000","last_became_active":"2024-03-30T00:33:15.169706+0000","last_became_peered":"2024-03-30T00:33:15.169706+0000","last_unstale":"2024-03-30T00:33:19.131563+0000","last_undegraded":"2024-03-30T00:33:19.131563+0000","last_fullsized":"2024-03-30T00:33:19.131563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:32.043071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.668729+0000","last_change":"2024-03-30T00:33:15.667061+0000","last_active":"2024-03-30T00:33:15.668729+0000","last_peered":"2024-03-30T00:33:15.668729+0000","last_clean":"2024-03-30T00:33:15.668729+0000","last_became_active":"2024-03-30T00:33:15.666763+0000","last_became_peered":"2024-03-30T00:33:15.666763+0000","last_unstale":"2024-03-30T00:33:15.668729+0000","last_undegraded":"2024-03-30T00:33:15.668729+0000","last_fullsized":"2024-03-30T00:33:15.668729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:46.558297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.133165+0000","last_change":"2024-03-30T00:33:15.176152+0000","last_active":"2024-03-30T00:33:19.133165+0000","last_peered":"2024-03-30T00:33:19.133165+0000","last_clean":"2024-03-30T00:33:19.133165+0000","last_became_active":"2024-03-30T00:33:15.175640+0000","last_became_peered":"2024-03-30T00:33:15.175640+0000","last_unstale":"2024-03-30T00:33:19.133165+0000","last_undegraded":"2024-03-30T00:33:19.133165+0000","last_fullsized":"2024-03-30T00:33:19.133165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:58:11.575577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290545+0000","last_change":"2024-03-30T00:33:15.672957+0000","last_active":"2024-03-30T00:33:17.290545+0000","last_peered":"2024-03-30T00:33:17.290545+0000","last_clean":"2024-03-30T00:33:17.290545+0000","last_became_active":"2024-03-30T00:33:15.668868+0000","last_became_peered":"2024-03-30T00:33:15.668868+0000","last_unstale":"2024-03-30T00:33:17.290545+0000","last_undegraded":"2024-03-30T00:33:17.290545+0000","last_fullsized":"2024-03-30T00:33:17.290545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:50.342693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.787067+0000","last_change":"2024-03-30T00:33:15.160345+0000","last_active":"2024-03-30T00:33:16.787067+0000","last_peered":"2024-03-30T00:33:16.787067+0000","last_clean":"2024-03-30T00:33:16.787067+0000","last_became_active":"2024-03-30T00:33:15.160026+0000","last_became_peered":"2024-03-30T00:33:15.160026+0000","last_unstale":"2024-03-30T00:33:16.787067+0000","last_undegraded":"2024-03-30T00:33:16.787067+0000","last_fullsized":"2024-03-30T00:33:16.787067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:06:51.177680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001362+0000","last_change":"2024-03-30T00:33:15.674058+0000","last_active":"2024-03-30T00:33:17.001362+0000","last_peered":"2024-03-30T00:33:17.001362+0000","last_clean":"2024-03-30T00:33:17.001362+0000","last_became_active":"2024-03-30T00:33:15.673165+0000","last_became_peered":"2024-03-30T00:33:15.673165+0000","last_unstale":"2024-03-30T00:33:17.001362+0000","last_undegraded":"2024-03-30T00:33:17.001362+0000","last_fullsized":"2024-03-30T00:33:17.001362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:14.869963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131675+0000","last_change":"2024-03-30T00:33:15.173178+0000","last_active":"2024-03-30T00:33:19.131675+0000","last_peered":"2024-03-30T00:33:19.131675+0000","last_clean":"2024-03-30T00:33:19.131675+0000","last_became_active":"2024-03-30T00:33:15.172830+0000","last_became_peered":"2024-03-30T00:33:15.172830+0000","last_unstale":"2024-03-30T00:33:19.131675+0000","last_undegraded":"2024-03-30T00:33:19.131675+0000","last_fullsized":"2024-03-30T00:33:19.131675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:49:09.823855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001435+0000","last_change":"2024-03-30T00:33:15.693485+0000","last_active":"2024-03-30T00:33:17.001435+0000","last_peered":"2024-03-30T00:33:17.001435+0000","last_clean":"2024-03-30T00:33:17.001435+0000","last_became_active":"2024-03-30T00:33:15.693066+0000","last_became_peered":"2024-03-30T00:33:15.693066+0000","last_unstale":"2024-03-30T00:33:17.001435+0000","last_undegraded":"2024-03-30T00:33:17.001435+0000","last_fullsized":"2024-03-30T00:33:17.001435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:38.161331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118339+0000","last_change":"2024-03-30T00:33:15.170808+0000","last_active":"2024-03-30T00:33:20.118339+0000","last_peered":"2024-03-30T00:33:20.118339+0000","last_clean":"2024-03-30T00:33:20.118339+0000","last_became_active":"2024-03-30T00:33:15.170448+0000","last_became_peered":"2024-03-30T00:33:15.170448+0000","last_unstale":"2024-03-30T00:33:20.118339+0000","last_undegraded":"2024-03-30T00:33:20.118339+0000","last_fullsized":"2024-03-30T00:33:20.118339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:08:28.179041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001385+0000","last_change":"2024-03-30T00:33:15.687619+0000","last_active":"2024-03-30T00:33:17.001385+0000","last_peered":"2024-03-30T00:33:17.001385+0000","last_clean":"2024-03-30T00:33:17.001385+0000","last_became_active":"2024-03-30T00:33:15.687313+0000","last_became_peered":"2024-03-30T00:33:15.687313+0000","last_unstale":"2024-03-30T00:33:17.001385+0000","last_undegraded":"2024-03-30T00:33:17.001385+0000","last_fullsized":"2024-03-30T00:33:17.001385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:36:51.182558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001678+0000","last_change":"2024-03-30T00:33:15.168601+0000","last_active":"2024-03-30T00:33:17.001678+0000","last_peered":"2024-03-30T00:33:17.001678+0000","last_clean":"2024-03-30T00:33:17.001678+0000","last_became_active":"2024-03-30T00:33:15.168065+0000","last_became_peered":"2024-03-30T00:33:15.168065+0000","last_unstale":"2024-03-30T00:33:17.001678+0000","last_undegraded":"2024-03-30T00:33:17.001678+0000","last_fullsized":"2024-03-30T00:33:17.001678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:22:36.477280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.054439+0000","last_change":"2024-03-30T00:33:15.678299+0000","last_active":"2024-03-30T00:33:19.054439+0000","last_peered":"2024-03-30T00:33:19.054439+0000","last_clean":"2024-03-30T00:33:19.054439+0000","last_became_active":"2024-03-30T00:33:15.675499+0000","last_became_peered":"2024-03-30T00:33:15.675499+0000","last_unstale":"2024-03-30T00:33:19.054439+0000","last_undegraded":"2024-03-30T00:33:19.054439+0000","last_fullsized":"2024-03-30T00:33:19.054439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:15.317125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000985+0000","last_change":"2024-03-30T00:33:15.172058+0000","last_active":"2024-03-30T00:33:17.000985+0000","last_peered":"2024-03-30T00:33:17.000985+0000","last_clean":"2024-03-30T00:33:17.000985+0000","last_became_active":"2024-03-30T00:33:15.167857+0000","last_became_peered":"2024-03-30T00:33:15.167857+0000","last_unstale":"2024-03-30T00:33:17.000985+0000","last_undegraded":"2024-03-30T00:33:17.000985+0000","last_fullsized":"2024-03-30T00:33:17.000985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:03:35.951559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001350+0000","last_change":"2024-03-30T00:33:15.682519+0000","last_active":"2024-03-30T00:33:17.001350+0000","last_peered":"2024-03-30T00:33:17.001350+0000","last_clean":"2024-03-30T00:33:17.001350+0000","last_became_active":"2024-03-30T00:33:15.681055+0000","last_became_peered":"2024-03-30T00:33:15.681055+0000","last_unstale":"2024-03-30T00:33:17.001350+0000","last_undegraded":"2024-03-30T00:33:17.001350+0000","last_fullsized":"2024-03-30T00:33:17.001350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:38.160151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290650+0000","last_change":"2024-03-30T00:33:15.673411+0000","last_active":"2024-03-30T00:33:17.290650+0000","last_peered":"2024-03-30T00:33:17.290650+0000","last_clean":"2024-03-30T00:33:17.290650+0000","last_became_active":"2024-03-30T00:33:15.672852+0000","last_became_peered":"2024-03-30T00:33:15.672852+0000","last_unstale":"2024-03-30T00:33:17.290650+0000","last_undegraded":"2024-03-30T00:33:17.290650+0000","last_fullsized":"2024-03-30T00:33:17.290650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:55:46.705785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660735+0000","last_change":"2024-03-30T00:33:15.680624+0000","last_active":"2024-03-30T00:33:16.660735+0000","last_peered":"2024-03-30T00:33:16.660735+0000","last_clean":"2024-03-30T00:33:16.660735+0000","last_became_active":"2024-03-30T00:33:15.679900+0000","last_became_peered":"2024-03-30T00:33:15.679900+0000","last_unstale":"2024-03-30T00:33:16.660735+0000","last_undegraded":"2024-03-30T00:33:16.660735+0000","last_fullsized":"2024-03-30T00:33:16.660735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:44.484608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660521+0000","last_change":"2024-03-30T00:33:15.672612+0000","last_active":"2024-03-30T00:33:16.660521+0000","last_peered":"2024-03-30T00:33:16.660521+0000","last_clean":"2024-03-30T00:33:16.660521+0000","last_became_active":"2024-03-30T00:33:15.671536+0000","last_became_peered":"2024-03-30T00:33:15.671536+0000","last_unstale":"2024-03-30T00:33:16.660521+0000","last_undegraded":"2024-03-30T00:33:16.660521+0000","last_fullsized":"2024-03-30T00:33:16.660521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:40.783988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002065+0000","last_change":"2024-03-30T00:33:15.681577+0000","last_active":"2024-03-30T00:33:17.002065+0000","last_peered":"2024-03-30T00:33:17.002065+0000","last_clean":"2024-03-30T00:33:17.002065+0000","last_became_active":"2024-03-30T00:33:15.681313+0000","last_became_peered":"2024-03-30T00:33:15.681313+0000","last_unstale":"2024-03-30T00:33:17.002065+0000","last_undegraded":"2024-03-30T00:33:17.002065+0000","last_fullsized":"2024-03-30T00:33:17.002065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:42:15.178957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001296+0000","last_change":"2024-03-30T00:33:15.688172+0000","last_active":"2024-03-30T00:33:17.001296+0000","last_peered":"2024-03-30T00:33:17.001296+0000","last_clean":"2024-03-30T00:33:17.001296+0000","last_became_active":"2024-03-30T00:33:15.687860+0000","last_became_peered":"2024-03-30T00:33:15.687860+0000","last_unstale":"2024-03-30T00:33:17.001296+0000","last_undegraded":"2024-03-30T00:33:17.001296+0000","last_fullsized":"2024-03-30T00:33:17.001296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:28.762798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290827+0000","last_change":"2024-03-30T00:33:15.673868+0000","last_active":"2024-03-30T00:33:17.290827+0000","last_peered":"2024-03-30T00:33:17.290827+0000","last_clean":"2024-03-30T00:33:17.290827+0000","last_became_active":"2024-03-30T00:33:15.673467+0000","last_became_peered":"2024-03-30T00:33:15.673467+0000","last_unstale":"2024-03-30T00:33:17.290827+0000","last_undegraded":"2024-03-30T00:33:17.290827+0000","last_fullsized":"2024-03-30T00:33:17.290827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:07:39.448967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.661014+0000","last_change":"2024-03-30T00:33:15.684639+0000","last_active":"2024-03-30T00:33:16.661014+0000","last_peered":"2024-03-30T00:33:16.661014+0000","last_clean":"2024-03-30T00:33:16.661014+0000","last_became_active":"2024-03-30T00:33:15.684357+0000","last_became_peered":"2024-03-30T00:33:15.684357+0000","last_unstale":"2024-03-30T00:33:16.661014+0000","last_undegraded":"2024-03-30T00:33:16.661014+0000","last_fullsized":"2024-03-30T00:33:16.661014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:02:50.995640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001730+0000","last_change":"2024-03-30T00:33:15.680810+0000","last_active":"2024-03-30T00:33:17.001730+0000","last_peered":"2024-03-30T00:33:17.001730+0000","last_clean":"2024-03-30T00:33:17.001730+0000","last_became_active":"2024-03-30T00:33:15.680457+0000","last_became_peered":"2024-03-30T00:33:15.680457+0000","last_unstale":"2024-03-30T00:33:17.001730+0000","last_undegraded":"2024-03-30T00:33:17.001730+0000","last_fullsized":"2024-03-30T00:33:17.001730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:02.575683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531880+0000","last_change":"2024-03-30T00:33:15.661644+0000","last_active":"2024-03-30T00:33:16.531880+0000","last_peered":"2024-03-30T00:33:16.531880+0000","last_clean":"2024-03-30T00:33:16.531880+0000","last_became_active":"2024-03-30T00:33:15.661359+0000","last_became_peered":"2024-03-30T00:33:15.661359+0000","last_unstale":"2024-03-30T00:33:16.531880+0000","last_undegraded":"2024-03-30T00:33:16.531880+0000","last_fullsized":"2024-03-30T00:33:16.531880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:44:39.894342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000646+0000","last_change":"2024-03-30T00:33:15.671491+0000","last_active":"2024-03-30T00:33:17.000646+0000","last_peered":"2024-03-30T00:33:17.000646+0000","last_clean":"2024-03-30T00:33:17.000646+0000","last_became_active":"2024-03-30T00:33:15.671110+0000","last_became_peered":"2024-03-30T00:33:15.671110+0000","last_unstale":"2024-03-30T00:33:17.000646+0000","last_undegraded":"2024-03-30T00:33:17.000646+0000","last_fullsized":"2024-03-30T00:33:17.000646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:57.623730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693121+0000","last_change":"2024-03-30T00:33:15.686795+0000","last_active":"2024-03-30T00:33:15.693121+0000","last_peered":"2024-03-30T00:33:15.693121+0000","last_clean":"2024-03-30T00:33:15.693121+0000","last_became_active":"2024-03-30T00:33:15.685715+0000","last_became_peered":"2024-03-30T00:33:15.685715+0000","last_unstale":"2024-03-30T00:33:15.693121+0000","last_undegraded":"2024-03-30T00:33:15.693121+0000","last_fullsized":"2024-03-30T00:33:15.693121+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:37:53.838823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001008+0000","last_change":"2024-03-30T00:33:15.687369+0000","last_active":"2024-03-30T00:33:17.001008+0000","last_peered":"2024-03-30T00:33:17.001008+0000","last_clean":"2024-03-30T00:33:17.001008+0000","last_became_active":"2024-03-30T00:33:15.686852+0000","last_became_peered":"2024-03-30T00:33:15.686852+0000","last_unstale":"2024-03-30T00:33:17.001008+0000","last_undegraded":"2024-03-30T00:33:17.001008+0000","last_fullsized":"2024-03-30T00:33:17.001008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:09:21.003739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290614+0000","last_change":"2024-03-30T00:33:15.676840+0000","last_active":"2024-03-30T00:33:17.290614+0000","last_peered":"2024-03-30T00:33:17.290614+0000","last_clean":"2024-03-30T00:33:17.290614+0000","last_became_active":"2024-03-30T00:33:15.675884+0000","last_became_peered":"2024-03-30T00:33:15.675884+0000","last_unstale":"2024-03-30T00:33:17.290614+0000","last_undegraded":"2024-03-30T00:33:17.290614+0000","last_fullsized":"2024-03-30T00:33:17.290614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:02.900843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66133,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":243,"num_read_kb":192,"num_write":156,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":198399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449109,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":901120,"data_stored":429559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.69099999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.81699999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.878}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":6.6029999999999998,"5min":6.6029999999999998,"15min":6.6029999999999998},"last":0.77400000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.432},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.75600000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.73999999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":9,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":6.6399999999999997,"5min":6.6399999999999997,"15min":6.6399999999999997},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.91500000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.85299999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678041,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1083330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.76700000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":1}]},{"osd":3,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.80900000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.83899999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":1.0209999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.86399999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.97999999999999998}]},{"osd":8,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.038}]},{"osd":9,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.90600000000000003}]},{"osd":11,"last update":"Sat Mar 30 00:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":0.92700000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841564,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":429795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71299999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.38300000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.77800000000000002,"15min":0.78900000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0329999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.622},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.82299999999999995,"15min":0.83799999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.018,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.754}]},{"osd":2,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82799999999999996,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.82399999999999995}]},{"osd":3,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.95399999999999996,"5min":1.278,"15min":1.278},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.91800000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.504}]},{"osd":4,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.86499999999999999,"15min":0.88800000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98499999999999999,"5min":1.476,"15min":1.476},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.85699999999999998,"15min":0.878},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92900000000000005,"5min":1.254,"15min":1.254},"last":0.80400000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.87,"15min":0.88600000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.89600000000000002,"15min":0.92200000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94799999999999995,"5min":1.577,"15min":1.577},"last":0.95899999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.86899999999999999,"15min":0.89500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0600000000000001,"5min":1.486,"15min":1.486},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.90200000000000002,"15min":0.92900000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.073,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.93899999999999995,"15min":0.96299999999999997},"min":{"1min":0.53100000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0669999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.028,"15min":1.0620000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.083,"5min":2.234,"15min":2.234},"last":0.878}]},{"osd":8,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.998,"15min":1.024},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.095,"5min":1.611,"15min":1.611},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.0369999999999999,"15min":1.075},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.998,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.998}]},{"osd":10,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.93100000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":429715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81799999999999995,"5min":0.88,"15min":0.88},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78800000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.51900000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":1.0109999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.38900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.72799999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.67100000000000004,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.876,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.67300000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.59799999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.83399999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76500000000000001,"15min":0.76600000000000001},"min":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0640000000000001,"5min":1.246,"15min":1.246},"last":0.81899999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.026,"15min":1.026},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.80000000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.80300000000000005,"15min":0.81799999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89100000000000001,"5min":1.054,"15min":1.054},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96699999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.88200000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.878,"15min":0.88900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.04,"5min":1.155,"15min":1.155},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.84299999999999997,"15min":0.84699999999999998},"min":{"1min":0.627,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.99099999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.95499999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.86299999999999999,"15min":0.84199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.8969999999999998,"5min":3.8969999999999998,"15min":3.8969999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.8250000000000002,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":0.93400000000000005}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.97199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1021158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.61299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.83499999999999996}]},{"osd":3,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.74299999999999999,"15min":0.73199999999999998},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.71999999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.67100000000000004,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.45000000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.56499999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0760000000000001,"5min":1.095,"15min":1.095},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.79600000000000004,"15min":0.79100000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1679999999999999,"5min":1.252,"15min":1.252},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.79900000000000004,"15min":0.79400000000000004},"min":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.54300000000000004}]},{"osd":8,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83399999999999996,"15min":0.83299999999999996},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.84599999999999997}]},{"osd":9,"last update":"Sat Mar 30 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86899999999999999,"15min":0.877},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.028,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.85499999999999998,"15min":0.86899999999999999},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98799999999999999,"5min":1.284,"15min":1.284},"last":0.59099999999999997}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":905216,"data_stored":429184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64900000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77700000000000002,"5min":0.873,"15min":0.873},"last":0.46600000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.46000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91100000000000003,"5min":1.002,"15min":1.002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0069999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.64800000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.877,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.44600000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.048,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.749,"15min":0.749},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.96899999999999997,"5min":1.238,"15min":1.238},"last":0.76200000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0329999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.55900000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1080000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.79300000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:33:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.67000000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0169999999999999,"5min":1.105,"15min":1.105},"last":0.877},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.63100000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77700000000000002}]},{"osd":8,"last update":"Sat Mar 30 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.89300000000000002,"15min":0.90700000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1899999999999999,"5min":1.252,"15min":1.252},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":0.86699999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.80600000000000005}]},{"osd":9,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.84299999999999997,"15min":0.86499999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.72399999999999998}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.83599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":970752,"data_stored":491596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:32:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.63700000000000001,"15min":0.59599999999999997},"min":{"1min":0.44700000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.72699999999999998}]},{"osd":2,"last update":"Sat Mar 30 00:33:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.61499999999999999,"15min":0.57099999999999995},"min":{"1min":0.46899999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0609999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.87},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65900000000000003,"15min":0.60199999999999998},"min":{"1min":0.42299999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0149999999999999,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.84899999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65100000000000002,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.125,"15min":1.125},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64000000000000001,"15min":0.59699999999999998},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.98899999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.70799999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.50600000000000001,"15min":0.48299999999999998},"min":{"1min":0.40999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.53700000000000003,"15min":0.505},"min":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.66300000000000003},"min":{"1min":0.53000000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96099999999999997,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.65200000000000002}]},{"osd":6,"last update":"Sat Mar 30 00:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71499999999999997,"15min":0.69599999999999995},"min":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.079,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.88100000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.64500000000000002,"15min":0.629},"min":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.755},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.65800000000000003,"15min":0.64700000000000002},"min":{"1min":0.504,"5min":0.438,"15min":0.438},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.77200000000000002}]},{"osd":8,"last update":"Sat Mar 30 00:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.76500000000000001,"15min":0.754},"min":{"1min":0.64200000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98899999999999999,"5min":1,"15min":1},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.88200000000000001,"15min":0.88300000000000001},"min":{"1min":0.63200000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0920000000000001,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.97299999999999998}]},{"osd":9,"last update":"Sat Mar 30 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.873,"15min":0.871},"min":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.125,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.032},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1200000000000001,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.98899999999999999}]},{"osd":10,"last update":"Sat Mar 30 00:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.0580000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":925696,"data_stored":430455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59099999999999997,"15min":0.57199999999999995},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66600000000000004,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96699999999999997,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.625}]},{"osd":2,"last update":"Sat Mar 30 00:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68999999999999995,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88400000000000001,"5min":5.8550000000000004,"15min":5.8550000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.72699999999999998,"15min":0.81299999999999994},"min":{"1min":0.41999999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.872,"5min":5.282,"15min":5.282},"last":0.63900000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.47999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.67900000000000005,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.77700000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86399999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.85599999999999998,"15min":0.93500000000000005},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97899999999999998,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.54100000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.377,"15min":0.377},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.503},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.629,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.113,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.52500000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.125,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2470000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.81399999999999995}]},{"osd":8,"last update":"Sat Mar 30 00:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.77800000000000002,"15min":0.79800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.93000000000000005,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.85399999999999998,"15min":0.88100000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93899999999999995,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.71499999999999997}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.84599999999999997,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":1.161,"15min":1.161},"last":0.753}]},{"osd":10,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.81499999999999995}]},{"osd":11,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.76800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":995328,"data_stored":492898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.625,"15min":0.622},"min":{"1min":0.46000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89100000000000001,"5min":1.131,"15min":1.131},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.40799999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83499999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54300000000000004}]},{"osd":1,"last update":"Sat Mar 30 00:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60199999999999998,"15min":0.58499999999999996},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.65200000000000002,"15min":0.621},"min":{"1min":0.51200000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.91100000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68200000000000005,"15min":0.66800000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93799999999999994,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.45000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.65100000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.65700000000000003,"15min":0.61899999999999999},"min":{"1min":0.64900000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.93999999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.55100000000000005,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57599999999999996,"15min":0.55000000000000004},"min":{"1min":0.439,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96099999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57299999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77300000000000002,"15min":0.74299999999999999},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0309999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.879},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72099999999999997,"15min":0.69699999999999995},"min":{"1min":0.55900000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":2.944}]},{"osd":7,"last update":"Sat Mar 30 00:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.65700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.83299999999999996}]},{"osd":8,"last update":"Sat Mar 30 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73599999999999999,"15min":0.72899999999999998},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.748},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.83799999999999997,"15min":0.82499999999999996},"min":{"1min":0.47399999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.97899999999999998}]},{"osd":9,"last update":"Sat Mar 30 00:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95799999999999996,"15min":0.95999999999999996},"min":{"1min":0.68500000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.137,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.80900000000000005}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.0760000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]}]},{"osd":3,"up_from":24,"seq":103079215158,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":429495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42499999999999999,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.378},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46800000000000003,"15min":0.437},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.36399999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.622,"15min":0.58599999999999997},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.82199999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.64200000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64800000000000002,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86699999999999999,"5min":0.996,"15min":0.996},"last":0.754},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.66400000000000003,"15min":0.63},"min":{"1min":0.58199999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.77100000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72599999999999998,"15min":0.69299999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83599999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.65500000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.46899999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.879,"5min":0.81100000000000005,"15min":0.79900000000000004},"min":{"1min":0.63300000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.80700000000000005}]},{"osd":8,"last update":"Sat Mar 30 00:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.66700000000000004}]},{"osd":9,"last update":"Sat Mar 30 00:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.87,"15min":0.88400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.504},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0209999999999999,"5min":1.099,"15min":1.099},"last":0.48599999999999999}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":909312,"data_stored":429573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.497}]},{"osd":1,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.47799999999999998,"15min":0.44900000000000001},"min":{"1min":0.35799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.47199999999999998,"15min":0.45000000000000001},"min":{"1min":0.39600000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.45600000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.024}]},{"osd":3,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.67900000000000005,"15min":0.65100000000000002},"min":{"1min":0.58699999999999997,"5min":0.439,"15min":0.439},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.68700000000000006,"15min":0.66300000000000003},"min":{"1min":0.498,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.71299999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68899999999999995,"15min":0.67700000000000005},"min":{"1min":0.42999999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94399999999999995,"5min":1.137,"15min":1.137},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.69699999999999995,"15min":0.67000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0369999999999999,"5min":1.071,"15min":1.071},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82399999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.77500000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.45600000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.52400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.61199999999999999}]},{"osd":8,"last update":"Sat Mar 30 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1619999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.78100000000000003,"15min":0.77300000000000002},"min":{"1min":0.63,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.79200000000000004}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.86499999999999999,"15min":0.88500000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97499999999999998,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.91200000000000003,"15min":0.92900000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0549999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.91900000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]}]},{"osd":5,"up_from":37,"seq":158913789999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32596,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712556,"statfs":{"total":95995035648,"available":95961657344,"internally_reserved":0,"allocated":1519616,"data_stored":1020559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.76400000000000001,"15min":0.82899999999999996},"min":{"1min":0.42199999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94799999999999995,"5min":8.077,"15min":8.077},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59699999999999998,"15min":0.57099999999999995},"min":{"1min":0.45900000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":3.121}]},{"osd":1,"last update":"Sat Mar 30 00:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.66000000000000003,"15min":0.63700000000000001},"min":{"1min":0.47199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76900000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.35499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65500000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.495},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.59099999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"min":{"1min":0.47599999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84699999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.80700000000000005,"15min":0.88500000000000001},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.70199999999999996,"15min":0.66800000000000004},"min":{"1min":0.54100000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.53000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.754}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64700000000000002,"15min":0.629},"min":{"1min":0.56000000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84999999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.72399999999999998,"15min":0.68600000000000005},"min":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:33:08 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.99399999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71299999999999997,"15min":0.70799999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.93899999999999995}]},{"osd":8,"last update":"Sat Mar 30 00:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.73699999999999999,"15min":0.71699999999999997},"min":{"1min":0.51400000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.85899999999999999}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.875,"15min":0.86899999999999999},"min":{"1min":0.70699999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.83499999999999996,"15min":0.83399999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.071,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.91300000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.97799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:33:59.697 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-03-30T00:34:00.057 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:34:00.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:00 smithi161 ceph-mon[42221]: pgmap v278: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-30T00:34:00.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:00 smithi161 ceph-mon[42221]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:00.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:00 smithi099 ceph-mon[38079]: pgmap v278: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-30T00:34:00.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:00 smithi099 ceph-mon[38079]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:00.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:00 smithi142 ceph-mon[41779]: pgmap v278: 97 pgs: 97 active+clean; 641 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-30T00:34:00.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:00 smithi142 ceph-mon[41779]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:01.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.144+0000 7fa100e00700 1 -- 172.21.15.99:0/2338027624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc106e10 msgr2=0x7fa0fc1071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:01.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.144+0000 7fa100e00700 1 --2- 172.21.15.99:0/2338027624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc106e10 0x7fa0fc1071f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009990 tx=0x7fa0ec009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:01.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.144+0000 7fa100e00700 1 -- 172.21.15.99:0/2338027624 shutdown_connections 2024-03-30T00:34:01.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.144+0000 7fa100e00700 1 --2- 172.21.15.99:0/2338027624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc0690f0 0x7fa0fc10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:01.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.144+0000 7fa100e00700 1 --2- 172.21.15.99:0/2338027624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc106e10 0x7fa0fc1071f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:01.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.144+0000 7fa100e00700 1 --2- 172.21.15.99:0/2338027624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc068750 0x7fa0fc068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:01.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.144+0000 7fa100e00700 1 -- 172.21.15.99:0/2338027624 >> 172.21.15.99:0/2338027624 conn(0x7fa0fc075d50 msgr2=0x7fa0fc076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:01.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.144+0000 7fa100e00700 1 -- 172.21.15.99:0/2338027624 shutdown_connections 2024-03-30T00:34:01.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.144+0000 7fa100e00700 1 -- 172.21.15.99:0/2338027624 wait complete. 2024-03-30T00:34:01.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.145+0000 7fa100e00700 1 Processor -- start 2024-03-30T00:34:01.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa100e00700 1 -- start start 2024-03-30T00:34:01.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa100e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc068750 0x7fa0fc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:01.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa100e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc0690f0 0x7fa0fc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:01.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa100e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc106e10 0x7fa0fc19dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa100e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc19e250 con 0x7fa0fc0690f0 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa100e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc19e3c0 con 0x7fa0fc106e10 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa100e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc19e580 con 0x7fa0fc068750 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa0f3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc0690f0 0x7fa0fc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa0f3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc0690f0 0x7fa0fc199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38834/0 (socket says 172.21.15.99:38834) 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa0f3e00700 1 -- 172.21.15.99:0/3507949329 learned_addr learned my addr 172.21.15.99:0/3507949329 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa0fa800700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc106e10 0x7fa0fc19dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.146+0000 7fa0f9e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc068750 0x7fa0fc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.147+0000 7fa0f3e00700 1 -- 172.21.15.99:0/3507949329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc068750 msgr2=0x7fa0fc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.147+0000 7fa0f3e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc068750 0x7fa0fc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.147+0000 7fa0f3e00700 1 -- 172.21.15.99:0/3507949329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc106e10 msgr2=0x7fa0fc19dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.147+0000 7fa0f3e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc106e10 0x7fa0fc19dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:01.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.147+0000 7fa0f3e00700 1 -- 172.21.15.99:0/3507949329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec0095f0 con 0x7fa0fc0690f0 2024-03-30T00:34:01.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.147+0000 7fa0f3e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc0690f0 0x7fa0fc199890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c920 tx=0x7fa0e400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:01.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.147+0000 7fa0f2a00700 1 -- 172.21.15.99:0/3507949329 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4004a30 con 0x7fa0fc0690f0 2024-03-30T00:34:01.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.147+0000 7fa0f2a00700 1 -- 172.21.15.99:0/3507949329 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa0e40054b0 con 0x7fa0fc0690f0 2024-03-30T00:34:01.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.147+0000 7fa0f2a00700 1 -- 172.21.15.99:0/3507949329 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4007b50 con 0x7fa0fc0690f0 2024-03-30T00:34:01.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.147+0000 7fa100e00700 1 -- 172.21.15.99:0/3507949329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc19e860 con 0x7fa0fc0690f0 2024-03-30T00:34:01.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.148+0000 7fa100e00700 1 -- 172.21.15.99:0/3507949329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc1a60c0 con 0x7fa0fc0690f0 2024-03-30T00:34:01.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.149+0000 7fa100e00700 1 -- 172.21.15.99:0/3507949329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc1024c0 con 0x7fa0fc0690f0 2024-03-30T00:34:01.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.151+0000 7fa0f2a00700 1 -- 172.21.15.99:0/3507949329 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa0e4007cb0 con 0x7fa0fc0690f0 2024-03-30T00:34:01.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.152+0000 7fa0f2a00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0e00a1230 0x7fa0e00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:01.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.152+0000 7fa0f2a00700 1 -- 172.21.15.99:0/3507949329 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa0e40cc490 con 0x7fa0fc0690f0 2024-03-30T00:34:01.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.152+0000 7fa0f9e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0e00a1230 0x7fa0e00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:01.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.153+0000 7fa0f9e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0e00a1230 0x7fa0e00a36f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00b5c0 tx=0x7fa0ec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:01.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.156+0000 7fa0f2a00700 1 -- 172.21.15.99:0/3507949329 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa0e4095b70 con 0x7fa0fc0690f0 2024-03-30T00:34:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.355+0000 7fa100e00700 1 -- 172.21.15.99:0/3507949329 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0fc110b90 con 0x7fa0e00a1230 2024-03-30T00:34:01.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.366+0000 7fa0f2a00700 1 -- 172.21.15.99:0/3507949329 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340136 (secure 0 0 0) 0x7fa0fc110b90 con 0x7fa0e00a1230 2024-03-30T00:34:01.379 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 -- 172.21.15.99:0/3507949329 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0e00a1230 msgr2=0x7fa0e00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0e00a1230 0x7fa0e00a36f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00b5c0 tx=0x7fa0ec005fb0 comp rx=0 tx=0).stop 2024-03-30T00:34:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 -- 172.21.15.99:0/3507949329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc0690f0 msgr2=0x7fa0fc199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc0690f0 0x7fa0fc199890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c920 tx=0x7fa0e400cc30 comp rx=0 tx=0).stop 2024-03-30T00:34:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 -- 172.21.15.99:0/3507949329 shutdown_connections 2024-03-30T00:34:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0e00a1230 0x7fa0e00a36f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc106e10 0x7fa0fc19dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0fc068750 0x7fa0fc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 --2- 172.21.15.99:0/3507949329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0fc0690f0 0x7fa0fc199890 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 -- 172.21.15.99:0/3507949329 >> 172.21.15.99:0/3507949329 conn(0x7fa0fc075d50 msgr2=0x7fa0fc10c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:01.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 -- 172.21.15.99:0/3507949329 shutdown_connections 2024-03-30T00:34:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:01.371+0000 7fa100e00700 1 -- 172.21.15.99:0/3507949329 wait complete. 2024-03-30T00:34:01.383 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-30T00:34:01.756 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-03-30T00:34:01.018946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658387,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":390,"num_read_kb":313,"num_write":340,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377840,"kb_used_data":12688,"kb_used_omap":18,"kb_used_meta":365037,"kb_avail":1124563984,"statfs":{"total":1151940427776,"available":1151553519616,"internally_reserved":0,"allocated":12992512,"data_stored":7120621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":373798214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5216,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":46,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002923"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290853+0000","last_change":"2024-03-30T00:33:15.686814+0000","last_active":"2024-03-30T00:33:17.290853+0000","last_peered":"2024-03-30T00:33:17.290853+0000","last_clean":"2024-03-30T00:33:17.290853+0000","last_became_active":"2024-03-30T00:33:15.686432+0000","last_became_peered":"2024-03-30T00:33:15.686432+0000","last_unstale":"2024-03-30T00:33:17.290853+0000","last_undegraded":"2024-03-30T00:33:17.290853+0000","last_fullsized":"2024-03-30T00:33:17.290853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:35:05.898973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002227+0000","last_change":"2024-03-30T00:33:15.678089+0000","last_active":"2024-03-30T00:33:17.002227+0000","last_peered":"2024-03-30T00:33:17.002227+0000","last_clean":"2024-03-30T00:33:17.002227+0000","last_became_active":"2024-03-30T00:33:15.677254+0000","last_became_peered":"2024-03-30T00:33:15.677254+0000","last_unstale":"2024-03-30T00:33:17.002227+0000","last_undegraded":"2024-03-30T00:33:17.002227+0000","last_fullsized":"2024-03-30T00:33:17.002227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:45.922652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002176+0000","last_change":"2024-03-30T00:33:15.686777+0000","last_active":"2024-03-30T00:33:17.002176+0000","last_peered":"2024-03-30T00:33:17.002176+0000","last_clean":"2024-03-30T00:33:17.002176+0000","last_became_active":"2024-03-30T00:33:15.686461+0000","last_became_peered":"2024-03-30T00:33:15.686461+0000","last_unstale":"2024-03-30T00:33:17.002176+0000","last_undegraded":"2024-03-30T00:33:17.002176+0000","last_fullsized":"2024-03-30T00:33:17.002176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:33:27.073089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002223+0000","last_change":"2024-03-30T00:33:15.681237+0000","last_active":"2024-03-30T00:33:17.002223+0000","last_peered":"2024-03-30T00:33:17.002223+0000","last_clean":"2024-03-30T00:33:17.002223+0000","last_became_active":"2024-03-30T00:33:15.680669+0000","last_became_peered":"2024-03-30T00:33:15.680669+0000","last_unstale":"2024-03-30T00:33:17.002223+0000","last_undegraded":"2024-03-30T00:33:17.002223+0000","last_fullsized":"2024-03-30T00:33:17.002223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:59.300262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.682286+0000","last_change":"2024-03-30T00:33:15.678825+0000","last_active":"2024-03-30T00:33:15.682286+0000","last_peered":"2024-03-30T00:33:15.682286+0000","last_clean":"2024-03-30T00:33:15.682286+0000","last_became_active":"2024-03-30T00:33:15.676864+0000","last_became_peered":"2024-03-30T00:33:15.676864+0000","last_unstale":"2024-03-30T00:33:15.682286+0000","last_undegraded":"2024-03-30T00:33:15.682286+0000","last_fullsized":"2024-03-30T00:33:15.682286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:12:09.724441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.688863+0000","last_change":"2024-03-30T00:33:15.681866+0000","last_active":"2024-03-30T00:33:15.688863+0000","last_peered":"2024-03-30T00:33:15.688863+0000","last_clean":"2024-03-30T00:33:15.688863+0000","last_became_active":"2024-03-30T00:33:15.678118+0000","last_became_peered":"2024-03-30T00:33:15.678118+0000","last_unstale":"2024-03-30T00:33:15.688863+0000","last_undegraded":"2024-03-30T00:33:15.688863+0000","last_fullsized":"2024-03-30T00:33:15.688863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:24.433027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001193+0000","last_change":"2024-03-30T00:33:15.675999+0000","last_active":"2024-03-30T00:33:17.001193+0000","last_peered":"2024-03-30T00:33:17.001193+0000","last_clean":"2024-03-30T00:33:17.001193+0000","last_became_active":"2024-03-30T00:33:15.675319+0000","last_became_peered":"2024-03-30T00:33:15.675319+0000","last_unstale":"2024-03-30T00:33:17.001193+0000","last_undegraded":"2024-03-30T00:33:17.001193+0000","last_fullsized":"2024-03-30T00:33:17.001193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:42.668747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000423+0000","last_change":"2024-03-30T00:33:15.679211+0000","last_active":"2024-03-30T00:33:17.000423+0000","last_peered":"2024-03-30T00:33:17.000423+0000","last_clean":"2024-03-30T00:33:17.000423+0000","last_became_active":"2024-03-30T00:33:15.675061+0000","last_became_peered":"2024-03-30T00:33:15.675061+0000","last_unstale":"2024-03-30T00:33:17.000423+0000","last_undegraded":"2024-03-30T00:33:17.000423+0000","last_fullsized":"2024-03-30T00:33:17.000423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:43.973542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.692417+0000","last_change":"2024-03-30T00:33:15.680842+0000","last_active":"2024-03-30T00:33:15.692417+0000","last_peered":"2024-03-30T00:33:15.692417+0000","last_clean":"2024-03-30T00:33:15.692417+0000","last_became_active":"2024-03-30T00:33:15.680307+0000","last_became_peered":"2024-03-30T00:33:15.680307+0000","last_unstale":"2024-03-30T00:33:15.692417+0000","last_undegraded":"2024-03-30T00:33:15.692417+0000","last_fullsized":"2024-03-30T00:33:15.692417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:59:40.331779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693556+0000","last_change":"2024-03-30T00:33:15.692397+0000","last_active":"2024-03-30T00:33:15.693556+0000","last_peered":"2024-03-30T00:33:15.693556+0000","last_clean":"2024-03-30T00:33:15.693556+0000","last_became_active":"2024-03-30T00:33:15.691226+0000","last_became_peered":"2024-03-30T00:33:15.691226+0000","last_unstale":"2024-03-30T00:33:15.693556+0000","last_undegraded":"2024-03-30T00:33:15.693556+0000","last_fullsized":"2024-03-30T00:33:15.693556+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:48.816644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001314+0000","last_change":"2024-03-30T00:33:15.688417+0000","last_active":"2024-03-30T00:33:17.001314+0000","last_peered":"2024-03-30T00:33:17.001314+0000","last_clean":"2024-03-30T00:33:17.001314+0000","last_became_active":"2024-03-30T00:33:15.687755+0000","last_became_peered":"2024-03-30T00:33:15.687755+0000","last_unstale":"2024-03-30T00:33:17.001314+0000","last_undegraded":"2024-03-30T00:33:17.001314+0000","last_fullsized":"2024-03-30T00:33:17.001314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:30:19.782280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002119+0000","last_change":"2024-03-30T00:33:15.681942+0000","last_active":"2024-03-30T00:33:17.002119+0000","last_peered":"2024-03-30T00:33:17.002119+0000","last_clean":"2024-03-30T00:33:17.002119+0000","last_became_active":"2024-03-30T00:33:15.679854+0000","last_became_peered":"2024-03-30T00:33:15.679854+0000","last_unstale":"2024-03-30T00:33:17.002119+0000","last_undegraded":"2024-03-30T00:33:17.002119+0000","last_fullsized":"2024-03-30T00:33:17.002119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:19.771959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001108+0000","last_change":"2024-03-30T00:33:15.682968+0000","last_active":"2024-03-30T00:33:17.001108+0000","last_peered":"2024-03-30T00:33:17.001108+0000","last_clean":"2024-03-30T00:33:17.001108+0000","last_became_active":"2024-03-30T00:33:15.681216+0000","last_became_peered":"2024-03-30T00:33:15.681216+0000","last_unstale":"2024-03-30T00:33:17.001108+0000","last_undegraded":"2024-03-30T00:33:17.001108+0000","last_fullsized":"2024-03-30T00:33:17.001108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:50.038843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001697+0000","last_change":"2024-03-30T00:33:15.677727+0000","last_active":"2024-03-30T00:33:17.001697+0000","last_peered":"2024-03-30T00:33:17.001697+0000","last_clean":"2024-03-30T00:33:17.001697+0000","last_became_active":"2024-03-30T00:33:15.676758+0000","last_became_peered":"2024-03-30T00:33:15.676758+0000","last_unstale":"2024-03-30T00:33:17.001697+0000","last_undegraded":"2024-03-30T00:33:17.001697+0000","last_fullsized":"2024-03-30T00:33:17.001697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:34.745920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001691+0000","last_change":"2024-03-30T00:33:15.684545+0000","last_active":"2024-03-30T00:33:17.001691+0000","last_peered":"2024-03-30T00:33:17.001691+0000","last_clean":"2024-03-30T00:33:17.001691+0000","last_became_active":"2024-03-30T00:33:15.682989+0000","last_became_peered":"2024-03-30T00:33:15.682989+0000","last_unstale":"2024-03-30T00:33:17.001691+0000","last_undegraded":"2024-03-30T00:33:17.001691+0000","last_fullsized":"2024-03-30T00:33:17.001691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:11.120981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.686822+0000","last_change":"2024-03-30T00:33:15.678122+0000","last_active":"2024-03-30T00:33:15.686822+0000","last_peered":"2024-03-30T00:33:15.686822+0000","last_clean":"2024-03-30T00:33:15.686822+0000","last_became_active":"2024-03-30T00:33:15.677681+0000","last_became_peered":"2024-03-30T00:33:15.677681+0000","last_unstale":"2024-03-30T00:33:15.686822+0000","last_undegraded":"2024-03-30T00:33:15.686822+0000","last_fullsized":"2024-03-30T00:33:15.686822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:50.454125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290980+0000","last_change":"2024-03-30T00:33:15.687689+0000","last_active":"2024-03-30T00:33:17.290980+0000","last_peered":"2024-03-30T00:33:17.290980+0000","last_clean":"2024-03-30T00:33:17.290980+0000","last_became_active":"2024-03-30T00:33:15.686009+0000","last_became_peered":"2024-03-30T00:33:15.686009+0000","last_unstale":"2024-03-30T00:33:17.290980+0000","last_undegraded":"2024-03-30T00:33:17.290980+0000","last_fullsized":"2024-03-30T00:33:17.290980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:17.234749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659641+0000","last_change":"2024-03-30T00:33:15.686053+0000","last_active":"2024-03-30T00:33:16.659641+0000","last_peered":"2024-03-30T00:33:16.659641+0000","last_clean":"2024-03-30T00:33:16.659641+0000","last_became_active":"2024-03-30T00:33:15.685723+0000","last_became_peered":"2024-03-30T00:33:15.685723+0000","last_unstale":"2024-03-30T00:33:16.659641+0000","last_undegraded":"2024-03-30T00:33:16.659641+0000","last_fullsized":"2024-03-30T00:33:16.659641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:27.322834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660740+0000","last_change":"2024-03-30T00:33:15.674783+0000","last_active":"2024-03-30T00:33:16.660740+0000","last_peered":"2024-03-30T00:33:16.660740+0000","last_clean":"2024-03-30T00:33:16.660740+0000","last_became_active":"2024-03-30T00:33:15.674152+0000","last_became_peered":"2024-03-30T00:33:15.674152+0000","last_unstale":"2024-03-30T00:33:16.660740+0000","last_undegraded":"2024-03-30T00:33:16.660740+0000","last_fullsized":"2024-03-30T00:33:16.660740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:39.517240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.119665+0000","last_change":"2024-03-30T00:33:15.172217+0000","last_active":"2024-03-30T00:33:20.119665+0000","last_peered":"2024-03-30T00:33:20.119665+0000","last_clean":"2024-03-30T00:33:20.119665+0000","last_became_active":"2024-03-30T00:33:15.171718+0000","last_became_peered":"2024-03-30T00:33:15.171718+0000","last_unstale":"2024-03-30T00:33:20.119665+0000","last_undegraded":"2024-03-30T00:33:20.119665+0000","last_fullsized":"2024-03-30T00:33:20.119665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:11.691572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001530+0000","last_change":"2024-03-30T00:33:15.669937+0000","last_active":"2024-03-30T00:33:17.001530+0000","last_peered":"2024-03-30T00:33:17.001530+0000","last_clean":"2024-03-30T00:33:17.001530+0000","last_became_active":"2024-03-30T00:33:15.668704+0000","last_became_peered":"2024-03-30T00:33:15.668704+0000","last_unstale":"2024-03-30T00:33:17.001530+0000","last_undegraded":"2024-03-30T00:33:17.001530+0000","last_fullsized":"2024-03-30T00:33:17.001530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:23.914755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:18.428835+0000","last_change":"2024-03-30T00:33:15.173663+0000","last_active":"2024-03-30T00:33:18.428835+0000","last_peered":"2024-03-30T00:33:18.428835+0000","last_clean":"2024-03-30T00:33:18.428835+0000","last_became_active":"2024-03-30T00:33:15.173288+0000","last_became_peered":"2024-03-30T00:33:15.173288+0000","last_unstale":"2024-03-30T00:33:18.428835+0000","last_undegraded":"2024-03-30T00:33:18.428835+0000","last_fullsized":"2024-03-30T00:33:18.428835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.435142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002383+0000","last_change":"2024-03-30T00:33:15.686757+0000","last_active":"2024-03-30T00:33:17.002383+0000","last_peered":"2024-03-30T00:33:17.002383+0000","last_clean":"2024-03-30T00:33:17.002383+0000","last_became_active":"2024-03-30T00:33:15.686306+0000","last_became_peered":"2024-03-30T00:33:15.686306+0000","last_unstale":"2024-03-30T00:33:17.002383+0000","last_undegraded":"2024-03-30T00:33:17.002383+0000","last_fullsized":"2024-03-30T00:33:17.002383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:50:21.048212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.117995+0000","last_change":"2024-03-30T00:33:15.173402+0000","last_active":"2024-03-30T00:33:20.117995+0000","last_peered":"2024-03-30T00:33:20.117995+0000","last_clean":"2024-03-30T00:33:20.117995+0000","last_became_active":"2024-03-30T00:33:15.172995+0000","last_became_peered":"2024-03-30T00:33:15.172995+0000","last_unstale":"2024-03-30T00:33:20.117995+0000","last_undegraded":"2024-03-30T00:33:20.117995+0000","last_fullsized":"2024-03-30T00:33:20.117995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:25.694792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.691152+0000","last_change":"2024-03-30T00:33:15.688292+0000","last_active":"2024-03-30T00:33:15.691152+0000","last_peered":"2024-03-30T00:33:15.691152+0000","last_clean":"2024-03-30T00:33:15.691152+0000","last_became_active":"2024-03-30T00:33:15.682722+0000","last_became_peered":"2024-03-30T00:33:15.682722+0000","last_unstale":"2024-03-30T00:33:15.691152+0000","last_undegraded":"2024-03-30T00:33:15.691152+0000","last_fullsized":"2024-03-30T00:33:15.691152+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:52:10.074656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.692186+0000","last_change":"2024-03-30T00:33:15.162316+0000","last_active":"2024-03-30T00:33:15.692186+0000","last_peered":"2024-03-30T00:33:15.692186+0000","last_clean":"2024-03-30T00:33:15.692186+0000","last_became_active":"2024-03-30T00:33:15.161805+0000","last_became_peered":"2024-03-30T00:33:15.161805+0000","last_unstale":"2024-03-30T00:33:15.692186+0000","last_undegraded":"2024-03-30T00:33:15.692186+0000","last_fullsized":"2024-03-30T00:33:15.692186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:26:16.470245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.532008+0000","last_change":"2024-03-30T00:33:15.675442+0000","last_active":"2024-03-30T00:33:16.532008+0000","last_peered":"2024-03-30T00:33:16.532008+0000","last_clean":"2024-03-30T00:33:16.532008+0000","last_became_active":"2024-03-30T00:33:15.675068+0000","last_became_peered":"2024-03-30T00:33:15.675068+0000","last_unstale":"2024-03-30T00:33:16.532008+0000","last_undegraded":"2024-03-30T00:33:16.532008+0000","last_fullsized":"2024-03-30T00:33:16.532008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:36:31.353262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:55.169272+0000","last_change":"2024-03-30T00:33:15.172087+0000","last_active":"2024-03-30T00:33:55.169272+0000","last_peered":"2024-03-30T00:33:55.169272+0000","last_clean":"2024-03-30T00:33:55.169272+0000","last_became_active":"2024-03-30T00:33:15.171768+0000","last_became_peered":"2024-03-30T00:33:15.171768+0000","last_unstale":"2024-03-30T00:33:55.169272+0000","last_undegraded":"2024-03-30T00:33:55.169272+0000","last_fullsized":"2024-03-30T00:33:55.169272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:17.198995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.668850+0000","last_change":"2024-03-30T00:33:15.665219+0000","last_active":"2024-03-30T00:33:15.668850+0000","last_peered":"2024-03-30T00:33:15.668850+0000","last_clean":"2024-03-30T00:33:15.668850+0000","last_became_active":"2024-03-30T00:33:15.664935+0000","last_became_peered":"2024-03-30T00:33:15.664935+0000","last_unstale":"2024-03-30T00:33:15.668850+0000","last_undegraded":"2024-03-30T00:33:15.668850+0000","last_fullsized":"2024-03-30T00:33:15.668850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:37.123937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.116508+0000","last_change":"2024-03-30T00:33:15.165668+0000","last_active":"2024-03-30T00:33:20.116508+0000","last_peered":"2024-03-30T00:33:20.116508+0000","last_clean":"2024-03-30T00:33:20.116508+0000","last_became_active":"2024-03-30T00:33:15.164637+0000","last_became_peered":"2024-03-30T00:33:15.164637+0000","last_unstale":"2024-03-30T00:33:20.116508+0000","last_undegraded":"2024-03-30T00:33:20.116508+0000","last_fullsized":"2024-03-30T00:33:20.116508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:47:59.389414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531981+0000","last_change":"2024-03-30T00:33:15.662357+0000","last_active":"2024-03-30T00:33:16.531981+0000","last_peered":"2024-03-30T00:33:16.531981+0000","last_clean":"2024-03-30T00:33:16.531981+0000","last_became_active":"2024-03-30T00:33:15.662011+0000","last_became_peered":"2024-03-30T00:33:15.662011+0000","last_unstale":"2024-03-30T00:33:16.531981+0000","last_undegraded":"2024-03-30T00:33:16.531981+0000","last_fullsized":"2024-03-30T00:33:16.531981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:23:18.386054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118436+0000","last_change":"2024-03-30T00:33:15.177269+0000","last_active":"2024-03-30T00:33:20.118436+0000","last_peered":"2024-03-30T00:33:20.118436+0000","last_clean":"2024-03-30T00:33:20.118436+0000","last_became_active":"2024-03-30T00:33:15.176937+0000","last_became_peered":"2024-03-30T00:33:15.176937+0000","last_unstale":"2024-03-30T00:33:20.118436+0000","last_undegraded":"2024-03-30T00:33:20.118436+0000","last_fullsized":"2024-03-30T00:33:20.118436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:44.658775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000691+0000","last_change":"2024-03-30T00:33:15.675832+0000","last_active":"2024-03-30T00:33:17.000691+0000","last_peered":"2024-03-30T00:33:17.000691+0000","last_clean":"2024-03-30T00:33:17.000691+0000","last_became_active":"2024-03-30T00:33:15.675481+0000","last_became_peered":"2024-03-30T00:33:15.675481+0000","last_unstale":"2024-03-30T00:33:17.000691+0000","last_undegraded":"2024-03-30T00:33:17.000691+0000","last_fullsized":"2024-03-30T00:33:17.000691+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:11.638056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001044+0000","last_change":"2024-03-30T00:33:15.171238+0000","last_active":"2024-03-30T00:33:17.001044+0000","last_peered":"2024-03-30T00:33:17.001044+0000","last_clean":"2024-03-30T00:33:17.001044+0000","last_became_active":"2024-03-30T00:33:15.170931+0000","last_became_peered":"2024-03-30T00:33:15.170931+0000","last_unstale":"2024-03-30T00:33:17.001044+0000","last_undegraded":"2024-03-30T00:33:17.001044+0000","last_fullsized":"2024-03-30T00:33:17.001044+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:43:50.178363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659336+0000","last_change":"2024-03-30T00:33:15.676014+0000","last_active":"2024-03-30T00:33:16.659336+0000","last_peered":"2024-03-30T00:33:16.659336+0000","last_clean":"2024-03-30T00:33:16.659336+0000","last_became_active":"2024-03-30T00:33:15.675553+0000","last_became_peered":"2024-03-30T00:33:15.675553+0000","last_unstale":"2024-03-30T00:33:16.659336+0000","last_undegraded":"2024-03-30T00:33:16.659336+0000","last_fullsized":"2024-03-30T00:33:16.659336+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:17.248242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'3","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:59.175591+0000","last_change":"2024-03-30T00:33:15.177033+0000","last_active":"2024-03-30T00:33:59.175591+0000","last_peered":"2024-03-30T00:33:59.175591+0000","last_clean":"2024-03-30T00:33:59.175591+0000","last_became_active":"2024-03-30T00:33:15.176602+0000","last_became_peered":"2024-03-30T00:33:15.176602+0000","last_unstale":"2024-03-30T00:33:59.175591+0000","last_undegraded":"2024-03-30T00:33:59.175591+0000","last_fullsized":"2024-03-30T00:33:59.175591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:10.804704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.691801+0000","last_change":"2024-03-30T00:33:15.683388+0000","last_active":"2024-03-30T00:33:15.691801+0000","last_peered":"2024-03-30T00:33:15.691801+0000","last_clean":"2024-03-30T00:33:15.691801+0000","last_became_active":"2024-03-30T00:33:15.682976+0000","last_became_peered":"2024-03-30T00:33:15.682976+0000","last_unstale":"2024-03-30T00:33:15.691801+0000","last_undegraded":"2024-03-30T00:33:15.691801+0000","last_fullsized":"2024-03-30T00:33:15.691801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:50:34.907265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131200+0000","last_change":"2024-03-30T00:33:15.167819+0000","last_active":"2024-03-30T00:33:19.131200+0000","last_peered":"2024-03-30T00:33:19.131200+0000","last_clean":"2024-03-30T00:33:19.131200+0000","last_became_active":"2024-03-30T00:33:15.167405+0000","last_became_peered":"2024-03-30T00:33:15.167405+0000","last_unstale":"2024-03-30T00:33:19.131200+0000","last_undegraded":"2024-03-30T00:33:19.131200+0000","last_fullsized":"2024-03-30T00:33:19.131200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:37.793792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002345+0000","last_change":"2024-03-30T00:33:15.690907+0000","last_active":"2024-03-30T00:33:17.002345+0000","last_peered":"2024-03-30T00:33:17.002345+0000","last_clean":"2024-03-30T00:33:17.002345+0000","last_became_active":"2024-03-30T00:33:15.690046+0000","last_became_peered":"2024-03-30T00:33:15.690046+0000","last_unstale":"2024-03-30T00:33:17.002345+0000","last_undegraded":"2024-03-30T00:33:17.002345+0000","last_fullsized":"2024-03-30T00:33:17.002345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:24.319685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.132456+0000","last_change":"2024-03-30T00:33:15.166633+0000","last_active":"2024-03-30T00:33:19.132456+0000","last_peered":"2024-03-30T00:33:19.132456+0000","last_clean":"2024-03-30T00:33:19.132456+0000","last_became_active":"2024-03-30T00:33:15.166308+0000","last_became_peered":"2024-03-30T00:33:15.166308+0000","last_unstale":"2024-03-30T00:33:19.132456+0000","last_undegraded":"2024-03-30T00:33:19.132456+0000","last_fullsized":"2024-03-30T00:33:19.132456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:53.102872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693949+0000","last_change":"2024-03-30T00:33:15.691880+0000","last_active":"2024-03-30T00:33:15.693949+0000","last_peered":"2024-03-30T00:33:15.693949+0000","last_clean":"2024-03-30T00:33:15.693949+0000","last_became_active":"2024-03-30T00:33:15.690780+0000","last_became_peered":"2024-03-30T00:33:15.690780+0000","last_unstale":"2024-03-30T00:33:15.693949+0000","last_undegraded":"2024-03-30T00:33:15.693949+0000","last_fullsized":"2024-03-30T00:33:15.693949+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:06.532709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'33","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:34:00.189362+0000","last_change":"2024-03-30T00:33:15.164287+0000","last_active":"2024-03-30T00:34:00.189362+0000","last_peered":"2024-03-30T00:34:00.189362+0000","last_clean":"2024-03-30T00:34:00.189362+0000","last_became_active":"2024-03-30T00:33:15.163557+0000","last_became_peered":"2024-03-30T00:33:15.163557+0000","last_unstale":"2024-03-30T00:34:00.189362+0000","last_undegraded":"2024-03-30T00:34:00.189362+0000","last_fullsized":"2024-03-30T00:34:00.189362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:41.117936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64271,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":92,"num_read_kb":108,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.683827+0000","last_change":"2024-03-30T00:33:15.682841+0000","last_active":"2024-03-30T00:33:15.683827+0000","last_peered":"2024-03-30T00:33:15.683827+0000","last_clean":"2024-03-30T00:33:15.683827+0000","last_became_active":"2024-03-30T00:33:15.682237+0000","last_became_peered":"2024-03-30T00:33:15.682237+0000","last_unstale":"2024-03-30T00:33:15.683827+0000","last_undegraded":"2024-03-30T00:33:15.683827+0000","last_fullsized":"2024-03-30T00:33:15.683827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:59:05.719216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118639+0000","last_change":"2024-03-30T00:33:15.165193+0000","last_active":"2024-03-30T00:33:20.118639+0000","last_peered":"2024-03-30T00:33:20.118639+0000","last_clean":"2024-03-30T00:33:20.118639+0000","last_became_active":"2024-03-30T00:33:15.161741+0000","last_became_peered":"2024-03-30T00:33:15.161741+0000","last_unstale":"2024-03-30T00:33:20.118639+0000","last_undegraded":"2024-03-30T00:33:20.118639+0000","last_fullsized":"2024-03-30T00:33:20.118639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:22:11.891064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001296+0000","last_change":"2024-03-30T00:33:15.680929+0000","last_active":"2024-03-30T00:33:17.001296+0000","last_peered":"2024-03-30T00:33:17.001296+0000","last_clean":"2024-03-30T00:33:17.001296+0000","last_became_active":"2024-03-30T00:33:15.676671+0000","last_became_peered":"2024-03-30T00:33:15.676671+0000","last_unstale":"2024-03-30T00:33:17.001296+0000","last_undegraded":"2024-03-30T00:33:17.001296+0000","last_fullsized":"2024-03-30T00:33:17.001296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:28:33.819135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.132787+0000","last_change":"2024-03-30T00:33:15.168029+0000","last_active":"2024-03-30T00:33:19.132787+0000","last_peered":"2024-03-30T00:33:19.132787+0000","last_clean":"2024-03-30T00:33:19.132787+0000","last_became_active":"2024-03-30T00:33:15.167636+0000","last_became_peered":"2024-03-30T00:33:15.167636+0000","last_unstale":"2024-03-30T00:33:19.132787+0000","last_undegraded":"2024-03-30T00:33:19.132787+0000","last_fullsized":"2024-03-30T00:33:19.132787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:25:47.834696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001171+0000","last_change":"2024-03-30T00:33:15.687977+0000","last_active":"2024-03-30T00:33:17.001171+0000","last_peered":"2024-03-30T00:33:17.001171+0000","last_clean":"2024-03-30T00:33:17.001171+0000","last_became_active":"2024-03-30T00:33:15.687491+0000","last_became_peered":"2024-03-30T00:33:15.687491+0000","last_unstale":"2024-03-30T00:33:17.001171+0000","last_undegraded":"2024-03-30T00:33:17.001171+0000","last_fullsized":"2024-03-30T00:33:17.001171+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:08:55.568039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'3","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:58.195061+0000","last_change":"2024-03-30T00:33:15.171826+0000","last_active":"2024-03-30T00:33:58.195061+0000","last_peered":"2024-03-30T00:33:58.195061+0000","last_clean":"2024-03-30T00:33:58.195061+0000","last_became_active":"2024-03-30T00:33:15.170749+0000","last_became_peered":"2024-03-30T00:33:15.170749+0000","last_unstale":"2024-03-30T00:33:58.195061+0000","last_undegraded":"2024-03-30T00:33:58.195061+0000","last_fullsized":"2024-03-30T00:33:58.195061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:57:52.034530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000714+0000","last_change":"2024-03-30T00:33:15.681668+0000","last_active":"2024-03-30T00:33:17.000714+0000","last_peered":"2024-03-30T00:33:17.000714+0000","last_clean":"2024-03-30T00:33:17.000714+0000","last_became_active":"2024-03-30T00:33:15.681129+0000","last_became_peered":"2024-03-30T00:33:15.681129+0000","last_unstale":"2024-03-30T00:33:17.000714+0000","last_undegraded":"2024-03-30T00:33:17.000714+0000","last_fullsized":"2024-03-30T00:33:17.000714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:51:19.552243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118932+0000","last_change":"2024-03-30T00:33:15.171016+0000","last_active":"2024-03-30T00:33:20.118932+0000","last_peered":"2024-03-30T00:33:20.118932+0000","last_clean":"2024-03-30T00:33:20.118932+0000","last_became_active":"2024-03-30T00:33:15.170058+0000","last_became_peered":"2024-03-30T00:33:15.170058+0000","last_unstale":"2024-03-30T00:33:20.118932+0000","last_undegraded":"2024-03-30T00:33:20.118932+0000","last_fullsized":"2024-03-30T00:33:20.118932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:07:39.910286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531944+0000","last_change":"2024-03-30T00:33:15.687012+0000","last_active":"2024-03-30T00:33:16.531944+0000","last_peered":"2024-03-30T00:33:16.531944+0000","last_clean":"2024-03-30T00:33:16.531944+0000","last_became_active":"2024-03-30T00:33:15.684030+0000","last_became_peered":"2024-03-30T00:33:15.684030+0000","last_unstale":"2024-03-30T00:33:16.531944+0000","last_undegraded":"2024-03-30T00:33:16.531944+0000","last_fullsized":"2024-03-30T00:33:16.531944+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:15:32.447119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.103201+0000","last_change":"2024-03-30T00:33:15.163633+0000","last_active":"2024-03-30T00:33:20.103201+0000","last_peered":"2024-03-30T00:33:20.103201+0000","last_clean":"2024-03-30T00:33:20.103201+0000","last_became_active":"2024-03-30T00:33:15.161339+0000","last_became_peered":"2024-03-30T00:33:15.161339+0000","last_unstale":"2024-03-30T00:33:20.103201+0000","last_undegraded":"2024-03-30T00:33:20.103201+0000","last_fullsized":"2024-03-30T00:33:20.103201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:15:25.194650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290780+0000","last_change":"2024-03-30T00:33:15.680617+0000","last_active":"2024-03-30T00:33:17.290780+0000","last_peered":"2024-03-30T00:33:17.290780+0000","last_clean":"2024-03-30T00:33:17.290780+0000","last_became_active":"2024-03-30T00:33:15.679137+0000","last_became_peered":"2024-03-30T00:33:15.679137+0000","last_unstale":"2024-03-30T00:33:17.290780+0000","last_undegraded":"2024-03-30T00:33:17.290780+0000","last_fullsized":"2024-03-30T00:33:17.290780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:26:34.449218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.659518+0000","last_change":"2024-03-30T00:33:15.177273+0000","last_active":"2024-03-30T00:33:16.659518+0000","last_peered":"2024-03-30T00:33:16.659518+0000","last_clean":"2024-03-30T00:33:16.659518+0000","last_became_active":"2024-03-30T00:33:15.175333+0000","last_became_peered":"2024-03-30T00:33:15.175333+0000","last_unstale":"2024-03-30T00:33:16.659518+0000","last_undegraded":"2024-03-30T00:33:16.659518+0000","last_fullsized":"2024-03-30T00:33:16.659518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:27:03.305671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002883+0000","last_change":"2024-03-30T00:33:15.674746+0000","last_active":"2024-03-30T00:33:17.002883+0000","last_peered":"2024-03-30T00:33:17.002883+0000","last_clean":"2024-03-30T00:33:17.002883+0000","last_became_active":"2024-03-30T00:33:15.674388+0000","last_became_peered":"2024-03-30T00:33:15.674388+0000","last_unstale":"2024-03-30T00:33:17.002883+0000","last_undegraded":"2024-03-30T00:33:17.002883+0000","last_fullsized":"2024-03-30T00:33:17.002883+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:21:15.024181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.054408+0000","last_change":"2024-03-30T00:33:15.173640+0000","last_active":"2024-03-30T00:33:19.054408+0000","last_peered":"2024-03-30T00:33:19.054408+0000","last_clean":"2024-03-30T00:33:19.054408+0000","last_became_active":"2024-03-30T00:33:15.173104+0000","last_became_peered":"2024-03-30T00:33:15.173104+0000","last_unstale":"2024-03-30T00:33:19.054408+0000","last_undegraded":"2024-03-30T00:33:19.054408+0000","last_fullsized":"2024-03-30T00:33:19.054408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:20:14.880730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.683575+0000","last_change":"2024-03-30T00:33:15.677824+0000","last_active":"2024-03-30T00:33:15.683575+0000","last_peered":"2024-03-30T00:33:15.683575+0000","last_clean":"2024-03-30T00:33:15.683575+0000","last_became_active":"2024-03-30T00:33:15.677435+0000","last_became_peered":"2024-03-30T00:33:15.677435+0000","last_unstale":"2024-03-30T00:33:15.683575+0000","last_undegraded":"2024-03-30T00:33:15.683575+0000","last_fullsized":"2024-03-30T00:33:15.683575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:22.992989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531931+0000","last_change":"2024-03-30T00:33:15.667428+0000","last_active":"2024-03-30T00:33:16.531931+0000","last_peered":"2024-03-30T00:33:16.531931+0000","last_clean":"2024-03-30T00:33:16.531931+0000","last_became_active":"2024-03-30T00:33:15.666634+0000","last_became_peered":"2024-03-30T00:33:15.666634+0000","last_unstale":"2024-03-30T00:33:16.531931+0000","last_undegraded":"2024-03-30T00:33:16.531931+0000","last_fullsized":"2024-03-30T00:33:16.531931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:56:14.710178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'4","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:56.191040+0000","last_change":"2024-03-30T00:33:15.173264+0000","last_active":"2024-03-30T00:33:56.191040+0000","last_peered":"2024-03-30T00:33:56.191040+0000","last_clean":"2024-03-30T00:33:56.191040+0000","last_became_active":"2024-03-30T00:33:15.172932+0000","last_became_peered":"2024-03-30T00:33:15.172932+0000","last_unstale":"2024-03-30T00:33:56.191040+0000","last_undegraded":"2024-03-30T00:33:56.191040+0000","last_fullsized":"2024-03-30T00:33:56.191040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:18:25.079284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000756+0000","last_change":"2024-03-30T00:33:15.686606+0000","last_active":"2024-03-30T00:33:17.000756+0000","last_peered":"2024-03-30T00:33:17.000756+0000","last_clean":"2024-03-30T00:33:17.000756+0000","last_became_active":"2024-03-30T00:33:15.686076+0000","last_became_peered":"2024-03-30T00:33:15.686076+0000","last_unstale":"2024-03-30T00:33:17.000756+0000","last_undegraded":"2024-03-30T00:33:17.000756+0000","last_fullsized":"2024-03-30T00:33:17.000756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:46.962659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.119040+0000","last_change":"2024-03-30T00:33:15.177263+0000","last_active":"2024-03-30T00:33:20.119040+0000","last_peered":"2024-03-30T00:33:20.119040+0000","last_clean":"2024-03-30T00:33:20.119040+0000","last_became_active":"2024-03-30T00:33:15.176743+0000","last_became_peered":"2024-03-30T00:33:15.176743+0000","last_unstale":"2024-03-30T00:33:20.119040+0000","last_undegraded":"2024-03-30T00:33:20.119040+0000","last_fullsized":"2024-03-30T00:33:20.119040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:56:47.491371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":322,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.669557+0000","last_change":"2024-03-30T00:32:12.678730+0000","last_active":"2024-03-30T00:33:15.669557+0000","last_peered":"2024-03-30T00:33:15.669557+0000","last_clean":"2024-03-30T00:33:15.669557+0000","last_became_active":"2024-03-30T00:32:12.678256+0000","last_became_peered":"2024-03-30T00:32:12.678256+0000","last_unstale":"2024-03-30T00:33:15.669557+0000","last_undegraded":"2024-03-30T00:33:15.669557+0000","last_fullsized":"2024-03-30T00:33:15.669557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_clean_scrub_stamp":"2024-03-30T00:29:17.768535+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:16.229072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000775+0000","last_change":"2024-03-30T00:33:15.685230+0000","last_active":"2024-03-30T00:33:17.000775+0000","last_peered":"2024-03-30T00:33:17.000775+0000","last_clean":"2024-03-30T00:33:17.000775+0000","last_became_active":"2024-03-30T00:33:15.684263+0000","last_became_peered":"2024-03-30T00:33:15.684263+0000","last_unstale":"2024-03-30T00:33:17.000775+0000","last_undegraded":"2024-03-30T00:33:17.000775+0000","last_fullsized":"2024-03-30T00:33:17.000775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:22:51.980318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.115562+0000","last_change":"2024-03-30T00:33:15.167490+0000","last_active":"2024-03-30T00:33:20.115562+0000","last_peered":"2024-03-30T00:33:20.115562+0000","last_clean":"2024-03-30T00:33:20.115562+0000","last_became_active":"2024-03-30T00:33:15.166888+0000","last_became_peered":"2024-03-30T00:33:15.166888+0000","last_unstale":"2024-03-30T00:33:20.115562+0000","last_undegraded":"2024-03-30T00:33:20.115562+0000","last_fullsized":"2024-03-30T00:33:20.115562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:00:32.571281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131306+0000","last_change":"2024-03-30T00:33:15.178388+0000","last_active":"2024-03-30T00:33:19.131306+0000","last_peered":"2024-03-30T00:33:19.131306+0000","last_clean":"2024-03-30T00:33:19.131306+0000","last_became_active":"2024-03-30T00:33:15.178160+0000","last_became_peered":"2024-03-30T00:33:15.178160+0000","last_unstale":"2024-03-30T00:33:19.131306+0000","last_undegraded":"2024-03-30T00:33:19.131306+0000","last_fullsized":"2024-03-30T00:33:19.131306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:28:51.567598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001200+0000","last_change":"2024-03-30T00:33:15.675426+0000","last_active":"2024-03-30T00:33:17.001200+0000","last_peered":"2024-03-30T00:33:17.001200+0000","last_clean":"2024-03-30T00:33:17.001200+0000","last_became_active":"2024-03-30T00:33:15.675083+0000","last_became_peered":"2024-03-30T00:33:15.675083+0000","last_unstale":"2024-03-30T00:33:17.001200+0000","last_undegraded":"2024-03-30T00:33:17.001200+0000","last_fullsized":"2024-03-30T00:33:17.001200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:37:03.086331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'6","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:28.599500+0000","last_change":"2024-03-30T00:33:15.171537+0000","last_active":"2024-03-30T00:33:28.599500+0000","last_peered":"2024-03-30T00:33:28.599500+0000","last_clean":"2024-03-30T00:33:28.599500+0000","last_became_active":"2024-03-30T00:33:15.171128+0000","last_became_peered":"2024-03-30T00:33:15.171128+0000","last_unstale":"2024-03-30T00:33:28.599500+0000","last_undegraded":"2024-03-30T00:33:28.599500+0000","last_fullsized":"2024-03-30T00:33:28.599500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:52.062144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001730+0000","last_change":"2024-03-30T00:33:15.674075+0000","last_active":"2024-03-30T00:33:17.001730+0000","last_peered":"2024-03-30T00:33:17.001730+0000","last_clean":"2024-03-30T00:33:17.001730+0000","last_became_active":"2024-03-30T00:33:15.672400+0000","last_became_peered":"2024-03-30T00:33:15.672400+0000","last_unstale":"2024-03-30T00:33:17.001730+0000","last_undegraded":"2024-03-30T00:33:17.001730+0000","last_fullsized":"2024-03-30T00:33:17.001730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:59.309835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131339+0000","last_change":"2024-03-30T00:33:15.168245+0000","last_active":"2024-03-30T00:33:19.131339+0000","last_peered":"2024-03-30T00:33:19.131339+0000","last_clean":"2024-03-30T00:33:19.131339+0000","last_became_active":"2024-03-30T00:33:15.167852+0000","last_became_peered":"2024-03-30T00:33:15.167852+0000","last_unstale":"2024-03-30T00:33:19.131339+0000","last_undegraded":"2024-03-30T00:33:19.131339+0000","last_fullsized":"2024-03-30T00:33:19.131339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:33:28.780614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002270+0000","last_change":"2024-03-30T00:33:15.679802+0000","last_active":"2024-03-30T00:33:17.002270+0000","last_peered":"2024-03-30T00:33:17.002270+0000","last_clean":"2024-03-30T00:33:17.002270+0000","last_became_active":"2024-03-30T00:33:15.679468+0000","last_became_peered":"2024-03-30T00:33:15.679468+0000","last_unstale":"2024-03-30T00:33:17.002270+0000","last_undegraded":"2024-03-30T00:33:17.002270+0000","last_fullsized":"2024-03-30T00:33:17.002270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:03:03.223713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131563+0000","last_change":"2024-03-30T00:33:15.170052+0000","last_active":"2024-03-30T00:33:19.131563+0000","last_peered":"2024-03-30T00:33:19.131563+0000","last_clean":"2024-03-30T00:33:19.131563+0000","last_became_active":"2024-03-30T00:33:15.169706+0000","last_became_peered":"2024-03-30T00:33:15.169706+0000","last_unstale":"2024-03-30T00:33:19.131563+0000","last_undegraded":"2024-03-30T00:33:19.131563+0000","last_fullsized":"2024-03-30T00:33:19.131563+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:32.043071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.668729+0000","last_change":"2024-03-30T00:33:15.667061+0000","last_active":"2024-03-30T00:33:15.668729+0000","last_peered":"2024-03-30T00:33:15.668729+0000","last_clean":"2024-03-30T00:33:15.668729+0000","last_became_active":"2024-03-30T00:33:15.666763+0000","last_became_peered":"2024-03-30T00:33:15.666763+0000","last_unstale":"2024-03-30T00:33:15.668729+0000","last_undegraded":"2024-03-30T00:33:15.668729+0000","last_fullsized":"2024-03-30T00:33:15.668729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:46.558297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.133165+0000","last_change":"2024-03-30T00:33:15.176152+0000","last_active":"2024-03-30T00:33:19.133165+0000","last_peered":"2024-03-30T00:33:19.133165+0000","last_clean":"2024-03-30T00:33:19.133165+0000","last_became_active":"2024-03-30T00:33:15.175640+0000","last_became_peered":"2024-03-30T00:33:15.175640+0000","last_unstale":"2024-03-30T00:33:19.133165+0000","last_undegraded":"2024-03-30T00:33:19.133165+0000","last_fullsized":"2024-03-30T00:33:19.133165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:58:11.575577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290545+0000","last_change":"2024-03-30T00:33:15.672957+0000","last_active":"2024-03-30T00:33:17.290545+0000","last_peered":"2024-03-30T00:33:17.290545+0000","last_clean":"2024-03-30T00:33:17.290545+0000","last_became_active":"2024-03-30T00:33:15.668868+0000","last_became_peered":"2024-03-30T00:33:15.668868+0000","last_unstale":"2024-03-30T00:33:17.290545+0000","last_undegraded":"2024-03-30T00:33:17.290545+0000","last_fullsized":"2024-03-30T00:33:17.290545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:50.342693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.787067+0000","last_change":"2024-03-30T00:33:15.160345+0000","last_active":"2024-03-30T00:33:16.787067+0000","last_peered":"2024-03-30T00:33:16.787067+0000","last_clean":"2024-03-30T00:33:16.787067+0000","last_became_active":"2024-03-30T00:33:15.160026+0000","last_became_peered":"2024-03-30T00:33:15.160026+0000","last_unstale":"2024-03-30T00:33:16.787067+0000","last_undegraded":"2024-03-30T00:33:16.787067+0000","last_fullsized":"2024-03-30T00:33:16.787067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:06:51.177680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001362+0000","last_change":"2024-03-30T00:33:15.674058+0000","last_active":"2024-03-30T00:33:17.001362+0000","last_peered":"2024-03-30T00:33:17.001362+0000","last_clean":"2024-03-30T00:33:17.001362+0000","last_became_active":"2024-03-30T00:33:15.673165+0000","last_became_peered":"2024-03-30T00:33:15.673165+0000","last_unstale":"2024-03-30T00:33:17.001362+0000","last_undegraded":"2024-03-30T00:33:17.001362+0000","last_fullsized":"2024-03-30T00:33:17.001362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:14.869963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.131675+0000","last_change":"2024-03-30T00:33:15.173178+0000","last_active":"2024-03-30T00:33:19.131675+0000","last_peered":"2024-03-30T00:33:19.131675+0000","last_clean":"2024-03-30T00:33:19.131675+0000","last_became_active":"2024-03-30T00:33:15.172830+0000","last_became_peered":"2024-03-30T00:33:15.172830+0000","last_unstale":"2024-03-30T00:33:19.131675+0000","last_undegraded":"2024-03-30T00:33:19.131675+0000","last_fullsized":"2024-03-30T00:33:19.131675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:49:09.823855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001435+0000","last_change":"2024-03-30T00:33:15.693485+0000","last_active":"2024-03-30T00:33:17.001435+0000","last_peered":"2024-03-30T00:33:17.001435+0000","last_clean":"2024-03-30T00:33:17.001435+0000","last_became_active":"2024-03-30T00:33:15.693066+0000","last_became_peered":"2024-03-30T00:33:15.693066+0000","last_unstale":"2024-03-30T00:33:17.001435+0000","last_undegraded":"2024-03-30T00:33:17.001435+0000","last_fullsized":"2024-03-30T00:33:17.001435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:38.161331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:20.118339+0000","last_change":"2024-03-30T00:33:15.170808+0000","last_active":"2024-03-30T00:33:20.118339+0000","last_peered":"2024-03-30T00:33:20.118339+0000","last_clean":"2024-03-30T00:33:20.118339+0000","last_became_active":"2024-03-30T00:33:15.170448+0000","last_became_peered":"2024-03-30T00:33:15.170448+0000","last_unstale":"2024-03-30T00:33:20.118339+0000","last_undegraded":"2024-03-30T00:33:20.118339+0000","last_fullsized":"2024-03-30T00:33:20.118339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:08:28.179041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001385+0000","last_change":"2024-03-30T00:33:15.687619+0000","last_active":"2024-03-30T00:33:17.001385+0000","last_peered":"2024-03-30T00:33:17.001385+0000","last_clean":"2024-03-30T00:33:17.001385+0000","last_became_active":"2024-03-30T00:33:15.687313+0000","last_became_peered":"2024-03-30T00:33:15.687313+0000","last_unstale":"2024-03-30T00:33:17.001385+0000","last_undegraded":"2024-03-30T00:33:17.001385+0000","last_fullsized":"2024-03-30T00:33:17.001385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:36:51.182558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001678+0000","last_change":"2024-03-30T00:33:15.168601+0000","last_active":"2024-03-30T00:33:17.001678+0000","last_peered":"2024-03-30T00:33:17.001678+0000","last_clean":"2024-03-30T00:33:17.001678+0000","last_became_active":"2024-03-30T00:33:15.168065+0000","last_became_peered":"2024-03-30T00:33:15.168065+0000","last_unstale":"2024-03-30T00:33:17.001678+0000","last_undegraded":"2024-03-30T00:33:17.001678+0000","last_fullsized":"2024-03-30T00:33:17.001678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:22:36.477280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:19.054439+0000","last_change":"2024-03-30T00:33:15.678299+0000","last_active":"2024-03-30T00:33:19.054439+0000","last_peered":"2024-03-30T00:33:19.054439+0000","last_clean":"2024-03-30T00:33:19.054439+0000","last_became_active":"2024-03-30T00:33:15.675499+0000","last_became_peered":"2024-03-30T00:33:15.675499+0000","last_unstale":"2024-03-30T00:33:19.054439+0000","last_undegraded":"2024-03-30T00:33:19.054439+0000","last_fullsized":"2024-03-30T00:33:19.054439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:15.317125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000985+0000","last_change":"2024-03-30T00:33:15.172058+0000","last_active":"2024-03-30T00:33:17.000985+0000","last_peered":"2024-03-30T00:33:17.000985+0000","last_clean":"2024-03-30T00:33:17.000985+0000","last_became_active":"2024-03-30T00:33:15.167857+0000","last_became_peered":"2024-03-30T00:33:15.167857+0000","last_unstale":"2024-03-30T00:33:17.000985+0000","last_undegraded":"2024-03-30T00:33:17.000985+0000","last_fullsized":"2024-03-30T00:33:17.000985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:03:35.951559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001350+0000","last_change":"2024-03-30T00:33:15.682519+0000","last_active":"2024-03-30T00:33:17.001350+0000","last_peered":"2024-03-30T00:33:17.001350+0000","last_clean":"2024-03-30T00:33:17.001350+0000","last_became_active":"2024-03-30T00:33:15.681055+0000","last_became_peered":"2024-03-30T00:33:15.681055+0000","last_unstale":"2024-03-30T00:33:17.001350+0000","last_undegraded":"2024-03-30T00:33:17.001350+0000","last_fullsized":"2024-03-30T00:33:17.001350+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:38.160151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290650+0000","last_change":"2024-03-30T00:33:15.673411+0000","last_active":"2024-03-30T00:33:17.290650+0000","last_peered":"2024-03-30T00:33:17.290650+0000","last_clean":"2024-03-30T00:33:17.290650+0000","last_became_active":"2024-03-30T00:33:15.672852+0000","last_became_peered":"2024-03-30T00:33:15.672852+0000","last_unstale":"2024-03-30T00:33:17.290650+0000","last_undegraded":"2024-03-30T00:33:17.290650+0000","last_fullsized":"2024-03-30T00:33:17.290650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:55:46.705785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660735+0000","last_change":"2024-03-30T00:33:15.680624+0000","last_active":"2024-03-30T00:33:16.660735+0000","last_peered":"2024-03-30T00:33:16.660735+0000","last_clean":"2024-03-30T00:33:16.660735+0000","last_became_active":"2024-03-30T00:33:15.679900+0000","last_became_peered":"2024-03-30T00:33:15.679900+0000","last_unstale":"2024-03-30T00:33:16.660735+0000","last_undegraded":"2024-03-30T00:33:16.660735+0000","last_fullsized":"2024-03-30T00:33:16.660735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:44.484608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.660521+0000","last_change":"2024-03-30T00:33:15.672612+0000","last_active":"2024-03-30T00:33:16.660521+0000","last_peered":"2024-03-30T00:33:16.660521+0000","last_clean":"2024-03-30T00:33:16.660521+0000","last_became_active":"2024-03-30T00:33:15.671536+0000","last_became_peered":"2024-03-30T00:33:15.671536+0000","last_unstale":"2024-03-30T00:33:16.660521+0000","last_undegraded":"2024-03-30T00:33:16.660521+0000","last_fullsized":"2024-03-30T00:33:16.660521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:40.783988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.002065+0000","last_change":"2024-03-30T00:33:15.681577+0000","last_active":"2024-03-30T00:33:17.002065+0000","last_peered":"2024-03-30T00:33:17.002065+0000","last_clean":"2024-03-30T00:33:17.002065+0000","last_became_active":"2024-03-30T00:33:15.681313+0000","last_became_peered":"2024-03-30T00:33:15.681313+0000","last_unstale":"2024-03-30T00:33:17.002065+0000","last_undegraded":"2024-03-30T00:33:17.002065+0000","last_fullsized":"2024-03-30T00:33:17.002065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:42:15.178957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001296+0000","last_change":"2024-03-30T00:33:15.688172+0000","last_active":"2024-03-30T00:33:17.001296+0000","last_peered":"2024-03-30T00:33:17.001296+0000","last_clean":"2024-03-30T00:33:17.001296+0000","last_became_active":"2024-03-30T00:33:15.687860+0000","last_became_peered":"2024-03-30T00:33:15.687860+0000","last_unstale":"2024-03-30T00:33:17.001296+0000","last_undegraded":"2024-03-30T00:33:17.001296+0000","last_fullsized":"2024-03-30T00:33:17.001296+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:28.762798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290827+0000","last_change":"2024-03-30T00:33:15.673868+0000","last_active":"2024-03-30T00:33:17.290827+0000","last_peered":"2024-03-30T00:33:17.290827+0000","last_clean":"2024-03-30T00:33:17.290827+0000","last_became_active":"2024-03-30T00:33:15.673467+0000","last_became_peered":"2024-03-30T00:33:15.673467+0000","last_unstale":"2024-03-30T00:33:17.290827+0000","last_undegraded":"2024-03-30T00:33:17.290827+0000","last_fullsized":"2024-03-30T00:33:17.290827+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:07:39.448967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.661014+0000","last_change":"2024-03-30T00:33:15.684639+0000","last_active":"2024-03-30T00:33:16.661014+0000","last_peered":"2024-03-30T00:33:16.661014+0000","last_clean":"2024-03-30T00:33:16.661014+0000","last_became_active":"2024-03-30T00:33:15.684357+0000","last_became_peered":"2024-03-30T00:33:15.684357+0000","last_unstale":"2024-03-30T00:33:16.661014+0000","last_undegraded":"2024-03-30T00:33:16.661014+0000","last_fullsized":"2024-03-30T00:33:16.661014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:02:50.995640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001730+0000","last_change":"2024-03-30T00:33:15.680810+0000","last_active":"2024-03-30T00:33:17.001730+0000","last_peered":"2024-03-30T00:33:17.001730+0000","last_clean":"2024-03-30T00:33:17.001730+0000","last_became_active":"2024-03-30T00:33:15.680457+0000","last_became_peered":"2024-03-30T00:33:15.680457+0000","last_unstale":"2024-03-30T00:33:17.001730+0000","last_undegraded":"2024-03-30T00:33:17.001730+0000","last_fullsized":"2024-03-30T00:33:17.001730+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:02.575683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:16.531880+0000","last_change":"2024-03-30T00:33:15.661644+0000","last_active":"2024-03-30T00:33:16.531880+0000","last_peered":"2024-03-30T00:33:16.531880+0000","last_clean":"2024-03-30T00:33:16.531880+0000","last_became_active":"2024-03-30T00:33:15.661359+0000","last_became_peered":"2024-03-30T00:33:15.661359+0000","last_unstale":"2024-03-30T00:33:16.531880+0000","last_undegraded":"2024-03-30T00:33:16.531880+0000","last_fullsized":"2024-03-30T00:33:16.531880+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:44:39.894342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.000646+0000","last_change":"2024-03-30T00:33:15.671491+0000","last_active":"2024-03-30T00:33:17.000646+0000","last_peered":"2024-03-30T00:33:17.000646+0000","last_clean":"2024-03-30T00:33:17.000646+0000","last_became_active":"2024-03-30T00:33:15.671110+0000","last_became_peered":"2024-03-30T00:33:15.671110+0000","last_unstale":"2024-03-30T00:33:17.000646+0000","last_undegraded":"2024-03-30T00:33:17.000646+0000","last_fullsized":"2024-03-30T00:33:17.000646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:57.623730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T00:33:15.693121+0000","last_change":"2024-03-30T00:33:15.686795+0000","last_active":"2024-03-30T00:33:15.693121+0000","last_peered":"2024-03-30T00:33:15.693121+0000","last_clean":"2024-03-30T00:33:15.693121+0000","last_became_active":"2024-03-30T00:33:15.685715+0000","last_became_peered":"2024-03-30T00:33:15.685715+0000","last_unstale":"2024-03-30T00:33:15.693121+0000","last_undegraded":"2024-03-30T00:33:15.693121+0000","last_fullsized":"2024-03-30T00:33:15.693121+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:37:53.838823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.001008+0000","last_change":"2024-03-30T00:33:15.687369+0000","last_active":"2024-03-30T00:33:17.001008+0000","last_peered":"2024-03-30T00:33:17.001008+0000","last_clean":"2024-03-30T00:33:17.001008+0000","last_became_active":"2024-03-30T00:33:15.686852+0000","last_became_peered":"2024-03-30T00:33:15.686852+0000","last_unstale":"2024-03-30T00:33:17.001008+0000","last_undegraded":"2024-03-30T00:33:17.001008+0000","last_fullsized":"2024-03-30T00:33:17.001008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:09:21.003739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T00:33:17.290614+0000","last_change":"2024-03-30T00:33:15.676840+0000","last_active":"2024-03-30T00:33:17.290614+0000","last_peered":"2024-03-30T00:33:17.290614+0000","last_clean":"2024-03-30T00:33:17.290614+0000","last_became_active":"2024-03-30T00:33:15.675884+0000","last_became_peered":"2024-03-30T00:33:15.675884+0000","last_unstale":"2024-03-30T00:33:17.290614+0000","last_undegraded":"2024-03-30T00:33:17.290614+0000","last_fullsized":"2024-03-30T00:33:17.290614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:02.900843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67785,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":264,"num_read_kb":204,"num_write":157,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449109,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36088,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709064,"statfs":{"total":95995035648,"available":95958081536,"internally_reserved":0,"allocated":901120,"data_stored":429559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.69099999999999995}]},{"osd":1,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.81699999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.878}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":6.6029999999999998,"5min":6.6029999999999998,"15min":6.6029999999999998},"last":0.77400000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.432},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.59899999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.75600000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.0649999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.73999999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":9,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":6.6399999999999997,"5min":6.6399999999999997,"15min":6.6399999999999997},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.91500000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.85299999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678042,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1084982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.76900000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.89200000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.81499999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.64100000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.80100000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.93700000000000006}]},{"osd":7,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.97999999999999998}]},{"osd":8,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.78900000000000003}]},{"osd":9,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.014},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.879}]},{"osd":11,"last update":"Sat Mar 30 00:33:33 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":0.91800000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841564,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":429795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71299999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.38300000000000001}]},{"osd":1,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.77800000000000002,"15min":0.78900000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0329999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.622},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.82299999999999995,"15min":0.83799999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.018,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.754}]},{"osd":2,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82799999999999996,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.82399999999999995}]},{"osd":3,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.95399999999999996,"5min":1.278,"15min":1.278},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.91800000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.504}]},{"osd":4,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.86499999999999999,"15min":0.88800000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98499999999999999,"5min":1.476,"15min":1.476},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.85699999999999998,"15min":0.878},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92900000000000005,"5min":1.254,"15min":1.254},"last":0.80400000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.87,"15min":0.88600000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.89600000000000002,"15min":0.92200000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94799999999999995,"5min":1.577,"15min":1.577},"last":0.95899999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.86899999999999999,"15min":0.89500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0600000000000001,"5min":1.486,"15min":1.486},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.90200000000000002,"15min":0.92900000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.073,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.93899999999999995,"15min":0.96299999999999997},"min":{"1min":0.53100000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0669999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.028,"15min":1.0620000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.083,"5min":2.234,"15min":2.234},"last":0.878}]},{"osd":8,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.998,"15min":1.024},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.095,"5min":1.611,"15min":1.611},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.0369999999999999,"15min":1.075},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.998,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.998}]},{"osd":10,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.93100000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103202,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":429715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81799999999999995,"5min":0.88,"15min":0.88},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.64400000000000002,"15min":0.65400000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.78800000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.68200000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.51900000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.873}]},{"osd":2,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.38900000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.72799999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63500000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.67100000000000004,"15min":0.68000000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.876,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.75900000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71899999999999997,"15min":0.71999999999999997},"min":{"1min":0.59799999999999998,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.83399999999999996,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76500000000000001,"15min":0.76600000000000001},"min":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0640000000000001,"5min":1.246,"15min":1.246},"last":0.72299999999999998}]},{"osd":5,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.026,"15min":1.026},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.46800000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.85399999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.80300000000000005,"15min":0.81799999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89100000000000001,"5min":1.054,"15min":1.054},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96699999999999997,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.82199999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.878,"15min":0.88900000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.04,"5min":1.155,"15min":1.155},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.84299999999999997,"15min":0.84699999999999998},"min":{"1min":0.627,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.99099999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90100000000000002}]},{"osd":9,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.86299999999999999,"15min":0.84199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.8969999999999998,"5min":3.8969999999999998,"15min":3.8969999999999998},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.8250000000000002,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":1.028}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":1.202}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]}]},{"osd":7,"up_from":50,"seq":214748364838,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":1021158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68500000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.58699999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.74299999999999999,"15min":0.73199999999999998},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.498}]},{"osd":4,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.67100000000000004,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.43099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.48199999999999998}]},{"osd":5,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.56499999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0760000000000001,"5min":1.095,"15min":1.095},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.79600000000000004,"15min":0.79100000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1679999999999999,"5min":1.252,"15min":1.252},"last":0.66400000000000003}]},{"osd":6,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.79900000000000004,"15min":0.79400000000000004},"min":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.73899999999999999}]},{"osd":8,"last update":"Sat Mar 30 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.83399999999999996,"15min":0.83299999999999996},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.90100000000000002}]},{"osd":9,"last update":"Sat Mar 30 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86899999999999999,"15min":0.877},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.028,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.85499999999999998,"15min":0.86899999999999999},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98799999999999999,"5min":1.284,"15min":1.284},"last":0.626}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.94699999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":905216,"data_stored":429184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64900000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77700000000000002,"5min":0.873,"15min":0.873},"last":0.46600000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.46000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61199999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91100000000000003,"5min":1.002,"15min":1.002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0069999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.64800000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.877,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.44600000000000001}]},{"osd":4,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.048,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.749,"15min":0.749},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.96899999999999997,"5min":1.238,"15min":1.238},"last":0.76200000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0329999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.55900000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1080000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.79300000000000004}]},{"osd":7,"last update":"Sat Mar 30 00:33:07 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.81599999999999995,"15min":0.81200000000000006},"min":{"1min":0.67000000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0169999999999999,"5min":1.105,"15min":1.105},"last":0.877},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.63100000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77700000000000002}]},{"osd":8,"last update":"Sat Mar 30 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.89300000000000002,"15min":0.90700000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1899999999999999,"5min":1.252,"15min":1.252},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":0.86699999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.80600000000000005}]},{"osd":9,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.84299999999999997,"15min":0.86499999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.72399999999999998}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.83599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36156,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708996,"statfs":{"total":95995035648,"available":95958011904,"internally_reserved":0,"allocated":970752,"data_stored":493248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64000000000000001,"15min":0.59699999999999998},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80100000000000005,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64400000000000002,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:33:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.61499999999999999,"15min":0.57099999999999995},"min":{"1min":0.46899999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0609999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.495},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65900000000000003,"15min":0.60199999999999998},"min":{"1min":0.42299999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0149999999999999,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.63}]},{"osd":3,"last update":"Sat Mar 30 00:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65100000000000002,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.125,"15min":1.125},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64000000000000001,"15min":0.59699999999999998},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.98899999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.71499999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.50600000000000001,"15min":0.48299999999999998},"min":{"1min":0.40999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.438},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.53700000000000003,"15min":0.505},"min":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.55600000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.66400000000000003,"15min":0.63800000000000001},"min":{"1min":0.50600000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.66300000000000003},"min":{"1min":0.53000000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96099999999999997,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.75700000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.746},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71499999999999997,"15min":0.69599999999999995},"min":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.079,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.83499999999999996}]},{"osd":7,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.64500000000000002,"15min":0.629},"min":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.65800000000000003,"15min":0.64700000000000002},"min":{"1min":0.504,"5min":0.438,"15min":0.438},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.85499999999999998}]},{"osd":8,"last update":"Sat Mar 30 00:33:30 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.76500000000000001,"15min":0.754},"min":{"1min":0.64200000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98899999999999999,"5min":1,"15min":1},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.88200000000000001,"15min":0.88300000000000001},"min":{"1min":0.63200000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0920000000000001,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.73499999999999999}]},{"osd":9,"last update":"Sat Mar 30 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.873,"15min":0.871},"min":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.125,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1200000000000001,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.86699999999999999}]},{"osd":10,"last update":"Sat Mar 30 00:33:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.90000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":925696,"data_stored":430455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:33:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59099999999999997,"15min":0.57199999999999995},"min":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66600000000000004,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96699999999999997,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.625}]},{"osd":2,"last update":"Sat Mar 30 00:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68999999999999995,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88400000000000001,"5min":5.8550000000000004,"15min":5.8550000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.72699999999999998,"15min":0.81299999999999994},"min":{"1min":0.41999999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.872,"5min":5.282,"15min":5.282},"last":0.63900000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.57699999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.47999999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.67900000000000005,"15min":0.65600000000000003},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.77700000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86399999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.85599999999999998,"15min":0.93500000000000005},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97899999999999998,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.54100000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.377,"15min":0.377},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.503},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.629,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.113,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.52500000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.125,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2470000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.81399999999999995}]},{"osd":8,"last update":"Sat Mar 30 00:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.77800000000000002,"15min":0.79800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.93000000000000005,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.85399999999999998,"15min":0.88100000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93899999999999995,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.71499999999999997}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.84599999999999997,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":1.161,"15min":1.161},"last":0.753}]},{"osd":10,"last update":"Sat Mar 30 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.81499999999999995}]},{"osd":11,"last update":"Sat Mar 30 00:33:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.76800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":995328,"data_stored":492898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.625,"15min":0.622},"min":{"1min":0.46000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89100000000000001,"5min":1.131,"15min":1.131},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.40799999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83499999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54300000000000004}]},{"osd":1,"last update":"Sat Mar 30 00:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60199999999999998,"15min":0.58499999999999996},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.65200000000000002,"15min":0.621},"min":{"1min":0.51200000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.91100000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68200000000000005,"15min":0.66800000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93799999999999994,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.45000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.65100000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.65700000000000003,"15min":0.61899999999999999},"min":{"1min":0.64900000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.93999999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.55100000000000005,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57599999999999996,"15min":0.55000000000000004},"min":{"1min":0.439,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96099999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57299999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77300000000000002,"15min":0.74299999999999999},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0309999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.879},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72099999999999997,"15min":0.69699999999999995},"min":{"1min":0.55900000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":2.944}]},{"osd":7,"last update":"Sat Mar 30 00:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.65700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.83299999999999996}]},{"osd":8,"last update":"Sat Mar 30 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73599999999999999,"15min":0.72899999999999998},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.748},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.83799999999999997,"15min":0.82499999999999996},"min":{"1min":0.47399999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.97899999999999998}]},{"osd":9,"last update":"Sat Mar 30 00:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95799999999999996,"15min":0.95999999999999996},"min":{"1min":0.68500000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.137,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.80900000000000005}]},{"osd":10,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.0760000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]}]},{"osd":3,"up_from":24,"seq":103079215158,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":429495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42499999999999999,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.378},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46800000000000003,"15min":0.437},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.36399999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.622,"15min":0.58599999999999997},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.82199999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.64200000000000002}]},{"osd":4,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64800000000000002,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.86699999999999999,"5min":0.996,"15min":0.996},"last":0.754},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.66400000000000003,"15min":0.63},"min":{"1min":0.58199999999999996,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.77100000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:33:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.72599999999999998,"15min":0.69299999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83599999999999997}]},{"osd":6,"last update":"Sat Mar 30 00:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.65500000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.46899999999999997}]},{"osd":7,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.879,"5min":0.81100000000000005,"15min":0.79900000000000004},"min":{"1min":0.63300000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.80700000000000005}]},{"osd":8,"last update":"Sat Mar 30 00:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.66700000000000004}]},{"osd":9,"last update":"Sat Mar 30 00:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.87,"15min":0.88400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.504},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0209999999999999,"5min":1.099,"15min":1.099},"last":0.48599999999999999}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":909312,"data_stored":429573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.497}]},{"osd":1,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.47799999999999998,"15min":0.44900000000000001},"min":{"1min":0.35799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.47199999999999998,"15min":0.45000000000000001},"min":{"1min":0.39600000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51600000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65800000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.67900000000000005,"15min":0.65100000000000002},"min":{"1min":0.58699999999999997,"5min":0.439,"15min":0.439},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.68700000000000006,"15min":0.66300000000000003},"min":{"1min":0.498,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:33:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68899999999999995,"15min":0.67700000000000005},"min":{"1min":0.42999999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94399999999999995,"5min":1.137,"15min":1.137},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.69699999999999995,"15min":0.67000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.82799999999999996}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0369999999999999,"5min":1.071,"15min":1.071},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82399999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.81299999999999994}]},{"osd":7,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.45600000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.52400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.88700000000000001}]},{"osd":8,"last update":"Sat Mar 30 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1619999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.78100000000000003,"15min":0.77300000000000002},"min":{"1min":0.63,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.94799999999999995}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.86499999999999999,"15min":0.88500000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97499999999999998,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.91200000000000003,"15min":0.92900000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0549999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.90400000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":1.042},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.96199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]}]},{"osd":5,"up_from":37,"seq":158913789999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32596,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712556,"statfs":{"total":95995035648,"available":95961657344,"internally_reserved":0,"allocated":1519616,"data_stored":1020559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.76400000000000001,"15min":0.82899999999999996},"min":{"1min":0.42199999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94799999999999995,"5min":8.077,"15min":8.077},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59699999999999998,"15min":0.57099999999999995},"min":{"1min":0.45900000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":3.121}]},{"osd":1,"last update":"Sat Mar 30 00:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.66000000000000003,"15min":0.63700000000000001},"min":{"1min":0.47199999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76900000000000002}]},{"osd":2,"last update":"Sat Mar 30 00:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.35499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65500000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.495},"min":{"1min":0.39100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.59099999999999997}]},{"osd":3,"last update":"Sat Mar 30 00:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"min":{"1min":0.47599999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84699999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.80700000000000005,"15min":0.88500000000000001},"min":{"1min":0.51000000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Mar 30 00:33:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.70199999999999996,"15min":0.66800000000000004},"min":{"1min":0.54100000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.53000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.754}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64700000000000002,"15min":0.629},"min":{"1min":0.56000000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84999999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.72399999999999998,"15min":0.68600000000000005},"min":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.68100000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:33:08 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.99399999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71299999999999997,"15min":0.70799999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.93899999999999995}]},{"osd":8,"last update":"Sat Mar 30 00:33:26 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.73699999999999999,"15min":0.71699999999999997},"min":{"1min":0.51400000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.85899999999999999}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.875,"15min":0.86899999999999999},"min":{"1min":0.70699999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.83499999999999996,"15min":0.83399999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.071,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.91300000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:33:10 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.97799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:34:01.762 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-30T00:34:01.762 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-30T00:34:01.763 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-30T00:34:01.763 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph health --format=json 2024-03-30T00:34:02.122 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:34:02.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:02 smithi099 ceph-mon[38079]: pgmap v279: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-30T00:34:02.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:02 smithi099 ceph-mon[38079]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:02.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:02 smithi142 ceph-mon[41779]: pgmap v279: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-30T00:34:02.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:02 smithi142 ceph-mon[41779]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:02.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:02 smithi161 ceph-mon[42221]: pgmap v279: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-30T00:34:02.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:02 smithi161 ceph-mon[42221]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:03.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.223+0000 7f97d2400700 1 -- 172.21.15.99:0/639123208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc100e60 msgr2=0x7f97cc1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:03.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.223+0000 7f97d2400700 1 --2- 172.21.15.99:0/639123208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc100e60 0x7f97cc1012e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f97b80099c0 tx=0x7f97b8009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:03.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.223+0000 7f97d2400700 1 -- 172.21.15.99:0/639123208 shutdown_connections 2024-03-30T00:34:03.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.223+0000 7f97d2400700 1 --2- 172.21.15.99:0/639123208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97cc101820 0x7f97cc10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:03.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.223+0000 7f97d2400700 1 --2- 172.21.15.99:0/639123208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97cc106ef0 0x7f97cc1072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:03.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.223+0000 7f97d2400700 1 --2- 172.21.15.99:0/639123208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc100e60 0x7f97cc1012e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:03.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.223+0000 7f97d2400700 1 -- 172.21.15.99:0/639123208 >> 172.21.15.99:0/639123208 conn(0x7f97cc077fa0 msgr2=0x7f97cc0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:03.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.223+0000 7f97d2400700 1 -- 172.21.15.99:0/639123208 shutdown_connections 2024-03-30T00:34:03.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.223+0000 7f97d2400700 1 -- 172.21.15.99:0/639123208 wait complete. 2024-03-30T00:34:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.224+0000 7f97d2400700 1 Processor -- start 2024-03-30T00:34:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.224+0000 7f97d2400700 1 -- start start 2024-03-30T00:34:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.224+0000 7f97d2400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97cc100e60 0x7f97cc199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.224+0000 7f97d2400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97cc101820 0x7f97cc199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.224+0000 7f97d2400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc106ef0 0x7f97cc19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.224+0000 7f97d2400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc19e210 con 0x7f97cc106ef0 2024-03-30T00:34:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.224+0000 7f97d2400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc19e380 con 0x7f97cc100e60 2024-03-30T00:34:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.224+0000 7f97d2400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc19e540 con 0x7f97cc101820 2024-03-30T00:34:03.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97cbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc106ef0 0x7f97cc19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97cbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc106ef0 0x7f97cc19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38862/0 (socket says 172.21.15.99:38862) 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97cbe00700 1 -- 172.21.15.99:0/3608404004 learned_addr learned my addr 172.21.15.99:0/3608404004 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97cb400700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97cc100e60 0x7f97cc199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97caa00700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97cc101820 0x7f97cc199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97cbe00700 1 -- 172.21.15.99:0/3608404004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97cc101820 msgr2=0x7f97cc199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97cbe00700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97cc101820 0x7f97cc199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97cbe00700 1 -- 172.21.15.99:0/3608404004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97cc100e60 msgr2=0x7f97cc199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97cbe00700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97cc100e60 0x7f97cc199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97cbe00700 1 -- 172.21.15.99:0/3608404004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b80095f0 con 0x7f97cc106ef0 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97caa00700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97cc101820 0x7f97cc199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.225+0000 7f97cbe00700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc106ef0 0x7f97cc19dc00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f97c400efc0 tx=0x7f97c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.226+0000 7f97c3e00700 1 -- 172.21.15.99:0/3608404004 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4018070 con 0x7f97cc106ef0 2024-03-30T00:34:03.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.226+0000 7f97c3e00700 1 -- 172.21.15.99:0/3608404004 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97c400f040 con 0x7f97cc106ef0 2024-03-30T00:34:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.226+0000 7f97d2400700 1 -- 172.21.15.99:0/3608404004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97cc19e7c0 con 0x7f97cc106ef0 2024-03-30T00:34:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.226+0000 7f97c3e00700 1 -- 172.21.15.99:0/3608404004 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400cbb0 con 0x7f97cc106ef0 2024-03-30T00:34:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.226+0000 7f97d2400700 1 -- 172.21.15.99:0/3608404004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc1a6080 con 0x7f97cc106ef0 2024-03-30T00:34:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.227+0000 7f97d2400700 1 -- 172.21.15.99:0/3608404004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97cc104180 con 0x7f97cc106ef0 2024-03-30T00:34:03.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.229+0000 7f97c3e00700 1 -- 172.21.15.99:0/3608404004 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f97c400cd90 con 0x7f97cc106ef0 2024-03-30T00:34:03.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.230+0000 7f97c3e00700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f97b00a1230 0x7f97b00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.230+0000 7f97c3e00700 1 -- 172.21.15.99:0/3608404004 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f97c40cc3b0 con 0x7f97cc106ef0 2024-03-30T00:34:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.230+0000 7f97cb400700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f97b00a1230 0x7f97b00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:03.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.231+0000 7f97cb400700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f97b00a1230 0x7f97b00a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f97bc005950 tx=0x7f97bc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:03.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.234+0000 7f97c3e00700 1 -- 172.21.15.99:0/3608404004 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f97c4095b20 con 0x7f97cc106ef0 2024-03-30T00:34:03.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.481+0000 7f97d2400700 1 -- 172.21.15.99:0/3608404004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f97cc1024f0 con 0x7f97cc106ef0 2024-03-30T00:34:03.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.481+0000 7f97c3e00700 1 -- 172.21.15.99:0/3608404004 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f97c4099140 con 0x7f97cc106ef0 2024-03-30T00:34:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:03.488 INFO:teuthology.orchestra.run.smithi099.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-30T00:34:03.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.483+0000 7f97d2400700 1 -- 172.21.15.99:0/3608404004 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f97b00a1230 msgr2=0x7f97b00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.483+0000 7f97d2400700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f97b00a1230 0x7f97b00a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f97bc005950 tx=0x7f97bc00b410 comp rx=0 tx=0).stop 2024-03-30T00:34:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.483+0000 7f97d2400700 1 -- 172.21.15.99:0/3608404004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc106ef0 msgr2=0x7f97cc19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.483+0000 7f97d2400700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc106ef0 0x7f97cc19dc00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f97c400efc0 tx=0x7f97c400c3a0 comp rx=0 tx=0).stop 2024-03-30T00:34:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.483+0000 7f97d2400700 1 -- 172.21.15.99:0/3608404004 shutdown_connections 2024-03-30T00:34:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.484+0000 7f97d2400700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97cc106ef0 0x7f97cc19dc00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.484+0000 7f97d2400700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f97b00a1230 0x7f97b00a36f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.484+0000 7f97d2400700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97cc100e60 0x7f97cc199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.484+0000 7f97d2400700 1 --2- 172.21.15.99:0/3608404004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97cc101820 0x7f97cc199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.484+0000 7f97d2400700 1 -- 172.21.15.99:0/3608404004 >> 172.21.15.99:0/3608404004 conn(0x7f97cc077fa0 msgr2=0x7f97cc10c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.484+0000 7f97d2400700 1 -- 172.21.15.99:0/3608404004 shutdown_connections 2024-03-30T00:34:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:03.484+0000 7f97d2400700 1 -- 172.21.15.99:0/3608404004 wait complete. 2024-03-30T00:34:03.851 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-30T00:34:03.852 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:34:04.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.041+0000 7f73a5a00700 1 -- 172.21.15.99:0/15767400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73a0100c80 msgr2=0x7f73a01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.041+0000 7f73a5a00700 1 --2- 172.21.15.99:0/15767400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73a0100c80 0x7f73a01010a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f73900099c0 tx=0x7f7390009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.041+0000 7f73a5a00700 1 -- 172.21.15.99:0/15767400 shutdown_connections 2024-03-30T00:34:04.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.041+0000 7f73a5a00700 1 --2- 172.21.15.99:0/15767400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a0102810 0x7f73a01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.041+0000 7f73a5a00700 1 --2- 172.21.15.99:0/15767400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73a0100c80 0x7f73a01010a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.041+0000 7f73a5a00700 1 --2- 172.21.15.99:0/15767400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a0101e70 0x7f73a01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.041+0000 7f73a5a00700 1 -- 172.21.15.99:0/15767400 >> 172.21.15.99:0/15767400 conn(0x7f73a00fc1f0 msgr2=0x7f73a00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:04.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.042+0000 7f73a5a00700 1 -- 172.21.15.99:0/15767400 shutdown_connections 2024-03-30T00:34:04.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.042+0000 7f73a5a00700 1 -- 172.21.15.99:0/15767400 wait complete. 2024-03-30T00:34:04.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.043+0000 7f73a5a00700 1 Processor -- start 2024-03-30T00:34:04.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.043+0000 7f73a5a00700 1 -- start start 2024-03-30T00:34:04.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.043+0000 7f73a5a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a0100c80 0x7f73a019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.043+0000 7f73a5a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a0101e70 0x7f73a019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.043+0000 7f73a5a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73a0102810 0x7f73a01a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.043+0000 7f73a5a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a01a4d00 con 0x7f73a0102810 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.043+0000 7f73a5a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a01a4e40 con 0x7f73a0100c80 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.043+0000 7f73a5a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a01a4fd0 con 0x7f73a0101e70 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f739f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73a0102810 0x7f73a01a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f739f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73a0102810 0x7f73a01a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38872/0 (socket says 172.21.15.99:38872) 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f739f400700 1 -- 172.21.15.99:0/3439351066 learned_addr learned my addr 172.21.15.99:0/3439351066 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f739ea00700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a0100c80 0x7f73a019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f739e000700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a0101e70 0x7f73a019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f739f400700 1 -- 172.21.15.99:0/3439351066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a0101e70 msgr2=0x7f73a019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f739f400700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a0101e70 0x7f73a019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f739f400700 1 -- 172.21.15.99:0/3439351066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a0100c80 msgr2=0x7f73a019d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f739f400700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a0100c80 0x7f73a019d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f739f400700 1 -- 172.21.15.99:0/3439351066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73900095f0 con 0x7f73a0102810 2024-03-30T00:34:04.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f739f400700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73a0102810 0x7f73a01a4730 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f739800ef30 tx=0x7f739800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:04.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f7397400700 1 -- 172.21.15.99:0/3439351066 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7398018070 con 0x7f73a0102810 2024-03-30T00:34:04.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.044+0000 7f7397400700 1 -- 172.21.15.99:0/3439351066 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f739800f040 con 0x7f73a0102810 2024-03-30T00:34:04.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.045+0000 7f7397400700 1 -- 172.21.15.99:0/3439351066 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739800cc70 con 0x7f73a0102810 2024-03-30T00:34:04.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.045+0000 7f73a5a00700 1 -- 172.21.15.99:0/3439351066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a01a5250 con 0x7f73a0102810 2024-03-30T00:34:04.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.045+0000 7f73a5a00700 1 -- 172.21.15.99:0/3439351066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a01a5760 con 0x7f73a0102810 2024-03-30T00:34:04.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.046+0000 7f73a5a00700 1 -- 172.21.15.99:0/3439351066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a01061b0 con 0x7f73a0102810 2024-03-30T00:34:04.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.047+0000 7f7397400700 1 -- 172.21.15.99:0/3439351066 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f7398007500 con 0x7f73a0102810 2024-03-30T00:34:04.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.049+0000 7f7397400700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f73840a1230 0x7f73840a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.049+0000 7f7397400700 1 -- 172.21.15.99:0/3439351066 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f73980cc370 con 0x7f73a0102810 2024-03-30T00:34:04.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.049+0000 7f739ea00700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f73840a1230 0x7f73840a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:04.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.049+0000 7f739ea00700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f73840a1230 0x7f73840a36f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7390009990 tx=0x7f7390003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:04.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.053+0000 7f7397400700 1 -- 172.21.15.99:0/3439351066 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7398095be0 con 0x7f73a0102810 2024-03-30T00:34:04.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.299+0000 7f73a5a00700 1 -- 172.21.15.99:0/3439351066 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73a010c410 con 0x7f73a0102810 2024-03-30T00:34:04.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.300+0000 7f7397400700 1 -- 172.21.15.99:0/3439351066 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22120 (secure 0 0 0) 0x7f7398099200 con 0x7f73a0102810 2024-03-30T00:34:04.308 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:04.308 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":83,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:33:16.651168+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:34:04.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 -- 172.21.15.99:0/3439351066 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f73840a1230 msgr2=0x7f73840a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f73840a1230 0x7f73840a36f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7390009990 tx=0x7f7390003680 comp rx=0 tx=0).stop 2024-03-30T00:34:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 -- 172.21.15.99:0/3439351066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73a0102810 msgr2=0x7f73a01a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73a0102810 0x7f73a01a4730 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f739800ef30 tx=0x7f739800c3a0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 -- 172.21.15.99:0/3439351066 shutdown_connections 2024-03-30T00:34:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73a0102810 0x7f73a01a4730 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f73840a1230 0x7f73840a36f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73a0100c80 0x7f73a019d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 --2- 172.21.15.99:0/3439351066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73a0101e70 0x7f73a019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 -- 172.21.15.99:0/3439351066 >> 172.21.15.99:0/3439351066 conn(0x7f73a00fc1f0 msgr2=0x7f73a01032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 -- 172.21.15.99:0/3439351066 shutdown_connections 2024-03-30T00:34:04.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.303+0000 7f73a5a00700 1 -- 172.21.15.99:0/3439351066 wait complete. 2024-03-30T00:34:04.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:33:14.085184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:33:15.090699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:34:04.330 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:34:04.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:04 smithi099 ceph-mon[38079]: pgmap v280: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-30T00:34:04.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:04 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3608404004' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-30T00:34:04.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.519+0000 7ff2cd600700 1 -- 172.21.15.99:0/1284924486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2c8102910 msgr2=0x7ff2c81091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.519+0000 7ff2cd600700 1 --2- 172.21.15.99:0/1284924486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2c8102910 0x7ff2c81091c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000b0d0 tx=0x7ff2c000b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.519+0000 7ff2cd600700 1 -- 172.21.15.99:0/1284924486 shutdown_connections 2024-03-30T00:34:04.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.519+0000 7ff2cd600700 1 --2- 172.21.15.99:0/1284924486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2c8102910 0x7ff2c81091c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.519+0000 7ff2cd600700 1 --2- 172.21.15.99:0/1284924486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c8100d80 0x7ff2c81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.519+0000 7ff2cd600700 1 --2- 172.21.15.99:0/1284924486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c8101f70 0x7ff2c81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.519+0000 7ff2cd600700 1 -- 172.21.15.99:0/1284924486 >> 172.21.15.99:0/1284924486 conn(0x7ff2c80fc2f0 msgr2=0x7ff2c80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:04.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.519+0000 7ff2cd600700 1 -- 172.21.15.99:0/1284924486 shutdown_connections 2024-03-30T00:34:04.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.519+0000 7ff2cd600700 1 -- 172.21.15.99:0/1284924486 wait complete. 2024-03-30T00:34:04.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.520+0000 7ff2cd600700 1 Processor -- start 2024-03-30T00:34:04.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2cd600700 1 -- start start 2024-03-30T00:34:04.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2cd600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c8100d80 0x7ff2c8198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2cd600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2c8101f70 0x7ff2c8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2cd600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c8102910 0x7ff2c81a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2cd600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c81a0a50 con 0x7ff2c8101f70 2024-03-30T00:34:04.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2cd600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c81a0b90 con 0x7ff2c8102910 2024-03-30T00:34:04.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2cd600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c81a0d20 con 0x7ff2c8100d80 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c6000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2c8101f70 0x7ff2c8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c6a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c8100d80 0x7ff2c8198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c6000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2c8101f70 0x7ff2c8199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38882/0 (socket says 172.21.15.99:38882) 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c6000700 1 -- 172.21.15.99:0/3665079270 learned_addr learned my addr 172.21.15.99:0/3665079270 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c7400700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c8102910 0x7ff2c81a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c6000700 1 -- 172.21.15.99:0/3665079270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c8100d80 msgr2=0x7ff2c8198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c6000700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c8100d80 0x7ff2c8198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c6000700 1 -- 172.21.15.99:0/3665079270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c8102910 msgr2=0x7ff2c81a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c6000700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c8102910 0x7ff2c81a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c6000700 1 -- 172.21.15.99:0/3665079270 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c0009d00 con 0x7ff2c8101f70 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c7400700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c8102910 0x7ff2c81a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.521+0000 7ff2c6a00700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c8100d80 0x7ff2c8198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.522+0000 7ff2c6000700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2c8101f70 0x7ff2c8199460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400d950 tx=0x7ff2b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.522+0000 7ff2bf400700 1 -- 172.21.15.99:0/3665079270 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4009940 con 0x7ff2c8101f70 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.522+0000 7ff2bf400700 1 -- 172.21.15.99:0/3665079270 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2b4010460 con 0x7ff2c8101f70 2024-03-30T00:34:04.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.522+0000 7ff2cd600700 1 -- 172.21.15.99:0/3665079270 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c81a0fa0 con 0x7ff2c8101f70 2024-03-30T00:34:04.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.522+0000 7ff2bf400700 1 -- 172.21.15.99:0/3665079270 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b400f620 con 0x7ff2c8101f70 2024-03-30T00:34:04.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.522+0000 7ff2cd600700 1 -- 172.21.15.99:0/3665079270 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c81a14d0 con 0x7ff2c8101f70 2024-03-30T00:34:04.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.523+0000 7ff2cd600700 1 -- 172.21.15.99:0/3665079270 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c8067af0 con 0x7ff2c8101f70 2024-03-30T00:34:04.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.525+0000 7ff2bf400700 1 -- 172.21.15.99:0/3665079270 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff2b40105d0 con 0x7ff2c8101f70 2024-03-30T00:34:04.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.527+0000 7ff2bf400700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2ac0a1230 0x7ff2ac0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.527+0000 7ff2bf400700 1 -- 172.21.15.99:0/3665079270 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff2b40cc680 con 0x7ff2c8101f70 2024-03-30T00:34:04.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.527+0000 7ff2c6a00700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2ac0a1230 0x7ff2ac0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:04.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.528+0000 7ff2c6a00700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2ac0a1230 0x7ff2ac0a36f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8101dd0 tx=0x7ff2b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:04.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.531+0000 7ff2bf400700 1 -- 172.21.15.99:0/3665079270 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff2b4095e60 con 0x7ff2c8101f70 2024-03-30T00:34:04.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:04 smithi142 ceph-mon[41779]: pgmap v280: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-30T00:34:04.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:04 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3608404004' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-30T00:34:04.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:04 smithi161 ceph-mon[42221]: pgmap v280: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-30T00:34:04.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:04 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3608404004' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-30T00:34:04.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.734+0000 7ff2cd600700 1 -- 172.21.15.99:0/3665079270 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2c810c450 con 0x7ff2c8101f70 2024-03-30T00:34:04.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.734+0000 7ff2bf400700 1 -- 172.21.15.99:0/3665079270 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2b4099480 con 0x7ff2c8101f70 2024-03-30T00:34:04.741 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.736+0000 7ff2cd600700 1 -- 172.21.15.99:0/3665079270 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2ac0a1230 msgr2=0x7ff2ac0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.736+0000 7ff2cd600700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2ac0a1230 0x7ff2ac0a36f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8101dd0 tx=0x7ff2b8009260 comp rx=0 tx=0).stop 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.736+0000 7ff2cd600700 1 -- 172.21.15.99:0/3665079270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2c8101f70 msgr2=0x7ff2c8199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.736+0000 7ff2cd600700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2c8101f70 0x7ff2c8199460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400d950 tx=0x7ff2b400dc60 comp rx=0 tx=0).stop 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.737+0000 7ff2cd600700 1 -- 172.21.15.99:0/3665079270 shutdown_connections 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.737+0000 7ff2cd600700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2ac0a1230 0x7ff2ac0a36f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.737+0000 7ff2cd600700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c8102910 0x7ff2c81a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.737+0000 7ff2cd600700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c8100d80 0x7ff2c8198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.737+0000 7ff2cd600700 1 --2- 172.21.15.99:0/3665079270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2c8101f70 0x7ff2c8199460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.737+0000 7ff2cd600700 1 -- 172.21.15.99:0/3665079270 >> 172.21.15.99:0/3665079270 conn(0x7ff2c80fc2f0 msgr2=0x7ff2c8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.737+0000 7ff2cd600700 1 -- 172.21.15.99:0/3665079270 shutdown_connections 2024-03-30T00:34:04.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.737+0000 7ff2cd600700 1 -- 172.21.15.99:0/3665079270 wait complete. 2024-03-30T00:34:04.761 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:34:04.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.950+0000 7f098ee00700 1 -- 172.21.15.99:0/4263972173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0988106a70 msgr2=0x7f0988108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.950+0000 7f098ee00700 1 --2- 172.21.15.99:0/4263972173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0988106a70 0x7f0988108ee0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f098000b0a0 tx=0x7f098000b3b0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.950+0000 7f098ee00700 1 -- 172.21.15.99:0/4263972173 shutdown_connections 2024-03-30T00:34:04.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.950+0000 7f098ee00700 1 --2- 172.21.15.99:0/4263972173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0988106a70 0x7f0988108ee0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.950+0000 7f098ee00700 1 --2- 172.21.15.99:0/4263972173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0988101810 0x7f0988103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.950+0000 7f098ee00700 1 --2- 172.21.15.99:0/4263972173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0988104140 0x7f0988106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.950+0000 7f098ee00700 1 -- 172.21.15.99:0/4263972173 >> 172.21.15.99:0/4263972173 conn(0x7f09880fb200 msgr2=0x7f09880fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:04.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.950+0000 7f098ee00700 1 -- 172.21.15.99:0/4263972173 shutdown_connections 2024-03-30T00:34:04.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.950+0000 7f098ee00700 1 -- 172.21.15.99:0/4263972173 wait complete. 2024-03-30T00:34:04.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.951+0000 7f098ee00700 1 Processor -- start 2024-03-30T00:34:04.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.951+0000 7f098ee00700 1 -- start start 2024-03-30T00:34:04.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.951+0000 7f098ee00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0988101810 0x7f0988101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.951+0000 7f098ee00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0988104140 0x7f09880ff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.951+0000 7f098ee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0988106a70 0x7f09880ffd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.951+0000 7f098ee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09881004a0 con 0x7f0988106a70 2024-03-30T00:34:04.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.951+0000 7f098ee00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0988100610 con 0x7f0988101810 2024-03-30T00:34:04.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.951+0000 7f098ee00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09881a0d60 con 0x7f0988104140 2024-03-30T00:34:04.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.952+0000 7f098ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0988106a70 0x7f09880ffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:04.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.952+0000 7f098ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0988106a70 0x7f09880ffd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36184/0 (socket says 172.21.15.99:36184) 2024-03-30T00:34:04.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.952+0000 7f098ce00700 1 -- 172.21.15.99:0/3409988583 learned_addr learned my addr 172.21.15.99:0/3409988583 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:04.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.952+0000 7f0987e00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0988101810 0x7f0988101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:04.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.952+0000 7f098ce00700 1 -- 172.21.15.99:0/3409988583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0988104140 msgr2=0x7f09880ff7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.952+0000 7f098ce00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0988104140 0x7f09880ff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.952+0000 7f098ce00700 1 -- 172.21.15.99:0/3409988583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0988101810 msgr2=0x7f0988101130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.952+0000 7f098ce00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0988101810 0x7f0988101130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.952+0000 7f098ce00700 1 -- 172.21.15.99:0/3409988583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0980009d00 con 0x7f0988106a70 2024-03-30T00:34:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.952+0000 7f098ce00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0988106a70 0x7f09880ffd00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0980007a30 tx=0x7f0980004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.952+0000 7f0984c00700 1 -- 172.21.15.99:0/3409988583 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098000e040 con 0x7f0988106a70 2024-03-30T00:34:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.953+0000 7f0984c00700 1 -- 172.21.15.99:0/3409988583 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09800056e0 con 0x7f0988106a70 2024-03-30T00:34:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.953+0000 7f098ee00700 1 -- 172.21.15.99:0/3409988583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09881a0f90 con 0x7f0988106a70 2024-03-30T00:34:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.953+0000 7f0984c00700 1 -- 172.21.15.99:0/3409988583 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098001db00 con 0x7f0988106a70 2024-03-30T00:34:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.953+0000 7f098ee00700 1 -- 172.21.15.99:0/3409988583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09881a1350 con 0x7f0988106a70 2024-03-30T00:34:04.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.954+0000 7f098ee00700 1 -- 172.21.15.99:0/3409988583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09880fcd80 con 0x7f0988106a70 2024-03-30T00:34:04.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.956+0000 7f0984c00700 1 -- 172.21.15.99:0/3409988583 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f0980005250 con 0x7f0988106a70 2024-03-30T00:34:04.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.958+0000 7f0984c00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f096c0a1230 0x7f096c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:04.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.958+0000 7f0984c00700 1 -- 172.21.15.99:0/3409988583 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f09800cee40 con 0x7f0988106a70 2024-03-30T00:34:04.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.958+0000 7f0987e00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f096c0a1230 0x7f096c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:04.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.959+0000 7f0987e00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f096c0a1230 0x7f096c0a36f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0978005950 tx=0x7f09780058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:04.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:04.961+0000 7f0984c00700 1 -- 172.21.15.99:0/3409988583 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f09800985b0 con 0x7f0988106a70 2024-03-30T00:34:05.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.172+0000 7f098ee00700 1 -- 172.21.15.99:0/3409988583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f098810c580 con 0x7f0988106a70 2024-03-30T00:34:05.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.172+0000 7f0984c00700 1 -- 172.21.15.99:0/3409988583 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f098009bbd0 con 0x7f0988106a70 2024-03-30T00:34:05.179 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-30T00:34:05.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 -- 172.21.15.99:0/3409988583 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f096c0a1230 msgr2=0x7f096c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:05.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f096c0a1230 0x7f096c0a36f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0978005950 tx=0x7f09780058e0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 -- 172.21.15.99:0/3409988583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0988106a70 msgr2=0x7f09880ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:05.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0988106a70 0x7f09880ffd00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0980007a30 tx=0x7f0980004d30 comp rx=0 tx=0).stop 2024-03-30T00:34:05.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 -- 172.21.15.99:0/3409988583 shutdown_connections 2024-03-30T00:34:05.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0988106a70 0x7f09880ffd00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f096c0a1230 0x7f096c0a36f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0988101810 0x7f0988101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 --2- 172.21.15.99:0/3409988583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0988104140 0x7f09880ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 -- 172.21.15.99:0/3409988583 >> 172.21.15.99:0/3409988583 conn(0x7f09880fb200 msgr2=0x7f0988102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:05.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 -- 172.21.15.99:0/3409988583 shutdown_connections 2024-03-30T00:34:05.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.175+0000 7f098ee00700 1 -- 172.21.15.99:0/3409988583 wait complete. 2024-03-30T00:34:05.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:34:05.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.389+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3738816085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6a0102910 msgr2=0x7fa6a01091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.389+0000 7fa6a5c00700 1 --2- 172.21.15.99:0/3738816085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6a0102910 0x7fa6a01091c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa69800b0d0 tx=0x7fa69800b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.389+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3738816085 shutdown_connections 2024-03-30T00:34:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.389+0000 7fa6a5c00700 1 --2- 172.21.15.99:0/3738816085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6a0102910 0x7fa6a01091c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.389+0000 7fa6a5c00700 1 --2- 172.21.15.99:0/3738816085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6a0100d80 0x7fa6a01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.389+0000 7fa6a5c00700 1 --2- 172.21.15.99:0/3738816085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6a0101f70 0x7fa6a01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.389+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3738816085 >> 172.21.15.99:0/3738816085 conn(0x7fa6a00fc2f0 msgr2=0x7fa6a00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.389+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3738816085 shutdown_connections 2024-03-30T00:34:05.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.389+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3738816085 wait complete. 2024-03-30T00:34:05.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.390+0000 7fa6a5c00700 1 Processor -- start 2024-03-30T00:34:05.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa6a5c00700 1 -- start start 2024-03-30T00:34:05.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa6a5c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6a0100d80 0x7fa6a0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:05.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa6a5c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6a0101f70 0x7fa6a00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa6a5c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6a0102910 0x7fa6a00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa6a5c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a0073e80 con 0x7fa6a0101f70 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa6a5c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a0073ff0 con 0x7fa6a0100d80 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa6a5c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a01a0e20 con 0x7fa6a0102910 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa69e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6a0101f70 0x7fa6a00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa69ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6a0100d80 0x7fa6a0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa69e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6a0101f70 0x7fa6a00731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36196/0 (socket says 172.21.15.99:36196) 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa69e000700 1 -- 172.21.15.99:0/3102496651 learned_addr learned my addr 172.21.15.99:0/3102496651 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.391+0000 7fa69f400700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6a0102910 0x7fa6a00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.392+0000 7fa69e000700 1 -- 172.21.15.99:0/3102496651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6a0102910 msgr2=0x7fa6a00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.392+0000 7fa69e000700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6a0102910 0x7fa6a00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.392+0000 7fa69e000700 1 -- 172.21.15.99:0/3102496651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6a0100d80 msgr2=0x7fa6a0074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.392+0000 7fa69e000700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6a0100d80 0x7fa6a0074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.392+0000 7fa69e000700 1 -- 172.21.15.99:0/3102496651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa698009d00 con 0x7fa6a0101f70 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.392+0000 7fa69e000700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6a0101f70 0x7fa6a00731a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00b530 tx=0x7fa68c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:05.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.392+0000 7fa697400700 1 -- 172.21.15.99:0/3102496651 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c011830 con 0x7fa6a0101f70 2024-03-30T00:34:05.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.392+0000 7fa697400700 1 -- 172.21.15.99:0/3102496651 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa68c011e20 con 0x7fa6a0101f70 2024-03-30T00:34:05.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.392+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3102496651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a01a10b0 con 0x7fa6a0101f70 2024-03-30T00:34:05.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.392+0000 7fa697400700 1 -- 172.21.15.99:0/3102496651 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c010dd0 con 0x7fa6a0101f70 2024-03-30T00:34:05.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.392+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3102496651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a01a1610 con 0x7fa6a0101f70 2024-03-30T00:34:05.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.393+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3102496651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a0108200 con 0x7fa6a0101f70 2024-03-30T00:34:05.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.395+0000 7fa697400700 1 -- 172.21.15.99:0/3102496651 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa68c011990 con 0x7fa6a0101f70 2024-03-30T00:34:05.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.396+0000 7fa697400700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa6840a1230 0x7fa6840a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:05.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.396+0000 7fa697400700 1 -- 172.21.15.99:0/3102496651 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa68c0cc2e0 con 0x7fa6a0101f70 2024-03-30T00:34:05.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.396+0000 7fa69ea00700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa6840a1230 0x7fa6840a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:05.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.397+0000 7fa69ea00700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa6840a1230 0x7fa6840a36f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0101dd0 tx=0x7fa69000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:05.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.401+0000 7fa697400700 1 -- 172.21.15.99:0/3102496651 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa68c095ad0 con 0x7fa6a0101f70 2024-03-30T00:34:05.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:05 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3439351066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:05.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:05 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3665079270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:05.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:05 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3409988583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:05.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:05 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3439351066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:05.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:05 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3665079270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:05.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:05 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3409988583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:05.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.608+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3102496651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6a010c6a0 con 0x7fa6a0101f70 2024-03-30T00:34:05.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.609+0000 7fa697400700 1 -- 172.21.15.99:0/3102496651 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa68c0990f0 con 0x7fa6a0101f70 2024-03-30T00:34:05.616 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-30T00:34:05.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.611+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3102496651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa6840a1230 msgr2=0x7fa6840a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:05.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.611+0000 7fa6a5c00700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa6840a1230 0x7fa6840a36f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0101dd0 tx=0x7fa69000b410 comp rx=0 tx=0).stop 2024-03-30T00:34:05.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.611+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3102496651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6a0101f70 msgr2=0x7fa6a00731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:05.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.611+0000 7fa6a5c00700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6a0101f70 0x7fa6a00731a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00b530 tx=0x7fa68c00b840 comp rx=0 tx=0).stop 2024-03-30T00:34:05.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.612+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3102496651 shutdown_connections 2024-03-30T00:34:05.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.612+0000 7fa6a5c00700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6a0102910 0x7fa6a00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.612+0000 7fa6a5c00700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa6840a1230 0x7fa6840a36f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.612+0000 7fa6a5c00700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6a0100d80 0x7fa6a0074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.612+0000 7fa6a5c00700 1 --2- 172.21.15.99:0/3102496651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6a0101f70 0x7fa6a00731a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.612+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3102496651 >> 172.21.15.99:0/3102496651 conn(0x7fa6a00fc2f0 msgr2=0x7fa6a0105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:05.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.612+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3102496651 shutdown_connections 2024-03-30T00:34:05.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.612+0000 7fa6a5c00700 1 -- 172.21.15.99:0/3102496651 wait complete. 2024-03-30T00:34:05.637 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:34:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:05 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3439351066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:05 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3665079270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:05 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3409988583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.820+0000 7f08e8a00700 1 -- 172.21.15.99:0/1339569940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08e4101e70 msgr2=0x7f08e41022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.820+0000 7f08e8a00700 1 --2- 172.21.15.99:0/1339569940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08e4101e70 0x7f08e41022d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f08d00099c0 tx=0x7f08d0009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.820+0000 7f08e8a00700 1 -- 172.21.15.99:0/1339569940 shutdown_connections 2024-03-30T00:34:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.820+0000 7f08e8a00700 1 --2- 172.21.15.99:0/1339569940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e4102810 0x7f08e41090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.820+0000 7f08e8a00700 1 --2- 172.21.15.99:0/1339569940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08e4100c80 0x7f08e41010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.820+0000 7f08e8a00700 1 --2- 172.21.15.99:0/1339569940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08e4101e70 0x7f08e41022d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.820+0000 7f08e8a00700 1 -- 172.21.15.99:0/1339569940 >> 172.21.15.99:0/1339569940 conn(0x7f08e40fc1f0 msgr2=0x7f08e40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.820+0000 7f08e8a00700 1 -- 172.21.15.99:0/1339569940 shutdown_connections 2024-03-30T00:34:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.820+0000 7f08e8a00700 1 -- 172.21.15.99:0/1339569940 wait complete. 2024-03-30T00:34:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.821+0000 7f08e8a00700 1 Processor -- start 2024-03-30T00:34:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.821+0000 7f08e8a00700 1 -- start start 2024-03-30T00:34:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.821+0000 7f08e8a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08e4100c80 0x7f08e4196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.821+0000 7f08e8a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e4101e70 0x7f08e4197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.821+0000 7f08e8a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08e4102810 0x7f08e419e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.821+0000 7f08e8a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e419e640 con 0x7f08e4100c80 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.821+0000 7f08e8a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e419e780 con 0x7f08e4102810 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.821+0000 7f08e8a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e419e910 con 0x7f08e4101e70 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.821+0000 7f08e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08e4100c80 0x7f08e4196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.821+0000 7f08e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08e4100c80 0x7f08e4196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36212/0 (socket says 172.21.15.99:36212) 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08e1e00700 1 -- 172.21.15.99:0/1203510207 learned_addr learned my addr 172.21.15.99:0/1203510207 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08e2800700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08e4102810 0x7f08e419e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08e1400700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e4101e70 0x7f08e4197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08e1e00700 1 -- 172.21.15.99:0/1203510207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e4101e70 msgr2=0x7f08e4197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08e1e00700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e4101e70 0x7f08e4197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08e1e00700 1 -- 172.21.15.99:0/1203510207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08e4102810 msgr2=0x7f08e419e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:05.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08e1e00700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08e4102810 0x7f08e419e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:05.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08e1e00700 1 -- 172.21.15.99:0/1203510207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d00095f0 con 0x7f08e4100c80 2024-03-30T00:34:05.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08e1e00700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08e4100c80 0x7f08e4196b10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f08d400d980 tx=0x7f08d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:05.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08daa00700 1 -- 172.21.15.99:0/1203510207 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4004020 con 0x7f08e4100c80 2024-03-30T00:34:05.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08daa00700 1 -- 172.21.15.99:0/1203510207 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08d400f690 con 0x7f08e4100c80 2024-03-30T00:34:05.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08daa00700 1 -- 172.21.15.99:0/1203510207 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d40039e0 con 0x7f08e4100c80 2024-03-30T00:34:05.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.822+0000 7f08e8a00700 1 -- 172.21.15.99:0/1203510207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e419ebf0 con 0x7f08e4100c80 2024-03-30T00:34:05.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.823+0000 7f08e8a00700 1 -- 172.21.15.99:0/1203510207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e419f0b0 con 0x7f08e4100c80 2024-03-30T00:34:05.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.823+0000 7f08e8a00700 1 -- 172.21.15.99:0/1203510207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e4108150 con 0x7f08e4100c80 2024-03-30T00:34:05.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.825+0000 7f08daa00700 1 -- 172.21.15.99:0/1203510207 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f08d400f800 con 0x7f08e4100c80 2024-03-30T00:34:05.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.827+0000 7f08daa00700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08c80a1230 0x7f08c80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:05.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.827+0000 7f08daa00700 1 -- 172.21.15.99:0/1203510207 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f08d40cc740 con 0x7f08e4100c80 2024-03-30T00:34:05.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.827+0000 7f08e1400700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08c80a1230 0x7f08c80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:05.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.827+0000 7f08e1400700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08c80a1230 0x7f08c80a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009990 tx=0x7f08d0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:05.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:05.829+0000 7f08daa00700 1 -- 172.21.15.99:0/1203510207 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f08d4095fa0 con 0x7f08e4100c80 2024-03-30T00:34:06.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.081+0000 7f08e8a00700 1 -- 172.21.15.99:0/1203510207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08e4067a60 con 0x7f08e4100c80 2024-03-30T00:34:06.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.082+0000 7f08daa00700 1 -- 172.21.15.99:0/1203510207 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10860 (secure 0 0 0) 0x7f08d40995c0 con 0x7f08e4100c80 2024-03-30T00:34:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:06.089 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:33:21.119410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:34:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.084+0000 7f08e8a00700 1 -- 172.21.15.99:0/1203510207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08c80a1230 msgr2=0x7f08c80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.084+0000 7f08e8a00700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08c80a1230 0x7f08c80a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009990 tx=0x7f08d0000bc0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.084+0000 7f08e8a00700 1 -- 172.21.15.99:0/1203510207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08e4100c80 msgr2=0x7f08e4196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.084+0000 7f08e8a00700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08e4100c80 0x7f08e4196b10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f08d400d980 tx=0x7f08d400dc90 comp rx=0 tx=0).stop 2024-03-30T00:34:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.085+0000 7f08e8a00700 1 -- 172.21.15.99:0/1203510207 shutdown_connections 2024-03-30T00:34:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.085+0000 7f08e8a00700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08c80a1230 0x7f08c80a36f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.085+0000 7f08e8a00700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08e4102810 0x7f08e419e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.085+0000 7f08e8a00700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08e4100c80 0x7f08e4196b10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.085+0000 7f08e8a00700 1 --2- 172.21.15.99:0/1203510207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08e4101e70 0x7f08e4197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.085+0000 7f08e8a00700 1 -- 172.21.15.99:0/1203510207 >> 172.21.15.99:0/1203510207 conn(0x7f08e40fc1f0 msgr2=0x7f08e4105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.085+0000 7f08e8a00700 1 -- 172.21.15.99:0/1203510207 shutdown_connections 2024-03-30T00:34:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.085+0000 7f08e8a00700 1 -- 172.21.15.99:0/1203510207 wait complete. 2024-03-30T00:34:06.093 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2024-03-30T00:34:06.110 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:34:06.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.295+0000 7f78de800700 1 -- 172.21.15.99:0/2394610166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d8106940 msgr2=0x7f78d8108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.295+0000 7f78de800700 1 --2- 172.21.15.99:0/2394610166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d8106940 0x7f78d8108db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78d000b0a0 tx=0x7f78d000b3b0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.295+0000 7f78de800700 1 -- 172.21.15.99:0/2394610166 shutdown_connections 2024-03-30T00:34:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.295+0000 7f78de800700 1 --2- 172.21.15.99:0/2394610166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d8106940 0x7f78d8108db0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.295+0000 7f78de800700 1 --2- 172.21.15.99:0/2394610166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d81016e0 0x7f78d8103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.295+0000 7f78de800700 1 --2- 172.21.15.99:0/2394610166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d8104010 0x7f78d8106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.295+0000 7f78de800700 1 -- 172.21.15.99:0/2394610166 >> 172.21.15.99:0/2394610166 conn(0x7f78d80fb0d0 msgr2=0x7f78d80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.295+0000 7f78de800700 1 -- 172.21.15.99:0/2394610166 shutdown_connections 2024-03-30T00:34:06.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.295+0000 7f78de800700 1 -- 172.21.15.99:0/2394610166 wait complete. 2024-03-30T00:34:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.296+0000 7f78de800700 1 Processor -- start 2024-03-30T00:34:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.297+0000 7f78de800700 1 -- start start 2024-03-30T00:34:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.297+0000 7f78de800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d81016e0 0x7f78d8196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.297+0000 7f78de800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d8104010 0x7f78d8197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.297+0000 7f78de800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d8106940 0x7f78d819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.297+0000 7f78de800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d819e710 con 0x7f78d8106940 2024-03-30T00:34:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.297+0000 7f78de800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d819e850 con 0x7f78d8104010 2024-03-30T00:34:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.297+0000 7f78de800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d819e9e0 con 0x7f78d81016e0 2024-03-30T00:34:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.297+0000 7f78d7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d8106940 0x7f78d819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:06.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.297+0000 7f78d7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d8106940 0x7f78d819e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36238/0 (socket says 172.21.15.99:36238) 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.297+0000 7f78d7e00700 1 -- 172.21.15.99:0/3924327494 learned_addr learned my addr 172.21.15.99:0/3924327494 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78d6a00700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d8104010 0x7f78d8197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78d7400700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d81016e0 0x7f78d8196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78d7e00700 1 -- 172.21.15.99:0/3924327494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d81016e0 msgr2=0x7f78d8196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78d7e00700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d81016e0 0x7f78d8196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78d7e00700 1 -- 172.21.15.99:0/3924327494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d8104010 msgr2=0x7f78d8197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78d7e00700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d8104010 0x7f78d8197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78d7e00700 1 -- 172.21.15.99:0/3924327494 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0009d00 con 0x7f78d8106940 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78d7e00700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d8106940 0x7f78d819e140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f78d000b780 tx=0x7f78d000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78cfe00700 1 -- 172.21.15.99:0/3924327494 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d000e040 con 0x7f78d8106940 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78cfe00700 1 -- 172.21.15.99:0/3924327494 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f78d0003c00 con 0x7f78d8106940 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78de800700 1 -- 172.21.15.99:0/3924327494 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d819ec60 con 0x7f78d8106940 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78cfe00700 1 -- 172.21.15.99:0/3924327494 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00052a0 con 0x7f78d8106940 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.298+0000 7f78de800700 1 -- 172.21.15.99:0/3924327494 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d819f030 con 0x7f78d8106940 2024-03-30T00:34:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.299+0000 7f78de800700 1 -- 172.21.15.99:0/3924327494 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d8067a80 con 0x7f78d8106940 2024-03-30T00:34:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.301+0000 7f78cfe00700 1 -- 172.21.15.99:0/3924327494 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f78d0005400 con 0x7f78d8106940 2024-03-30T00:34:06.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.302+0000 7f78cfe00700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f78bc0a11b0 0x7f78bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:06.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.302+0000 7f78cfe00700 1 -- 172.21.15.99:0/3924327494 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f78d00cdcb0 con 0x7f78d8106940 2024-03-30T00:34:06.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.303+0000 7f78d7400700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f78bc0a11b0 0x7f78bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.303+0000 7f78d7400700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f78bc0a11b0 0x7f78bc0a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009bf0 tx=0x7f78c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.306+0000 7f78cfe00700 1 -- 172.21.15.99:0/3924327494 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f78d8067a80 con 0x7f78d8106940 2024-03-30T00:34:06.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:06 smithi099 ceph-mon[38079]: pgmap v281: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-30T00:34:06.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:06 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3102496651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:06.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:06 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1203510207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:06.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.546+0000 7f78de800700 1 -- 172.21.15.99:0/3924327494 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78d810c2f0 con 0x7f78d8106940 2024-03-30T00:34:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.547+0000 7f78cfe00700 1 -- 172.21.15.99:0/3924327494 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22120 (secure 0 0 0) 0x7f78d0097420 con 0x7f78d8106940 2024-03-30T00:34:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:06.555 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":83,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:33:16.651168+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 -- 172.21.15.99:0/3924327494 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f78bc0a11b0 msgr2=0x7f78bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f78bc0a11b0 0x7f78bc0a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009bf0 tx=0x7f78c8009260 comp rx=0 tx=0).stop 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 -- 172.21.15.99:0/3924327494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d8106940 msgr2=0x7f78d819e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d8106940 0x7f78d819e140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f78d000b780 tx=0x7f78d000bab0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 -- 172.21.15.99:0/3924327494 shutdown_connections 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d8106940 0x7f78d819e140 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78d81016e0 0x7f78d8196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f78bc0a11b0 0x7f78bc0a3670 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 --2- 172.21.15.99:0/3924327494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78d8104010 0x7f78d8197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 -- 172.21.15.99:0/3924327494 >> 172.21.15.99:0/3924327494 conn(0x7f78d80fb0d0 msgr2=0x7f78d8102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 -- 172.21.15.99:0/3924327494 shutdown_connections 2024-03-30T00:34:06.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.550+0000 7f78de800700 1 -- 172.21.15.99:0/3924327494 wait complete. 2024-03-30T00:34:06.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:06 smithi142 ceph-mon[41779]: pgmap v281: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-30T00:34:06.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:06 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3102496651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:06.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:06 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1203510207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:06.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:33:14.085184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:33:15.090699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:34:06.576 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:34:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:06 smithi161 ceph-mon[42221]: pgmap v281: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-30T00:34:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:06 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3102496651' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:06 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1203510207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.763+0000 7fb4eae00700 1 -- 172.21.15.99:0/1561664609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4100d80 msgr2=0x7fb4e41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.763+0000 7fb4eae00700 1 --2- 172.21.15.99:0/1561664609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4100d80 0x7fb4e41011a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40099c0 tx=0x7fb4d4009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.763+0000 7fb4eae00700 1 -- 172.21.15.99:0/1561664609 shutdown_connections 2024-03-30T00:34:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.763+0000 7fb4eae00700 1 --2- 172.21.15.99:0/1561664609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4102910 0x7fb4e41091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.763+0000 7fb4eae00700 1 --2- 172.21.15.99:0/1561664609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4100d80 0x7fb4e41011a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.763+0000 7fb4eae00700 1 --2- 172.21.15.99:0/1561664609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4e4101f70 0x7fb4e41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.763+0000 7fb4eae00700 1 -- 172.21.15.99:0/1561664609 >> 172.21.15.99:0/1561664609 conn(0x7fb4e40fc2f0 msgr2=0x7fb4e40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.763+0000 7fb4eae00700 1 -- 172.21.15.99:0/1561664609 shutdown_connections 2024-03-30T00:34:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.763+0000 7fb4eae00700 1 -- 172.21.15.99:0/1561664609 wait complete. 2024-03-30T00:34:06.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.764+0000 7fb4eae00700 1 Processor -- start 2024-03-30T00:34:06.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.764+0000 7fb4eae00700 1 -- start start 2024-03-30T00:34:06.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4eae00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4100d80 0x7fb4e4196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:06.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4eae00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4101f70 0x7fb4e41971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:06.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4eae00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4e4102910 0x7fb4e419e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:06.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4eae00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419e790 con 0x7fb4e4100d80 2024-03-30T00:34:06.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4eae00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419e900 con 0x7fb4e4102910 2024-03-30T00:34:06.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4eae00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419eac0 con 0x7fb4e4101f70 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4e3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4100d80 0x7fb4e4196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4e3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4100d80 0x7fb4e4196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36250/0 (socket says 172.21.15.99:36250) 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4e3e00700 1 -- 172.21.15.99:0/3945892182 learned_addr learned my addr 172.21.15.99:0/3945892182 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4e3400700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4101f70 0x7fb4e41971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4e3e00700 1 -- 172.21.15.99:0/3945892182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4101f70 msgr2=0x7fb4e41971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4e3e00700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4101f70 0x7fb4e41971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4e3e00700 1 -- 172.21.15.99:0/3945892182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4e4102910 msgr2=0x7fb4e419e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4e3e00700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4e4102910 0x7fb4e419e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.765+0000 7fb4e3e00700 1 -- 172.21.15.99:0/3945892182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d40095f0 con 0x7fb4e4100d80 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.766+0000 7fb4e3e00700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4100d80 0x7fb4e4196ca0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4000c00 tx=0x7fb4d400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.766+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3945892182 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d401d070 con 0x7fb4e4100d80 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.766+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3945892182 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4d4005030 con 0x7fb4e4100d80 2024-03-30T00:34:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.766+0000 7fb4eae00700 1 -- 172.21.15.99:0/3945892182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e419ed40 con 0x7fb4e4100d80 2024-03-30T00:34:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.766+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3945892182 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4004b10 con 0x7fb4e4100d80 2024-03-30T00:34:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.766+0000 7fb4eae00700 1 -- 172.21.15.99:0/3945892182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e419f1c0 con 0x7fb4e4100d80 2024-03-30T00:34:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.767+0000 7fb4eae00700 1 -- 172.21.15.99:0/3945892182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e41082a0 con 0x7fb4e4100d80 2024-03-30T00:34:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.769+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3945892182 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb4d4005430 con 0x7fb4e4100d80 2024-03-30T00:34:06.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.770+0000 7fb4e0c00700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb4c80a1230 0x7fb4c80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:06.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.770+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3945892182 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb4d40cd980 con 0x7fb4e4100d80 2024-03-30T00:34:06.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.770+0000 7fb4e3400700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb4c80a1230 0x7fb4c80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:06.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.771+0000 7fb4e3400700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb4c80a1230 0x7fb4c80a36f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009c60 tx=0x7fb4d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:06.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.772+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3945892182 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb4d4097210 con 0x7fb4e4100d80 2024-03-30T00:34:06.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.977+0000 7fb4eae00700 1 -- 172.21.15.99:0/3945892182 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4e410c620 con 0x7fb4e4100d80 2024-03-30T00:34:06.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.978+0000 7fb4e0c00700 1 -- 172.21.15.99:0/3945892182 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4d409a830 con 0x7fb4e4100d80 2024-03-30T00:34:06.985 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:34:06.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.980+0000 7fb4eae00700 1 -- 172.21.15.99:0/3945892182 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb4c80a1230 msgr2=0x7fb4c80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.980+0000 7fb4eae00700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb4c80a1230 0x7fb4c80a36f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009c60 tx=0x7fb4d0009260 comp rx=0 tx=0).stop 2024-03-30T00:34:06.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.980+0000 7fb4eae00700 1 -- 172.21.15.99:0/3945892182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4100d80 msgr2=0x7fb4e4196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:06.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.980+0000 7fb4eae00700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4100d80 0x7fb4e4196ca0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4000c00 tx=0x7fb4d400f690 comp rx=0 tx=0).stop 2024-03-30T00:34:06.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.980+0000 7fb4eae00700 1 -- 172.21.15.99:0/3945892182 shutdown_connections 2024-03-30T00:34:06.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.981+0000 7fb4eae00700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb4c80a1230 0x7fb4c80a36f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.981+0000 7fb4eae00700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4e4102910 0x7fb4e419e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.981+0000 7fb4eae00700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e4100d80 0x7fb4e4196ca0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.981+0000 7fb4eae00700 1 --2- 172.21.15.99:0/3945892182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4e4101f70 0x7fb4e41971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:06.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.981+0000 7fb4eae00700 1 -- 172.21.15.99:0/3945892182 >> 172.21.15.99:0/3945892182 conn(0x7fb4e40fc2f0 msgr2=0x7fb4e41053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:06.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.981+0000 7fb4eae00700 1 -- 172.21.15.99:0/3945892182 shutdown_connections 2024-03-30T00:34:06.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:06.981+0000 7fb4eae00700 1 -- 172.21.15.99:0/3945892182 wait complete. 2024-03-30T00:34:07.006 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:34:07.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.195+0000 7f9b1ea00700 1 -- 172.21.15.99:0/1114365979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18102910 msgr2=0x7f9b181091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:07.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.195+0000 7f9b1ea00700 1 --2- 172.21.15.99:0/1114365979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18102910 0x7f9b181091c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000b0d0 tx=0x7f9b1000b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.195+0000 7f9b1ea00700 1 -- 172.21.15.99:0/1114365979 shutdown_connections 2024-03-30T00:34:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.195+0000 7f9b1ea00700 1 --2- 172.21.15.99:0/1114365979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18102910 0x7f9b181091c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.195+0000 7f9b1ea00700 1 --2- 172.21.15.99:0/1114365979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b18100d80 0x7f9b181011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.195+0000 7f9b1ea00700 1 --2- 172.21.15.99:0/1114365979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b18101f70 0x7f9b181023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.195+0000 7f9b1ea00700 1 -- 172.21.15.99:0/1114365979 >> 172.21.15.99:0/1114365979 conn(0x7f9b180fc2f0 msgr2=0x7f9b180fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.195+0000 7f9b1ea00700 1 -- 172.21.15.99:0/1114365979 shutdown_connections 2024-03-30T00:34:07.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.195+0000 7f9b1ea00700 1 -- 172.21.15.99:0/1114365979 wait complete. 2024-03-30T00:34:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.196+0000 7f9b1ea00700 1 Processor -- start 2024-03-30T00:34:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.196+0000 7f9b1ea00700 1 -- start start 2024-03-30T00:34:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b1ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18100d80 0x7f9b18198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:07.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b1ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b18101f70 0x7f9b181994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b1ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b18102910 0x7f9b181a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b1ea00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b181a0a80 con 0x7f9b18100d80 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b1ea00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b181a0bc0 con 0x7f9b18101f70 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b1ea00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b181a0d50 con 0x7f9b18102910 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b17e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18100d80 0x7f9b18198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b17e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18100d80 0x7f9b18198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36258/0 (socket says 172.21.15.99:36258) 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b17e00700 1 -- 172.21.15.99:0/679123197 learned_addr learned my addr 172.21.15.99:0/679123197 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b17400700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b18101f70 0x7f9b181994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b1ca00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b18102910 0x7f9b181a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b17e00700 1 -- 172.21.15.99:0/679123197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b18102910 msgr2=0x7f9b181a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b17e00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b18102910 0x7f9b181a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b17e00700 1 -- 172.21.15.99:0/679123197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b18101f70 msgr2=0x7f9b181994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:07.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b17e00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b18101f70 0x7f9b181994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b17e00700 1 -- 172.21.15.99:0/679123197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b10009d00 con 0x7f9b18100d80 2024-03-30T00:34:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.197+0000 7f9b1ca00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b18102910 0x7f9b181a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.198+0000 7f9b17e00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18100d80 0x7f9b18198fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800e970 tx=0x7f9b0800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.198+0000 7f9b14c00700 1 -- 172.21.15.99:0/679123197 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08009790 con 0x7f9b18100d80 2024-03-30T00:34:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.198+0000 7f9b14c00700 1 -- 172.21.15.99:0/679123197 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b0800eec0 con 0x7f9b18100d80 2024-03-30T00:34:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.198+0000 7f9b14c00700 1 -- 172.21.15.99:0/679123197 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08003c10 con 0x7f9b18100d80 2024-03-30T00:34:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.198+0000 7f9b1ea00700 1 -- 172.21.15.99:0/679123197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b181a0fd0 con 0x7f9b18100d80 2024-03-30T00:34:07.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.198+0000 7f9b1ea00700 1 -- 172.21.15.99:0/679123197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b181a1400 con 0x7f9b18100d80 2024-03-30T00:34:07.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.199+0000 7f9b1ea00700 1 -- 172.21.15.99:0/679123197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b18067af0 con 0x7f9b18100d80 2024-03-30T00:34:07.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.201+0000 7f9b14c00700 1 -- 172.21.15.99:0/679123197 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f9b08003db0 con 0x7f9b18100d80 2024-03-30T00:34:07.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.202+0000 7f9b14c00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9afc0a1230 0x7f9afc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:07.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.202+0000 7f9b14c00700 1 -- 172.21.15.99:0/679123197 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9b08014070 con 0x7f9b18100d80 2024-03-30T00:34:07.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.202+0000 7f9b17400700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9afc0a1230 0x7f9afc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:07.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.203+0000 7f9b17400700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9afc0a1230 0x7f9afc0a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009c60 tx=0x7f9b04009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:07.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.206+0000 7f9b14c00700 1 -- 172.21.15.99:0/679123197 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9b08096270 con 0x7f9b18100d80 2024-03-30T00:34:07.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.413+0000 7f9b1ea00700 1 -- 172.21.15.99:0/679123197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b181a1650 con 0x7f9b18100d80 2024-03-30T00:34:07.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.413+0000 7f9b14c00700 1 -- 172.21.15.99:0/679123197 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b08099890 con 0x7f9b18100d80 2024-03-30T00:34:07.420 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-30T00:34:07.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 -- 172.21.15.99:0/679123197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9afc0a1230 msgr2=0x7f9afc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:07.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9afc0a1230 0x7f9afc0a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009c60 tx=0x7f9b04009260 comp rx=0 tx=0).stop 2024-03-30T00:34:07.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 -- 172.21.15.99:0/679123197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18100d80 msgr2=0x7f9b18198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:07.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18100d80 0x7f9b18198fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800e970 tx=0x7f9b0800ec80 comp rx=0 tx=0).stop 2024-03-30T00:34:07.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 -- 172.21.15.99:0/679123197 shutdown_connections 2024-03-30T00:34:07.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b18102910 0x7f9b181a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b18100d80 0x7f9b18198fa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9afc0a1230 0x7f9afc0a36f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 --2- 172.21.15.99:0/679123197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b18101f70 0x7f9b181994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 -- 172.21.15.99:0/679123197 >> 172.21.15.99:0/679123197 conn(0x7f9b180fc2f0 msgr2=0x7f9b18072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 -- 172.21.15.99:0/679123197 shutdown_connections 2024-03-30T00:34:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.416+0000 7f9b1ea00700 1 -- 172.21.15.99:0/679123197 wait complete. 2024-03-30T00:34:07.439 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:07 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3924327494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:07.439 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:07 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3945892182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:07.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:34:07.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:07 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3924327494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:07.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:07 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3945892182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:07.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.629+0000 7ff7eca00700 1 -- 172.21.15.99:0/4084590784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e8101f70 msgr2=0x7ff7e81023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:07.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.629+0000 7ff7eca00700 1 --2- 172.21.15.99:0/4084590784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e8101f70 0x7ff7e81023d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40099c0 tx=0x7ff7d4009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.630+0000 7ff7eca00700 1 -- 172.21.15.99:0/4084590784 shutdown_connections 2024-03-30T00:34:07.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.630+0000 7ff7eca00700 1 --2- 172.21.15.99:0/4084590784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8102910 0x7ff7e81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.630+0000 7ff7eca00700 1 --2- 172.21.15.99:0/4084590784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7e8100d80 0x7ff7e81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.630+0000 7ff7eca00700 1 --2- 172.21.15.99:0/4084590784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e8101f70 0x7ff7e81023d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.630+0000 7ff7eca00700 1 -- 172.21.15.99:0/4084590784 >> 172.21.15.99:0/4084590784 conn(0x7ff7e80fc2f0 msgr2=0x7ff7e80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:07.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.630+0000 7ff7eca00700 1 -- 172.21.15.99:0/4084590784 shutdown_connections 2024-03-30T00:34:07.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.630+0000 7ff7eca00700 1 -- 172.21.15.99:0/4084590784 wait complete. 2024-03-30T00:34:07.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7eca00700 1 Processor -- start 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7eca00700 1 -- start start 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7eca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8100d80 0x7ff7e8198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7eca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e8101f70 0x7ff7e8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7eca00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7e8102910 0x7ff7e81a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7eca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e81a0a00 con 0x7ff7e8101f70 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7eca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e81a0b40 con 0x7ff7e8100d80 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7eca00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e81a0cd0 con 0x7ff7e8102910 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7e5400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e8101f70 0x7ff7e8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7e5400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e8101f70 0x7ff7e8199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36280/0 (socket says 172.21.15.99:36280) 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7e5400700 1 -- 172.21.15.99:0/2292492118 learned_addr learned my addr 172.21.15.99:0/2292492118 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.631+0000 7ff7e5e00700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8100d80 0x7ff7e8198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7e5400700 1 -- 172.21.15.99:0/2292492118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7e8102910 msgr2=0x7ff7e81a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:07.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7e6800700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7e8102910 0x7ff7e81a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7e5400700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7e8102910 0x7ff7e81a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7e5400700 1 -- 172.21.15.99:0/2292492118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8100d80 msgr2=0x7ff7e8198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7e5400700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8100d80 0x7ff7e8198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7e5400700 1 -- 172.21.15.99:0/2292492118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40095f0 con 0x7ff7e8101f70 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7e5e00700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8100d80 0x7ff7e8198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7e6800700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7e8102910 0x7ff7e81a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7e5400700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e8101f70 0x7ff7e8199460 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009990 tx=0x7ff7d400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7dea00700 1 -- 172.21.15.99:0/2292492118 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d401d070 con 0x7ff7e8101f70 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7dea00700 1 -- 172.21.15.99:0/2292492118 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff7d400bd10 con 0x7ff7e8101f70 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7eca00700 1 -- 172.21.15.99:0/2292492118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e81a0f50 con 0x7ff7e8101f70 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7dea00700 1 -- 172.21.15.99:0/2292492118 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4017650 con 0x7ff7e8101f70 2024-03-30T00:34:07.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.632+0000 7ff7eca00700 1 -- 172.21.15.99:0/2292492118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e81a1310 con 0x7ff7e8101f70 2024-03-30T00:34:07.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.634+0000 7ff7eca00700 1 -- 172.21.15.99:0/2292492118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e8067af0 con 0x7ff7e8101f70 2024-03-30T00:34:07.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.635+0000 7ff7dea00700 1 -- 172.21.15.99:0/2292492118 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff7d4022470 con 0x7ff7e8101f70 2024-03-30T00:34:07.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.637+0000 7ff7dea00700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7cc0a11b0 0x7ff7cc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:07.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.637+0000 7ff7dea00700 1 -- 172.21.15.99:0/2292492118 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff7d40ce2e0 con 0x7ff7e8101f70 2024-03-30T00:34:07.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.637+0000 7ff7e5e00700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7cc0a11b0 0x7ff7cc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:07.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.637+0000 7ff7e5e00700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7cc0a11b0 0x7ff7cc0a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8101dd0 tx=0x7ff7d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:07.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.639+0000 7ff7dea00700 1 -- 172.21.15.99:0/2292492118 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff7d4097b50 con 0x7ff7e8101f70 2024-03-30T00:34:07.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:07 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3924327494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:07.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:07 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3945892182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:07.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.845+0000 7ff7eca00700 1 -- 172.21.15.99:0/2292492118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff7e810c6a0 con 0x7ff7e8101f70 2024-03-30T00:34:07.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.845+0000 7ff7dea00700 1 -- 172.21.15.99:0/2292492118 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff7d409b170 con 0x7ff7e8101f70 2024-03-30T00:34:07.852 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-30T00:34:07.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.847+0000 7ff7eca00700 1 -- 172.21.15.99:0/2292492118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7cc0a11b0 msgr2=0x7ff7cc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:07.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.847+0000 7ff7eca00700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7cc0a11b0 0x7ff7cc0a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8101dd0 tx=0x7ff7d8009260 comp rx=0 tx=0).stop 2024-03-30T00:34:07.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.847+0000 7ff7eca00700 1 -- 172.21.15.99:0/2292492118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e8101f70 msgr2=0x7ff7e8199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:07.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.847+0000 7ff7eca00700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e8101f70 0x7ff7e8199460 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009990 tx=0x7ff7d400fab0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.848+0000 7ff7eca00700 1 -- 172.21.15.99:0/2292492118 shutdown_connections 2024-03-30T00:34:07.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.848+0000 7ff7eca00700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7e8102910 0x7ff7e81a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.848+0000 7ff7eca00700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7cc0a11b0 0x7ff7cc0a3670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.848+0000 7ff7eca00700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e8100d80 0x7ff7e8198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.848+0000 7ff7eca00700 1 --2- 172.21.15.99:0/2292492118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7e8101f70 0x7ff7e8199460 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:07.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.848+0000 7ff7eca00700 1 -- 172.21.15.99:0/2292492118 >> 172.21.15.99:0/2292492118 conn(0x7ff7e80fc2f0 msgr2=0x7ff7e8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:07.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.848+0000 7ff7eca00700 1 -- 172.21.15.99:0/2292492118 shutdown_connections 2024-03-30T00:34:07.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:07.848+0000 7ff7eca00700 1 -- 172.21.15.99:0/2292492118 wait complete. 2024-03-30T00:34:07.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:34:08.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.058+0000 7f1cb4400700 1 -- 172.21.15.99:0/2631249009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac075770 msgr2=0x7f1cac075b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:08.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.058+0000 7f1cb4400700 1 --2- 172.21.15.99:0/2631249009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac075770 0x7f1cac075b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009990 tx=0x7f1ca0009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.059+0000 7f1cb4400700 1 -- 172.21.15.99:0/2631249009 shutdown_connections 2024-03-30T00:34:08.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.059+0000 7f1cb4400700 1 --2- 172.21.15.99:0/2631249009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cac077370 0x7f1cac077bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.059+0000 7f1cb4400700 1 --2- 172.21.15.99:0/2631249009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac075770 0x7f1cac075b90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.059+0000 7f1cb4400700 1 --2- 172.21.15.99:0/2631249009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac0769b0 0x7f1cac076e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.059+0000 7f1cb4400700 1 -- 172.21.15.99:0/2631249009 >> 172.21.15.99:0/2631249009 conn(0x7f1cac0fe420 msgr2=0x7f1cac100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:08.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.059+0000 7f1cb4400700 1 -- 172.21.15.99:0/2631249009 shutdown_connections 2024-03-30T00:34:08.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.059+0000 7f1cb4400700 1 -- 172.21.15.99:0/2631249009 wait complete. 2024-03-30T00:34:08.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb4400700 1 Processor -- start 2024-03-30T00:34:08.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb4400700 1 -- start start 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb4400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cac075770 0x7f1cac19d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb4400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac0769b0 0x7f1cac19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb4400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac077370 0x7f1cac1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb4400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac1a4c90 con 0x7f1cac077370 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb4400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac1a4dd0 con 0x7f1cac0769b0 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb4400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac1a4f60 con 0x7f1cac075770 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb2400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac077370 0x7f1cac1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb2400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac077370 0x7f1cac1a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36292/0 (socket says 172.21.15.99:36292) 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb2400700 1 -- 172.21.15.99:0/20823429 learned_addr learned my addr 172.21.15.99:0/20823429 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.060+0000 7f1cb1a00700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cac075770 0x7f1cac19d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1cb1000700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac0769b0 0x7f1cac19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1cb2400700 1 -- 172.21.15.99:0/20823429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cac075770 msgr2=0x7f1cac19d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:08.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1cb2400700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cac075770 0x7f1cac19d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1cb2400700 1 -- 172.21.15.99:0/20823429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac0769b0 msgr2=0x7f1cac19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:08.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1cb2400700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac0769b0 0x7f1cac19d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1cb2400700 1 -- 172.21.15.99:0/20823429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca00095f0 con 0x7f1cac077370 2024-03-30T00:34:08.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1cb2400700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac077370 0x7f1cac1a46c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4007ee0 tx=0x7f1ca400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:08.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1caa000700 1 -- 172.21.15.99:0/20823429 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca400dd20 con 0x7f1cac077370 2024-03-30T00:34:08.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1caa000700 1 -- 172.21.15.99:0/20823429 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ca400f040 con 0x7f1cac077370 2024-03-30T00:34:08.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1cb4400700 1 -- 172.21.15.99:0/20823429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac1a5240 con 0x7f1cac077370 2024-03-30T00:34:08.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1caa000700 1 -- 172.21.15.99:0/20823429 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4004750 con 0x7f1cac077370 2024-03-30T00:34:08.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.061+0000 7f1cb4400700 1 -- 172.21.15.99:0/20823429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac1a5750 con 0x7f1cac077370 2024-03-30T00:34:08.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.062+0000 7f1cb4400700 1 -- 172.21.15.99:0/20823429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac10fc10 con 0x7f1cac077370 2024-03-30T00:34:08.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.064+0000 7f1caa000700 1 -- 172.21.15.99:0/20823429 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1ca4003a00 con 0x7f1cac077370 2024-03-30T00:34:08.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.065+0000 7f1caa000700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1c940a11b0 0x7f1c940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:08.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.066+0000 7f1caa000700 1 -- 172.21.15.99:0/20823429 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1ca40cca30 con 0x7f1cac077370 2024-03-30T00:34:08.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.066+0000 7f1cb1a00700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1c940a11b0 0x7f1c940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:08.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.066+0000 7f1cb1a00700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1c940a11b0 0x7f1c940a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0005910 tx=0x7f1ca000b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:08.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.068+0000 7f1caa000700 1 -- 172.21.15.99:0/20823429 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1ca4096110 con 0x7f1cac077370 2024-03-30T00:34:08.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.319+0000 7f1cb4400700 1 -- 172.21.15.99:0/20823429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cac067e60 con 0x7f1cac077370 2024-03-30T00:34:08.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.319+0000 7f1caa000700 1 -- 172.21.15.99:0/20823429 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10860 (secure 0 0 0) 0x7f1ca4023090 con 0x7f1cac077370 2024-03-30T00:34:08.326 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:08.327 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:33:21.119410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:34:08.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.322+0000 7f1cb4400700 1 -- 172.21.15.99:0/20823429 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1c940a11b0 msgr2=0x7f1c940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:08.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.322+0000 7f1cb4400700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1c940a11b0 0x7f1c940a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0005910 tx=0x7f1ca000b610 comp rx=0 tx=0).stop 2024-03-30T00:34:08.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.322+0000 7f1cb4400700 1 -- 172.21.15.99:0/20823429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac077370 msgr2=0x7f1cac1a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:08.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.322+0000 7f1cb4400700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac077370 0x7f1cac1a46c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4007ee0 tx=0x7f1ca400d3b0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.322+0000 7f1cb4400700 1 -- 172.21.15.99:0/20823429 shutdown_connections 2024-03-30T00:34:08.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.322+0000 7f1cb4400700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cac077370 0x7f1cac1a46c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.322+0000 7f1cb4400700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cac075770 0x7f1cac19d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.322+0000 7f1cb4400700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1c940a11b0 0x7f1c940a3670 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.322+0000 7f1cb4400700 1 --2- 172.21.15.99:0/20823429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac0769b0 0x7f1cac19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.322+0000 7f1cb4400700 1 -- 172.21.15.99:0/20823429 >> 172.21.15.99:0/20823429 conn(0x7f1cac0fe420 msgr2=0x7f1cac0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:08.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.322+0000 7f1cb4400700 1 -- 172.21.15.99:0/20823429 shutdown_connections 2024-03-30T00:34:08.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.323+0000 7f1cb4400700 1 -- 172.21.15.99:0/20823429 wait complete. 2024-03-30T00:34:08.330 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2024-03-30T00:34:08.348 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:34:08.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.532+0000 7f5893800700 1 -- 172.21.15.99:0/3386243238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588c102810 msgr2=0x7f588c1090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:08.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.532+0000 7f5893800700 1 --2- 172.21.15.99:0/3386243238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588c102810 0x7f588c1090c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f588400b0d0 tx=0x7f588400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.532+0000 7f5893800700 1 -- 172.21.15.99:0/3386243238 shutdown_connections 2024-03-30T00:34:08.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.532+0000 7f5893800700 1 --2- 172.21.15.99:0/3386243238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588c102810 0x7f588c1090c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.532+0000 7f5893800700 1 --2- 172.21.15.99:0/3386243238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f588c100c80 0x7f588c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.532+0000 7f5893800700 1 --2- 172.21.15.99:0/3386243238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c101e70 0x7f588c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.532+0000 7f5893800700 1 -- 172.21.15.99:0/3386243238 >> 172.21.15.99:0/3386243238 conn(0x7f588c0fc1f0 msgr2=0x7f588c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:08.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.532+0000 7f5893800700 1 -- 172.21.15.99:0/3386243238 shutdown_connections 2024-03-30T00:34:08.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.532+0000 7f5893800700 1 -- 172.21.15.99:0/3386243238 wait complete. 2024-03-30T00:34:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.533+0000 7f5893800700 1 Processor -- start 2024-03-30T00:34:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f5893800700 1 -- start start 2024-03-30T00:34:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f5893800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588c100c80 0x7f588c198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:08.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f5893800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c101e70 0x7f588c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:08.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f5893800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f588c102810 0x7f588c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:08.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f5893800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c1a08e0 con 0x7f588c100c80 2024-03-30T00:34:08.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f5893800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c1a0a50 con 0x7f588c101e70 2024-03-30T00:34:08.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f5893800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c1a0c10 con 0x7f588c102810 2024-03-30T00:34:08.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f5890e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588c100c80 0x7f588c198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:08.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f5890e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588c100c80 0x7f588c198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36304/0 (socket says 172.21.15.99:36304) 2024-03-30T00:34:08.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f5890e00700 1 -- 172.21.15.99:0/2842601271 learned_addr learned my addr 172.21.15.99:0/2842601271 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:08.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f5891800700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f588c102810 0x7f588c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:08.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.534+0000 7f588be00700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c101e70 0x7f588c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:08.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.535+0000 7f5890e00700 1 -- 172.21.15.99:0/2842601271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f588c102810 msgr2=0x7f588c1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:08.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.535+0000 7f5890e00700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f588c102810 0x7f588c1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.535+0000 7f5890e00700 1 -- 172.21.15.99:0/2842601271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c101e70 msgr2=0x7f588c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:08.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.535+0000 7f5890e00700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c101e70 0x7f588c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.535+0000 7f5890e00700 1 -- 172.21.15.99:0/2842601271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884009d00 con 0x7f588c100c80 2024-03-30T00:34:08.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.535+0000 7f5890e00700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588c100c80 0x7f588c198df0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f588000d980 tx=0x7f588000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:08.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.535+0000 7f5889600700 1 -- 172.21.15.99:0/2842601271 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588000de60 con 0x7f588c100c80 2024-03-30T00:34:08.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.535+0000 7f5889600700 1 -- 172.21.15.99:0/2842601271 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58800044b0 con 0x7f588c100c80 2024-03-30T00:34:08.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.535+0000 7f5893800700 1 -- 172.21.15.99:0/2842601271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c1a0ef0 con 0x7f588c100c80 2024-03-30T00:34:08.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.535+0000 7f5889600700 1 -- 172.21.15.99:0/2842601271 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880003b40 con 0x7f588c100c80 2024-03-30T00:34:08.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.535+0000 7f5893800700 1 -- 172.21.15.99:0/2842601271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c1a1320 con 0x7f588c100c80 2024-03-30T00:34:08.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.536+0000 7f5893800700 1 -- 172.21.15.99:0/2842601271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c067a10 con 0x7f588c100c80 2024-03-30T00:34:08.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.538+0000 7f5889600700 1 -- 172.21.15.99:0/2842601271 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f5880003ca0 con 0x7f588c100c80 2024-03-30T00:34:08.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.540+0000 7f5889600700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58740a1230 0x7f58740a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:08.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.541+0000 7f5889600700 1 -- 172.21.15.99:0/2842601271 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f58800cc500 con 0x7f588c100c80 2024-03-30T00:34:08.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.541+0000 7f588be00700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58740a1230 0x7f58740a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:08.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.541+0000 7f588be00700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58740a1230 0x7f58740a36f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5878009bf0 tx=0x7f5878009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:08.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.545+0000 7f5889600700 1 -- 172.21.15.99:0/2842601271 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5880095ce0 con 0x7f588c100c80 2024-03-30T00:34:08.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:08 smithi142 ceph-mon[41779]: pgmap v282: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-30T00:34:08.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:08 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/679123197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:08.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:08 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2292492118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:08.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:08 smithi161 ceph-mon[42221]: pgmap v282: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-30T00:34:08.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:08 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/679123197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:08.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:08 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2292492118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:08.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:08 smithi099 ceph-mon[38079]: pgmap v282: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-30T00:34:08.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:08 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/679123197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:08.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:08 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2292492118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:08.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.783+0000 7f5893800700 1 -- 172.21.15.99:0/2842601271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f588c10c350 con 0x7f588c100c80 2024-03-30T00:34:08.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.785+0000 7f5889600700 1 -- 172.21.15.99:0/2842601271 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22120 (secure 0 0 0) 0x7f5880009940 con 0x7f588c100c80 2024-03-30T00:34:08.792 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:08.792 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":83,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:33:16.651168+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:34:08.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.787+0000 7f5893800700 1 -- 172.21.15.99:0/2842601271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58740a1230 msgr2=0x7f58740a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:08.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.787+0000 7f5893800700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58740a1230 0x7f58740a36f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5878009bf0 tx=0x7f5878009260 comp rx=0 tx=0).stop 2024-03-30T00:34:08.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.787+0000 7f5893800700 1 -- 172.21.15.99:0/2842601271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588c100c80 msgr2=0x7f588c198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:08.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.787+0000 7f5893800700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588c100c80 0x7f588c198df0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f588000d980 tx=0x7f588000dd40 comp rx=0 tx=0).stop 2024-03-30T00:34:08.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.787+0000 7f5893800700 1 -- 172.21.15.99:0/2842601271 shutdown_connections 2024-03-30T00:34:08.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.787+0000 7f5893800700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f588c102810 0x7f588c1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.787+0000 7f5893800700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f588c100c80 0x7f588c198df0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.787+0000 7f5893800700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f58740a1230 0x7f58740a36f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.787+0000 7f5893800700 1 --2- 172.21.15.99:0/2842601271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f588c101e70 0x7f588c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:08.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.788+0000 7f5893800700 1 -- 172.21.15.99:0/2842601271 >> 172.21.15.99:0/2842601271 conn(0x7f588c0fc1f0 msgr2=0x7f588c107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:08.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.788+0000 7f5893800700 1 -- 172.21.15.99:0/2842601271 shutdown_connections 2024-03-30T00:34:08.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.788+0000 7f5893800700 1 -- 172.21.15.99:0/2842601271 wait complete. 2024-03-30T00:34:08.813 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:34:09.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.999+0000 7fc2bba00700 1 -- 172.21.15.99:0/175415963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2b4101e70 msgr2=0x7fc2b41022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.999+0000 7fc2bba00700 1 --2- 172.21.15.99:0/175415963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2b4101e70 0x7fc2b41022d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc2a00099c0 tx=0x7fc2a0009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.999+0000 7fc2bba00700 1 -- 172.21.15.99:0/175415963 shutdown_connections 2024-03-30T00:34:09.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.999+0000 7fc2bba00700 1 --2- 172.21.15.99:0/175415963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2b4102810 0x7fc2b41090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.999+0000 7fc2bba00700 1 --2- 172.21.15.99:0/175415963 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b4100c80 0x7fc2b41010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.999+0000 7fc2bba00700 1 --2- 172.21.15.99:0/175415963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2b4101e70 0x7fc2b41022d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.999+0000 7fc2bba00700 1 -- 172.21.15.99:0/175415963 >> 172.21.15.99:0/175415963 conn(0x7fc2b40fc1f0 msgr2=0x7fc2b40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.999+0000 7fc2bba00700 1 -- 172.21.15.99:0/175415963 shutdown_connections 2024-03-30T00:34:09.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:08.999+0000 7fc2bba00700 1 -- 172.21.15.99:0/175415963 wait complete. 2024-03-30T00:34:09.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.000+0000 7fc2bba00700 1 Processor -- start 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.000+0000 7fc2bba00700 1 -- start start 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2bba00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2b4100c80 0x7fc2b4074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2bba00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b4101e70 0x7fc2b40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2bba00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2b4102810 0x7fc2b40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2bba00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4073e80 con 0x7fc2b4102810 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2bba00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4073ff0 con 0x7fc2b4100c80 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2bba00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b41a0bb0 con 0x7fc2b4101e70 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2b9a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2b4102810 0x7fc2b40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2b9a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2b4102810 0x7fc2b40736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36322/0 (socket says 172.21.15.99:36322) 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2b9a00700 1 -- 172.21.15.99:0/945578128 learned_addr learned my addr 172.21.15.99:0/945578128 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2b3e00700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b4101e70 0x7fc2b40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2b9000700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2b4100c80 0x7fc2b4074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2b9a00700 1 -- 172.21.15.99:0/945578128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b4101e70 msgr2=0x7fc2b40731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2b9a00700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b4101e70 0x7fc2b40731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2b9a00700 1 -- 172.21.15.99:0/945578128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2b4100c80 msgr2=0x7fc2b4074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2b9a00700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2b4100c80 0x7fc2b4074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.001+0000 7fc2b9a00700 1 -- 172.21.15.99:0/945578128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a00095f0 con 0x7fc2b4102810 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.002+0000 7fc2b9a00700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2b4102810 0x7fc2b40736e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00ef30 tx=0x7fc2ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.002+0000 7fc2b1600700 1 -- 172.21.15.99:0/945578128 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac018070 con 0x7fc2b4102810 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.002+0000 7fc2bba00700 1 -- 172.21.15.99:0/945578128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b41a0de0 con 0x7fc2b4102810 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.002+0000 7fc2b1600700 1 -- 172.21.15.99:0/945578128 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2ac00f040 con 0x7fc2b4102810 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.002+0000 7fc2b1600700 1 -- 172.21.15.99:0/945578128 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac00cbd0 con 0x7fc2b4102810 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.002+0000 7fc2bba00700 1 -- 172.21.15.99:0/945578128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b41a1260 con 0x7fc2b4102810 2024-03-30T00:34:09.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.003+0000 7fc2bba00700 1 -- 172.21.15.99:0/945578128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b4108170 con 0x7fc2b4102810 2024-03-30T00:34:09.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.005+0000 7fc2b1600700 1 -- 172.21.15.99:0/945578128 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc2ac007500 con 0x7fc2b4102810 2024-03-30T00:34:09.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.006+0000 7fc2b1600700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc29c0a11b0 0x7fc29c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.006+0000 7fc2b1600700 1 -- 172.21.15.99:0/945578128 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc2ac0cc4c0 con 0x7fc2b4102810 2024-03-30T00:34:09.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.006+0000 7fc2b9000700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc29c0a11b0 0x7fc29c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.007+0000 7fc2b9000700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc29c0a11b0 0x7fc29c0a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4101cd0 tx=0x7fc2a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.010+0000 7fc2b1600700 1 -- 172.21.15.99:0/945578128 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc2ac095c30 con 0x7fc2b4102810 2024-03-30T00:34:09.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.268+0000 7fc2bba00700 1 -- 172.21.15.99:0/945578128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2b4067a10 con 0x7fc2b4102810 2024-03-30T00:34:09.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.268+0000 7fc2b1600700 1 -- 172.21.15.99:0/945578128 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10860 (secure 0 0 0) 0x7fc2ac00cea0 con 0x7fc2b4102810 2024-03-30T00:34:09.275 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:09.275 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:33:21.119410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:34:09.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 -- 172.21.15.99:0/945578128 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc29c0a11b0 msgr2=0x7fc29c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc29c0a11b0 0x7fc29c0a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4101cd0 tx=0x7fc2a8009260 comp rx=0 tx=0).stop 2024-03-30T00:34:09.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 -- 172.21.15.99:0/945578128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2b4102810 msgr2=0x7fc2b40736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2b4102810 0x7fc2b40736e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00ef30 tx=0x7fc2ac00c3a0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 -- 172.21.15.99:0/945578128 shutdown_connections 2024-03-30T00:34:09.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2b4102810 0x7fc2b40736e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc29c0a11b0 0x7fc29c0a3670 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc2b4100c80 0x7fc2b4074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 --2- 172.21.15.99:0/945578128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2b4101e70 0x7fc2b40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 -- 172.21.15.99:0/945578128 >> 172.21.15.99:0/945578128 conn(0x7fc2b40fc1f0 msgr2=0x7fc2b41052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 -- 172.21.15.99:0/945578128 shutdown_connections 2024-03-30T00:34:09.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.271+0000 7fc2bba00700 1 -- 172.21.15.99:0/945578128 wait complete. 2024-03-30T00:34:09.279 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2024-03-30T00:34:09.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-30T00:33:15.648899+0000', 'modified': '2024-03-30T00:33:21.119410+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24572, 'mds_1': 24559, 'mds_2': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24473': {'gid': 24473, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6833/780858221', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 780858221}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 780858221}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24502': {'gid': 24502, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.161:6835/80094727', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 80094727}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 80094727}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24503': {'gid': 24503, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/1641167461', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 1641167461}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 1641167461}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24544': {'gid': 24544, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6839/1486341215', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 1486341215}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 1486341215}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24559': {'gid': 24559, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.99:6841/1391823621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6840', 'nonce': 1391823621}, {'type': 'v1', 'addr': '172.21.15.99:6841', 'nonce': 1391823621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6839/2862512313', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 2862512313}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 2862512313}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:34:09.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-30T00:34:09.298 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-30T00:34:09.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.484+0000 7f2373800700 1 -- 172.21.15.99:0/18714598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c101f70 msgr2=0x7f236c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.484+0000 7f2373800700 1 --2- 172.21.15.99:0/18714598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c101f70 0x7f236c1023d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f23580099c0 tx=0x7f2358009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.484+0000 7f2373800700 1 -- 172.21.15.99:0/18714598 shutdown_connections 2024-03-30T00:34:09.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.484+0000 7f2373800700 1 --2- 172.21.15.99:0/18714598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c102910 0x7f236c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.484+0000 7f2373800700 1 --2- 172.21.15.99:0/18714598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c100d80 0x7f236c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.484+0000 7f2373800700 1 --2- 172.21.15.99:0/18714598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c101f70 0x7f236c1023d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.484+0000 7f2373800700 1 -- 172.21.15.99:0/18714598 >> 172.21.15.99:0/18714598 conn(0x7f236c0fc2f0 msgr2=0x7f236c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.484+0000 7f2373800700 1 -- 172.21.15.99:0/18714598 shutdown_connections 2024-03-30T00:34:09.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.484+0000 7f2373800700 1 -- 172.21.15.99:0/18714598 wait complete. 2024-03-30T00:34:09.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.485+0000 7f2373800700 1 Processor -- start 2024-03-30T00:34:09.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.485+0000 7f2373800700 1 -- start start 2024-03-30T00:34:09.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.485+0000 7f2373800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c100d80 0x7f236c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.485+0000 7f2373800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c101f70 0x7f236c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.485+0000 7f2373800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c102910 0x7f236c1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.485+0000 7f2373800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c1a09c0 con 0x7f236c100d80 2024-03-30T00:34:09.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.485+0000 7f2373800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c1a0b00 con 0x7f236c102910 2024-03-30T00:34:09.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.485+0000 7f2373800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c1a0c90 con 0x7f236c101f70 2024-03-30T00:34:09.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.485+0000 7f2370e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c100d80 0x7f236c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2370e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c100d80 0x7f236c198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36336/0 (socket says 172.21.15.99:36336) 2024-03-30T00:34:09.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2370e00700 1 -- 172.21.15.99:0/2353502927 learned_addr learned my addr 172.21.15.99:0/2353502927 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2371800700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c102910 0x7f236c1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2370e00700 1 -- 172.21.15.99:0/2353502927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c101f70 msgr2=0x7f236c1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2370e00700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c101f70 0x7f236c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2370e00700 1 -- 172.21.15.99:0/2353502927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c102910 msgr2=0x7f236c1a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2370e00700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c102910 0x7f236c1a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2370e00700 1 -- 172.21.15.99:0/2353502927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23580095f0 con 0x7f236c100d80 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2371800700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c102910 0x7f236c1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2370e00700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c100d80 0x7f236c198e90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f236000e970 tx=0x7f236000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2369600700 1 -- 172.21.15.99:0/2353502927 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360009790 con 0x7f236c100d80 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2369600700 1 -- 172.21.15.99:0/2353502927 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23600044b0 con 0x7f236c100d80 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2373800700 1 -- 172.21.15.99:0/2353502927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c1a0f70 con 0x7f236c100d80 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2369600700 1 -- 172.21.15.99:0/2353502927 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360003dd0 con 0x7f236c100d80 2024-03-30T00:34:09.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.486+0000 7f2373800700 1 -- 172.21.15.99:0/2353502927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c1a13f0 con 0x7f236c100d80 2024-03-30T00:34:09.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.489+0000 7f2369600700 1 -- 172.21.15.99:0/2353502927 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f2360004000 con 0x7f236c100d80 2024-03-30T00:34:09.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.491+0000 7f2369600700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23540a1230 0x7f23540a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.491+0000 7f2387416b80 1 Processor -- start 2024-03-30T00:34:09.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.491+0000 7f2369600700 1 -- 172.21.15.99:0/2353502927 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f2360014070 con 0x7f236c100d80 2024-03-30T00:34:09.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.491+0000 7f236be00700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23540a1230 0x7f23540a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.491+0000 7f2387416b80 1 -- start start 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.491+0000 7f2387416b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 0x560cae1baba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.491+0000 7f2387416b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bb0e0 0x560cae1bb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.491+0000 7f2387416b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1bc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.491+0000 7f2387416b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae112940 con 0x560cae1bbf80 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2387416b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae110300 con 0x560cae1bb0e0 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2387416b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae10bbf0 con 0x560cae1ba780 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f236be00700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23540a1230 0x7f23540a36f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f235800bf10 tx=0x7f2358003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2371800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1bc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2371800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1bc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36340/0 (socket says 172.21.15.99:36340) 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2371800700 1 -- 172.21.15.99:0/408022323 learned_addr learned my addr 172.21.15.99:0/408022323 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f236be00700 1 --2- 172.21.15.99:0/408022323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bb0e0 0x560cae1bb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2370e00700 1 --2- 172.21.15.99:0/408022323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 0x560cae1baba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2371800700 1 -- 172.21.15.99:0/408022323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 msgr2=0x560cae1baba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2371800700 1 --2- 172.21.15.99:0/408022323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 0x560cae1baba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2371800700 1 -- 172.21.15.99:0/408022323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bb0e0 msgr2=0x560cae1bb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2371800700 1 --2- 172.21.15.99:0/408022323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bb0e0 0x560cae1bb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2371800700 1 -- 172.21.15.99:0/408022323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cadf8d360 con 0x560cae1bbf80 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2370e00700 1 --2- 172.21.15.99:0/408022323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 0x560cae1baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f236be00700 1 --2- 172.21.15.99:0/408022323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bb0e0 0x560cae1bb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.492+0000 7f2371800700 1 --2- 172.21.15.99:0/408022323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1bc400 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2364009e50 tx=0x7f23640077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943c3662293e41ec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f235e000700 1 -- 172.21.15.99:0/408022323 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364014610 con 0x560cae1bbf80 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f235e000700 1 -- 172.21.15.99:0/408022323 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f236400f050 con 0x560cae1bbf80 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f235e000700 1 -- 172.21.15.99:0/408022323 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364013a20 con 0x560cae1bbf80 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f2387416b80 1 -- 172.21.15.99:0/408022323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 msgr2=0x560cae1bc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f2387416b80 1 --2- 172.21.15.99:0/408022323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1bc400 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2364009e50 tx=0x7f23640077e0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f2387416b80 1 -- 172.21.15.99:0/408022323 shutdown_connections 2024-03-30T00:34:09.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f2387416b80 1 --2- 172.21.15.99:0/408022323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1bc400 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f2387416b80 1 --2- 172.21.15.99:0/408022323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 0x560cae1baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f2387416b80 1 --2- 172.21.15.99:0/408022323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bb0e0 0x560cae1bb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f2387416b80 1 -- 172.21.15.99:0/408022323 >> 172.21.15.99:0/408022323 conn(0x560cae119650 msgr2=0x560cae1b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f2387416b80 1 -- 172.21.15.99:0/408022323 shutdown_connections 2024-03-30T00:34:09.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.493+0000 7f2387416b80 1 -- 172.21.15.99:0/408022323 wait complete. 2024-03-30T00:34:09.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.494+0000 7f2387416b80 1 -- start start 2024-03-30T00:34:09.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.494+0000 7f2387416b80 1 Processor -- start 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.494+0000 7f2387416b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1ba780 0x560cae1c4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2387416b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1bb0e0 0x560cae1c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2387416b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2387416b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae0a7d30 con 0x560cae1bbf80 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2387416b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae0a79d0 con 0x560cae1ba780 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2387416b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cadf8f650 con 0x560cae1bb0e0 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2387416b80 10 client.0 _ll_register_callbacks cb 0x560cae0b1420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2371800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2371800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36344/0 (socket says 172.21.15.99:36344) 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2371800700 1 -- 172.21.15.99:0/2265926619 learned_addr learned my addr 172.21.15.99:0/2265926619 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2370e00700 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1ba780 0x560cae1c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f236be00700 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1bb0e0 0x560cae1c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2371800700 1 -- 172.21.15.99:0/2265926619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1bb0e0 msgr2=0x560cae1c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2371800700 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1bb0e0 0x560cae1c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2371800700 1 -- 172.21.15.99:0/2265926619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1ba780 msgr2=0x560cae1c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2371800700 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1ba780 0x560cae1c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2371800700 1 -- 172.21.15.99:0/2265926619 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2364009a80 con 0x560cae1bbf80 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f236be00700 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1bb0e0 0x560cae1c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2371800700 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1cbb60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2364013a00 tx=0x7f2364013a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d814be07133ff128 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.495+0000 7f2352a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2352a00700 1 -- 172.21.15.99:0/2265926619 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364014610 con 0x560cae1bbf80 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2387416b80 1 -- 172.21.15.99:0/2265926619 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cae0cc330 con 0x560cae1bbf80 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2352a00700 1 -- 172.21.15.99:0/2265926619 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f2364014770 con 0x560cae1bbf80 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2352a00700 1 -- 172.21.15.99:0/2265926619 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2364013ee0 con 0x560cae1bbf80 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2387416b80 10 client.15123 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2387416b80 1 -- 172.21.15.99:0/2265926619 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560cadf39b10 con 0x560cae1bbf80 2024-03-30T00:34:09.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2352a00700 1 -- 172.21.15.99:0/2265926619 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560cae0cc330 con 0x560cae1bbf80 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2352a00700 1 -- 172.21.15.99:0/2265926619 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2364027750 con 0x560cae1bbf80 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2387416b80 10 client.15123 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2387416b80 10 client.15123 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2387416b80 20 client.15123 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2387416b80 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560cae1cf750 0x560cae1d4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2387416b80 4 client.15123 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.496+0000 7f2387416b80 1 -- 172.21.15.99:0/2265926619 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cae1046b0 con 0x560cae1cf750 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.497+0000 7f2370e00700 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560cae1cf750 0x560cae1d4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.497+0000 7f2370e00700 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560cae1cf750 0x560cae1d4a50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f632d80d2f7e1a server_cookie=11ccba08686073de in_seq=0 out_seq=0 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.497+0000 7f2352a00700 10 client.15123 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2352a00700 1 -- 172.21.15.99:0/2265926619 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560cae1046b0 con 0x560cae1cf750 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2352a00700 10 client.15123 handle_command_reply: tid=0 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 1 client.15123 shutdown 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 1 -- 172.21.15.99:0/2265926619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 msgr2=0x560cae1cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1cbb60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2364013a00 tx=0x7f2364013a30 comp rx=0 tx=0).stop 2024-03-30T00:34:09.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 1 -- 172.21.15.99:0/2265926619 shutdown_connections 2024-03-30T00:34:09.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560cae1cf750 0x560cae1d4a50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1cbb60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2352a00700 0 client.15123 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:34:09.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1ba780 0x560cae1c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 1 --2- 172.21.15.99:0/2265926619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1bb0e0 0x560cae1c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 1 -- 172.21.15.99:0/2265926619 >> 172.21.15.99:0/2265926619 conn(0x560cae119650 msgr2=0x560cae1b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 1 -- 172.21.15.99:0/2265926619 shutdown_connections 2024-03-30T00:34:09.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 1 -- 172.21.15.99:0/2265926619 wait complete. 2024-03-30T00:34:09.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.499+0000 7f2387416b80 20 client.15123 trim_cache size 0 max 16384 2024-03-30T00:34:09.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.517+0000 7f2387416b80 1 Processor -- start 2024-03-30T00:34:09.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.517+0000 7f2387416b80 1 -- start start 2024-03-30T00:34:09.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.517+0000 7f2387416b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 0x560cae1cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.517+0000 7f2387416b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bb0e0 0x560cae1bc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.517+0000 7f2387416b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.517+0000 7f2387416b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae1046b0 con 0x560cae1bbf80 2024-03-30T00:34:09.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.517+0000 7f2387416b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae0cc330 con 0x560cae1bb0e0 2024-03-30T00:34:09.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.517+0000 7f2387416b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae104990 con 0x560cae1ba780 2024-03-30T00:34:09.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f2371800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f2371800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36352/0 (socket says 172.21.15.99:36352) 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f2371800700 1 -- 172.21.15.99:0/3764498748 learned_addr learned my addr 172.21.15.99:0/3764498748 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f2370e00700 1 --2- 172.21.15.99:0/3764498748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 0x560cae1cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f2371800700 1 -- 172.21.15.99:0/3764498748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 msgr2=0x560cae1cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f2371800700 1 --2- 172.21.15.99:0/3764498748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 0x560cae1cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f2371800700 1 -- 172.21.15.99:0/3764498748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bb0e0 msgr2=0x560cae1bc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f2371800700 1 --2- 172.21.15.99:0/3764498748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bb0e0 0x560cae1bc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f2371800700 1 -- 172.21.15.99:0/3764498748 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cae0d1e50 con 0x560cae1bbf80 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f2370e00700 1 --2- 172.21.15.99:0/3764498748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 0x560cae1cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f2371800700 1 --2- 172.21.15.99:0/3764498748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1b4ec0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2364013770 tx=0x7f2364006160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da2cd3d801784fb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f235ea00700 1 -- 172.21.15.99:0/3764498748 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364036070 con 0x560cae1bbf80 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f235ea00700 1 -- 172.21.15.99:0/3764498748 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2364013ee0 con 0x560cae1bbf80 2024-03-30T00:34:09.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.518+0000 7f235ea00700 1 -- 172.21.15.99:0/3764498748 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364003a00 con 0x560cae1bbf80 2024-03-30T00:34:09.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.519+0000 7f2387416b80 1 -- 172.21.15.99:0/3764498748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 msgr2=0x560cae1b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.519+0000 7f2387416b80 1 --2- 172.21.15.99:0/3764498748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1b4ec0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2364013770 tx=0x7f2364006160 comp rx=0 tx=0).stop 2024-03-30T00:34:09.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.519+0000 7f2387416b80 1 -- 172.21.15.99:0/3764498748 shutdown_connections 2024-03-30T00:34:09.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.519+0000 7f2387416b80 1 --2- 172.21.15.99:0/3764498748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1bbf80 0x560cae1b4ec0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.519+0000 7f2387416b80 1 --2- 172.21.15.99:0/3764498748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1ba780 0x560cae1cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.519+0000 7f2387416b80 1 --2- 172.21.15.99:0/3764498748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bb0e0 0x560cae1bc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.519+0000 7f2387416b80 1 -- 172.21.15.99:0/3764498748 >> 172.21.15.99:0/3764498748 conn(0x560cae119650 msgr2=0x560cae1c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.519+0000 7f2387416b80 1 -- 172.21.15.99:0/3764498748 shutdown_connections 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.519+0000 7f2387416b80 1 -- 172.21.15.99:0/3764498748 wait complete. 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.519+0000 7f2387416b80 1 -- start start 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2387416b80 1 Processor -- start 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2387416b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1ba780 0x560cae1c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2387416b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1bb0e0 0x560cae1c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2387416b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bbf80 0x560cae224040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2387416b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae1c9420 con 0x560cae1ba780 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2387416b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae1c9590 con 0x560cae1bbf80 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2387416b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae224580 con 0x560cae1bb0e0 2024-03-30T00:34:09.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2387416b80 10 client.0 _ll_register_callbacks cb 0x560cae0b1420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2370e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1ba780 0x560cae1c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2370e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1ba780 0x560cae1c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36368/0 (socket says 172.21.15.99:36368) 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2370e00700 1 -- 172.21.15.99:0/1321192247 learned_addr learned my addr 172.21.15.99:0/1321192247 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f236be00700 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1bb0e0 0x560cae1c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.520+0000 7f2371800700 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bbf80 0x560cae224040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2370e00700 1 -- 172.21.15.99:0/1321192247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1bb0e0 msgr2=0x560cae1c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2370e00700 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1bb0e0 0x560cae1c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2370e00700 1 -- 172.21.15.99:0/1321192247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bbf80 msgr2=0x560cae224040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2370e00700 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bbf80 0x560cae224040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2370e00700 1 -- 172.21.15.99:0/1321192247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cadf39b10 con 0x560cae1ba780 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2370e00700 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1ba780 0x560cae1c89a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2360099f70 tx=0x7f23600963a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120f7c1624f97489 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2352a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2352a00700 1 -- 172.21.15.99:0/1321192247 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600a2460 con 0x560cae1ba780 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2352a00700 1 -- 172.21.15.99:0/1321192247 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f2360098380 con 0x560cae1ba780 2024-03-30T00:34:09.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2387416b80 1 -- 172.21.15.99:0/1321192247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f236400f050 con 0x560cae1ba780 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2352a00700 1 -- 172.21.15.99:0/1321192247 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2360016df0 con 0x560cae1ba780 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2387416b80 10 client.15129 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.521+0000 7f2387416b80 1 -- 172.21.15.99:0/1321192247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560cadf91c80 con 0x560cae1ba780 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.522+0000 7f2352a00700 1 -- 172.21.15.99:0/1321192247 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f236400f050 con 0x560cae1ba780 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.522+0000 7f2352a00700 1 -- 172.21.15.99:0/1321192247 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f236009a240 con 0x560cae1ba780 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.522+0000 7f2387416b80 10 client.15129 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.522+0000 7f2387416b80 10 client.15129 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.522+0000 7f2387416b80 20 client.15129 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.522+0000 7f2387416b80 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560cae229320 0x560cae225cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.522+0000 7f2387416b80 4 client.15129 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.522+0000 7f2387416b80 1 -- 172.21.15.99:0/1321192247 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560cadf5aca0 con 0x560cae229320 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.522+0000 7f236be00700 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560cae229320 0x560cae225cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f236be00700 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560cae229320 0x560cae225cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c8e09ba300a00d3 server_cookie=ac27944584532a5e in_seq=0 out_seq=0 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2352a00700 10 client.15129 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2352a00700 1 -- 172.21.15.99:0/1321192247 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560cadf5aca0 con 0x560cae229320 2024-03-30T00:34:09.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2352a00700 10 client.15129 handle_command_reply: tid=0 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2387416b80 1 client.15129 shutdown 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2387416b80 1 -- 172.21.15.99:0/1321192247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1ba780 msgr2=0x560cae1c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2387416b80 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1ba780 0x560cae1c89a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2360099f70 tx=0x7f23600963a0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2387416b80 1 -- 172.21.15.99:0/1321192247 shutdown_connections 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2387416b80 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560cae229320 0x560cae225cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2387416b80 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560cae1bbf80 0x560cae224040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2387416b80 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560cae1ba780 0x560cae1c89a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2387416b80 1 --2- 172.21.15.99:0/1321192247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae1bb0e0 0x560cae1c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2352a00700 0 client.15129 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.523+0000 7f2387416b80 1 -- 172.21.15.99:0/1321192247 >> 172.21.15.99:0/1321192247 conn(0x560cae119650 msgr2=0x560cae1bf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2387416b80 1 -- 172.21.15.99:0/1321192247 shutdown_connections 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2387416b80 1 -- 172.21.15.99:0/1321192247 wait complete. 2024-03-30T00:34:09.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2387416b80 20 client.15129 trim_cache size 0 max 16384 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "03801b1a-ee2c-11ee-b647-cb9ed24678a4", 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 2, 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24502, 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 49.420547820000003, 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 84, 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 75.729643621999998 2024-03-30T00:34:09.535 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:34:09.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 -- 172.21.15.99:0/2353502927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23540a1230 msgr2=0x7f23540a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23540a1230 0x7f23540a36f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f235800bf10 tx=0x7f2358003680 comp rx=0 tx=0).stop 2024-03-30T00:34:09.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 -- 172.21.15.99:0/2353502927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c100d80 msgr2=0x7f236c198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c100d80 0x7f236c198e90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f236000e970 tx=0x7f236000ed30 comp rx=0 tx=0).stop 2024-03-30T00:34:09.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 -- 172.21.15.99:0/2353502927 shutdown_connections 2024-03-30T00:34:09.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23540a1230 0x7f23540a36f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f236c102910 0x7f236c1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c100d80 0x7f236c198e90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 --2- 172.21.15.99:0/2353502927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c101f70 0x7f236c1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 -- 172.21.15.99:0/2353502927 >> 172.21.15.99:0/2353502927 conn(0x7f236c0fc2f0 msgr2=0x7f236c1077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 -- 172.21.15.99:0/2353502927 shutdown_connections 2024-03-30T00:34:09.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.524+0000 7f2373800700 1 -- 172.21.15.99:0/2353502927 wait complete. 2024-03-30T00:34:09.547 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:09 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/20823429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:09.547 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:09 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2842601271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:09.547 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-03-30T00:34:09.547 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-30T00:34:09.547 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-30T00:34:09.547 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-30T00:34:09.547 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-30T00:34:09.547 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-03-30T00:34:09.547 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-03-30T00:34:09.547 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8, 5, 7]}]: dispatch 2024-03-30T00:34:09.547 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]: dispatch 2024-03-30T00:34:09.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-30T00:34:09.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:09 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/20823429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:09.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:09 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2842601271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:09.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-03-30T00:34:09.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-30T00:34:09.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-30T00:34:09.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-30T00:34:09.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-30T00:34:09.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-03-30T00:34:09.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-03-30T00:34:09.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8, 5, 7]}]: dispatch 2024-03-30T00:34:09.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]: dispatch 2024-03-30T00:34:09.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:09 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/20823429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:09.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:09 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2842601271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:09.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-03-30T00:34:09.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-30T00:34:09.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-30T00:34:09.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-30T00:34:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-30T00:34:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-03-30T00:34:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-03-30T00:34:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8, 5, 7]}]: dispatch 2024-03-30T00:34:09.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]: dispatch 2024-03-30T00:34:09.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.733+0000 7f1980800700 1 -- 172.21.15.99:0/1178197324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1978101f70 msgr2=0x7f19781023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.733+0000 7f1980800700 1 --2- 172.21.15.99:0/1178197324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1978101f70 0x7f19781023d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f19680099c0 tx=0x7f1968009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.733+0000 7f1980800700 1 -- 172.21.15.99:0/1178197324 shutdown_connections 2024-03-30T00:34:09.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.733+0000 7f1980800700 1 --2- 172.21.15.99:0/1178197324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1978102910 0x7f19781091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.733+0000 7f1980800700 1 --2- 172.21.15.99:0/1178197324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1978100d80 0x7f19781011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.733+0000 7f1980800700 1 --2- 172.21.15.99:0/1178197324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1978101f70 0x7f19781023d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.733+0000 7f1980800700 1 -- 172.21.15.99:0/1178197324 >> 172.21.15.99:0/1178197324 conn(0x7f19780fc2f0 msgr2=0x7f19780fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.733+0000 7f1980800700 1 -- 172.21.15.99:0/1178197324 shutdown_connections 2024-03-30T00:34:09.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.733+0000 7f1980800700 1 -- 172.21.15.99:0/1178197324 wait complete. 2024-03-30T00:34:09.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.734+0000 7f1980800700 1 Processor -- start 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.734+0000 7f1980800700 1 -- start start 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.734+0000 7f1980800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1978100d80 0x7f1978198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.734+0000 7f1980800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1978101f70 0x7f1978199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.734+0000 7f1980800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1978102910 0x7f19781a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.734+0000 7f1980800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a0960 con 0x7f1978100d80 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.734+0000 7f1980800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a0aa0 con 0x7f1978101f70 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.734+0000 7f1980800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a0c30 con 0x7f1978102910 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.734+0000 7f197de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1978100d80 0x7f1978198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.734+0000 7f197de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1978100d80 0x7f1978198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36394/0 (socket says 172.21.15.99:36394) 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.734+0000 7f197de00700 1 -- 172.21.15.99:0/2264076416 learned_addr learned my addr 172.21.15.99:0/2264076416 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.735+0000 7f197de00700 1 -- 172.21.15.99:0/2264076416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1978102910 msgr2=0x7f19781a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.735+0000 7f197de00700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1978102910 0x7f19781a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.735+0000 7f197de00700 1 -- 172.21.15.99:0/2264076416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1978101f70 msgr2=0x7f1978199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.735+0000 7f197de00700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1978101f70 0x7f1978199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.735+0000 7f197de00700 1 -- 172.21.15.99:0/2264076416 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680095f0 con 0x7f1978100d80 2024-03-30T00:34:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.735+0000 7f197de00700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1978100d80 0x7f1978198e30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f196c00e970 tx=0x7f196c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.735+0000 7f1972a00700 1 -- 172.21.15.99:0/2264076416 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c009790 con 0x7f1978100d80 2024-03-30T00:34:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.735+0000 7f1972a00700 1 -- 172.21.15.99:0/2264076416 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f196c0044b0 con 0x7f1978100d80 2024-03-30T00:34:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.735+0000 7f1972a00700 1 -- 172.21.15.99:0/2264076416 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c003dd0 con 0x7f1978100d80 2024-03-30T00:34:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.735+0000 7f1980800700 1 -- 172.21.15.99:0/2264076416 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19781a0f10 con 0x7f1978100d80 2024-03-30T00:34:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.735+0000 7f1980800700 1 -- 172.21.15.99:0/2264076416 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19781a1330 con 0x7f1978100d80 2024-03-30T00:34:09.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.738+0000 7f1972a00700 1 -- 172.21.15.99:0/2264076416 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f196c003f30 con 0x7f1978100d80 2024-03-30T00:34:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.739+0000 7f1994377b80 1 Processor -- start 2024-03-30T00:34:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f1972a00700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f19600a11b0 0x7f19600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f1994377b80 1 -- start start 2024-03-30T00:34:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f1972a00700 1 -- 172.21.15.99:0/2264076416 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f196c014070 con 0x7f1978100d80 2024-03-30T00:34:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f197e800700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f19600a11b0 0x7f19600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f1994377b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 0x56001b45cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f1994377b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b45d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f1994377b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b45e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f1994377b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b3b4940 con 0x56001b45df80 2024-03-30T00:34:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f1994377b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b3b2300 con 0x56001b45c780 2024-03-30T00:34:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f1994377b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b3adbf0 con 0x56001b45d0e0 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f197de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 0x56001b45cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f197de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 0x56001b45cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:40592/0 (socket says 172.21.15.99:40592) 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f197de00700 1 -- 172.21.15.99:0/2231921746 learned_addr learned my addr 172.21.15.99:0/2231921746 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.740+0000 7f197d400700 1 --2- 172.21.15.99:0/2231921746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b45d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.741+0000 7f197e800700 1 --2- 172.21.15.99:0/2231921746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b45e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.741+0000 7f197de00700 1 -- 172.21.15.99:0/2231921746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 msgr2=0x56001b45d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.741+0000 7f197de00700 1 --2- 172.21.15.99:0/2231921746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b45d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.741+0000 7f197de00700 1 -- 172.21.15.99:0/2231921746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 msgr2=0x56001b45e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.741+0000 7f197de00700 1 --2- 172.21.15.99:0/2231921746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b45e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.741+0000 7f197e800700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f19600a11b0 0x7f19600a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19740060b0 tx=0x7f1974009f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.741+0000 7f197de00700 1 -- 172.21.15.99:0/2231921746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56001b22f360 con 0x56001b45c780 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.741+0000 7f197de00700 1 --2- 172.21.15.99:0/2231921746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 0x56001b45cba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f196c003730 tx=0x7f196c099cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=845e18eae9f5869a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.741+0000 7f1967000700 1 -- 172.21.15.99:0/2231921746 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c09a080 con 0x56001b45c780 2024-03-30T00:34:09.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.741+0000 7f1967000700 1 -- 172.21.15.99:0/2231921746 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f196c09deb0 con 0x56001b45c780 2024-03-30T00:34:09.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.741+0000 7f1967000700 1 -- 172.21.15.99:0/2231921746 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c0041e0 con 0x56001b45c780 2024-03-30T00:34:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.742+0000 7f1994377b80 1 -- 172.21.15.99:0/2231921746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 msgr2=0x56001b45cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.742+0000 7f1994377b80 1 --2- 172.21.15.99:0/2231921746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 0x56001b45cba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f196c003730 tx=0x7f196c099cc0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.742+0000 7f1994377b80 1 -- 172.21.15.99:0/2231921746 shutdown_connections 2024-03-30T00:34:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.742+0000 7f1994377b80 1 --2- 172.21.15.99:0/2231921746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b45e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.742+0000 7f1994377b80 1 --2- 172.21.15.99:0/2231921746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 0x56001b45cba0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.742+0000 7f1994377b80 1 --2- 172.21.15.99:0/2231921746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b45d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.742+0000 7f1994377b80 1 -- 172.21.15.99:0/2231921746 >> 172.21.15.99:0/2231921746 conn(0x56001b3bb650 msgr2=0x56001b457950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.742+0000 7f1994377b80 1 -- 172.21.15.99:0/2231921746 shutdown_connections 2024-03-30T00:34:09.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.742+0000 7f1994377b80 1 -- 172.21.15.99:0/2231921746 wait complete. 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.743+0000 7f1994377b80 1 -- start start 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.743+0000 7f1994377b80 1 Processor -- start 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.743+0000 7f1994377b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45c780 0x56001b466610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.743+0000 7f1994377b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45d0e0 0x56001b466b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.743+0000 7f1994377b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b46db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.743+0000 7f1994377b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b349d30 con 0x56001b45df80 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.743+0000 7f1994377b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b3499d0 con 0x56001b45d0e0 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.743+0000 7f1994377b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b231650 con 0x56001b45c780 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.743+0000 7f1994377b80 10 client.0 _ll_register_callbacks cb 0x56001b353420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f197e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b46db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f197e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b46db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36408/0 (socket says 172.21.15.99:36408) 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f197e800700 1 -- 172.21.15.99:0/1002443606 learned_addr learned my addr 172.21.15.99:0/1002443606 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f197d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45d0e0 0x56001b466b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f197de00700 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45c780 0x56001b466610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f197e800700 1 -- 172.21.15.99:0/1002443606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45c780 msgr2=0x56001b466610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f197e800700 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45c780 0x56001b466610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f197e800700 1 -- 172.21.15.99:0/1002443606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45d0e0 msgr2=0x56001b466b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f197e800700 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45d0e0 0x56001b466b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f197e800700 1 -- 172.21.15.99:0/1002443606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19781a1330 con 0x56001b45df80 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f197e800700 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b46db90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1974018a60 tx=0x7f1974009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859c62a5a5b188ef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f195b400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.744+0000 7f195b400700 1 -- 172.21.15.99:0/1002443606 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974016820 con 0x56001b45df80 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f1994377b80 1 -- 172.21.15.99:0/1002443606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56001b36e330 con 0x56001b45df80 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f195b400700 1 -- 172.21.15.99:0/1002443606 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1974016980 con 0x56001b45df80 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f195b400700 1 -- 172.21.15.99:0/1002443606 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1974016b40 con 0x56001b45df80 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f1994377b80 10 client.15141 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f1994377b80 1 -- 172.21.15.99:0/1002443606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56001b1dbb10 con 0x56001b45df80 2024-03-30T00:34:09.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f195b400700 1 -- 172.21.15.99:0/1002443606 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1974030020 con 0x56001b45df80 2024-03-30T00:34:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f195b400700 1 -- 172.21.15.99:0/1002443606 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1974021be0 con 0x56001b45df80 2024-03-30T00:34:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f1994377b80 10 client.15141 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f1994377b80 10 client.15141 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:34:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f1994377b80 20 client.15141 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f1994377b80 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x56001b470d90 0x56001b476bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f1994377b80 4 client.15141 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:34:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.745+0000 7f1994377b80 1 -- 172.21.15.99:0/1002443606 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56001b3a66b0 con 0x56001b470d90 2024-03-30T00:34:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.746+0000 7f197de00700 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x56001b470d90 0x56001b476bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.746+0000 7f195b400700 10 client.15141 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:34:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.746+0000 7f197de00700 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x56001b470d90 0x56001b476bb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ee24fc1bc3dc299 server_cookie=fcc4b21bb803a582 in_seq=0 out_seq=0 2024-03-30T00:34:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f195b400700 1 -- 172.21.15.99:0/1002443606 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56001b3a66b0 con 0x56001b470d90 2024-03-30T00:34:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f195b400700 10 client.15141 handle_command_reply: tid=0 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 1 client.15141 shutdown 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 1 -- 172.21.15.99:0/1002443606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 msgr2=0x56001b46db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b46db90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1974018a60 tx=0x7f1974009620 comp rx=0 tx=0).stop 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 1 -- 172.21.15.99:0/1002443606 shutdown_connections 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x56001b470d90 0x56001b476bb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b46db90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45c780 0x56001b466610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 1 --2- 172.21.15.99:0/1002443606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45d0e0 0x56001b466b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 1 -- 172.21.15.99:0/1002443606 >> 172.21.15.99:0/1002443606 conn(0x56001b3bb650 msgr2=0x56001b45b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f195b400700 0 client.15141 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 1 -- 172.21.15.99:0/1002443606 shutdown_connections 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 1 -- 172.21.15.99:0/1002443606 wait complete. 2024-03-30T00:34:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.747+0000 7f1994377b80 20 client.15141 trim_cache size 0 max 16384 2024-03-30T00:34:09.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.765+0000 7f1994377b80 1 Processor -- start 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.765+0000 7f1994377b80 1 -- start start 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f1994377b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 0x56001b46e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f1994377b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b45e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f1994377b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b456ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f1994377b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b3a66b0 con 0x56001b45df80 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f1994377b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b36bad0 con 0x56001b45c780 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f1994377b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b3aa850 con 0x56001b45d0e0 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f197e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b456ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f197e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b456ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36410/0 (socket says 172.21.15.99:36410) 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f197e800700 1 -- 172.21.15.99:0/2444643037 learned_addr learned my addr 172.21.15.99:0/2444643037 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f197de00700 1 --2- 172.21.15.99:0/2444643037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 0x56001b46e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f197d400700 1 --2- 172.21.15.99:0/2444643037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b45e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f197e800700 1 -- 172.21.15.99:0/2444643037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 msgr2=0x56001b45e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f197e800700 1 --2- 172.21.15.99:0/2444643037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b45e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f197e800700 1 -- 172.21.15.99:0/2444643037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 msgr2=0x56001b46e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f197e800700 1 --2- 172.21.15.99:0/2444643037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 0x56001b46e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.766+0000 7f197e800700 1 -- 172.21.15.99:0/2444643037 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56001b373e50 con 0x56001b45df80 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f197e800700 1 --2- 172.21.15.99:0/2444643037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b456ec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1974004ad0 tx=0x7f1974007890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7685b496ae4cfa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f1967a00700 1 -- 172.21.15.99:0/2444643037 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197402a570 con 0x56001b45df80 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f1967a00700 1 -- 172.21.15.99:0/2444643037 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56001b36e330 con 0x56001b45df80 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f1967a00700 1 -- 172.21.15.99:0/2444643037 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974016930 con 0x56001b45df80 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f1994377b80 1 -- 172.21.15.99:0/2444643037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 msgr2=0x56001b456ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f1994377b80 1 --2- 172.21.15.99:0/2444643037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b456ec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1974004ad0 tx=0x7f1974007890 comp rx=0 tx=0).stop 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f1994377b80 1 -- 172.21.15.99:0/2444643037 shutdown_connections 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f1994377b80 1 --2- 172.21.15.99:0/2444643037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45df80 0x56001b456ec0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f1994377b80 1 --2- 172.21.15.99:0/2444643037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45c780 0x56001b46e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f1994377b80 1 --2- 172.21.15.99:0/2444643037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b45e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f1994377b80 1 -- 172.21.15.99:0/2444643037 >> 172.21.15.99:0/2444643037 conn(0x56001b3bb650 msgr2=0x56001b4644f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.767+0000 7f1994377b80 1 -- 172.21.15.99:0/2444643037 shutdown_connections 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.768+0000 7f1994377b80 1 -- 172.21.15.99:0/2444643037 wait complete. 2024-03-30T00:34:09.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.768+0000 7f1994377b80 1 -- start start 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.768+0000 7f1994377b80 1 Processor -- start 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.768+0000 7f1994377b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45c780 0x56001b46d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.768+0000 7f1994377b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b46a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.768+0000 7f1994377b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45df80 0x56001b46a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.768+0000 7f1994377b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b46ae00 con 0x56001b45c780 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.768+0000 7f1994377b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b46af70 con 0x56001b45df80 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.768+0000 7f1994377b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001b4c61a0 con 0x56001b45d0e0 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.768+0000 7f1994377b80 10 client.0 _ll_register_callbacks cb 0x56001b353420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45c780 0x56001b46d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45c780 0x56001b46d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36420/0 (socket says 172.21.15.99:36420) 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197de00700 1 -- 172.21.15.99:0/194556874 learned_addr learned my addr 172.21.15.99:0/194556874 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197e800700 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45df80 0x56001b46a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197d400700 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b46a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197de00700 1 -- 172.21.15.99:0/194556874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 msgr2=0x56001b46a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197de00700 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b46a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197de00700 1 -- 172.21.15.99:0/194556874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45df80 msgr2=0x56001b46a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197de00700 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45df80 0x56001b46a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197de00700 1 -- 172.21.15.99:0/194556874 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56001b1dbb10 con 0x56001b45c780 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197d400700 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b46a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f197de00700 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45c780 0x56001b46d8e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f196c000c00 tx=0x7f196c0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e0f254d70950d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f195b400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f195b400700 1 -- 172.21.15.99:0/194556874 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c0a2500 con 0x56001b45c780 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.769+0000 7f195b400700 1 -- 172.21.15.99:0/194556874 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f196c00cbd0 con 0x56001b45c780 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f1994377b80 1 -- 172.21.15.99:0/194556874 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f196c09deb0 con 0x56001b45c780 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f195b400700 1 -- 172.21.15.99:0/194556874 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f196c016df0 con 0x56001b45c780 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f1994377b80 10 client.15147 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f1994377b80 1 -- 172.21.15.99:0/194556874 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56001b233c80 con 0x56001b45c780 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f195b400700 1 -- 172.21.15.99:0/194556874 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f196c09deb0 con 0x56001b45c780 2024-03-30T00:34:09.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f195b400700 1 -- 172.21.15.99:0/194556874 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f196c07c020 con 0x56001b45c780 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f1994377b80 10 client.15147 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f1994377b80 10 client.15147 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f1994377b80 20 client.15147 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f1994377b80 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x56001b4cac20 0x56001b4c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f1994377b80 4 client.15147 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f1994377b80 1 -- 172.21.15.99:0/194556874 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56001b1fcca0 con 0x56001b4cac20 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.770+0000 7f197d400700 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x56001b4cac20 0x56001b4c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f197d400700 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x56001b4cac20 0x56001b4c7590 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11d5c6cd0c86b234 server_cookie=cf5a3c851d10f2a0 in_seq=0 out_seq=0 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f195b400700 10 client.15147 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f195b400700 1 -- 172.21.15.99:0/194556874 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56001b1fcca0 con 0x56001b4cac20 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f195b400700 10 client.15147 handle_command_reply: tid=0 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f1994377b80 1 client.15147 shutdown 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f1994377b80 1 -- 172.21.15.99:0/194556874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45c780 msgr2=0x56001b46d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f1994377b80 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45c780 0x56001b46d8e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f196c000c00 tx=0x7f196c0047a0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f1994377b80 1 -- 172.21.15.99:0/194556874 shutdown_connections 2024-03-30T00:34:09.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f1994377b80 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x56001b4cac20 0x56001b4c7590 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f1994377b80 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56001b45df80 0x56001b46a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f1994377b80 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56001b45c780 0x56001b46d8e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f1994377b80 1 --2- 172.21.15.99:0/194556874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56001b45d0e0 0x56001b46a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f195b400700 0 client.15147 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:34:09.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.771+0000 7f1994377b80 1 -- 172.21.15.99:0/194556874 >> 172.21.15.99:0/194556874 conn(0x56001b3bb650 msgr2=0x56001b464470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1994377b80 1 -- 172.21.15.99:0/194556874 shutdown_connections 2024-03-30T00:34:09.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1994377b80 1 -- 172.21.15.99:0/194556874 wait complete. 2024-03-30T00:34:09.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1994377b80 20 client.15147 trim_cache size 0 max 16384 2024-03-30T00:34:09.785 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "03801b1a-ee2c-11ee-b647-cb9ed24678a4", 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 1, 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24559, 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 50.651921192000003, 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 84, 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 63.574645289000003 2024-03-30T00:34:09.786 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:34:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1980800700 1 -- 172.21.15.99:0/2264076416 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f19600a11b0 msgr2=0x7f19600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1980800700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f19600a11b0 0x7f19600a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19740060b0 tx=0x7f1974009f70 comp rx=0 tx=0).stop 2024-03-30T00:34:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1980800700 1 -- 172.21.15.99:0/2264076416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1978100d80 msgr2=0x7f1978198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1980800700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1978100d80 0x7f1978198e30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f196c00e970 tx=0x7f196c00ed30 comp rx=0 tx=0).stop 2024-03-30T00:34:09.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1980800700 1 -- 172.21.15.99:0/2264076416 shutdown_connections 2024-03-30T00:34:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1980800700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1978102910 0x7f19781a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1980800700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1978100d80 0x7f1978198e30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1980800700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f19600a11b0 0x7f19600a3670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1980800700 1 --2- 172.21.15.99:0/2264076416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1978101f70 0x7f1978199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.772+0000 7f1980800700 1 -- 172.21.15.99:0/2264076416 >> 172.21.15.99:0/2264076416 conn(0x7f19780fc2f0 msgr2=0x7f1978107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.773+0000 7f1980800700 1 -- 172.21.15.99:0/2264076416 shutdown_connections 2024-03-30T00:34:09.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.773+0000 7f1980800700 1 -- 172.21.15.99:0/2264076416 wait complete. 2024-03-30T00:34:09.797 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-30T00:34:09.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.983+0000 7f9c67a00700 1 -- 172.21.15.99:0/1900823023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c60101f70 msgr2=0x7f9c601023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.983+0000 7f9c67a00700 1 --2- 172.21.15.99:0/1900823023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c60101f70 0x7f9c601023d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0099c0 tx=0x7f9c4c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.983+0000 7f9c67a00700 1 -- 172.21.15.99:0/1900823023 shutdown_connections 2024-03-30T00:34:09.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.983+0000 7f9c67a00700 1 --2- 172.21.15.99:0/1900823023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c60102910 0x7f9c601091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.983+0000 7f9c67a00700 1 --2- 172.21.15.99:0/1900823023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c60100d80 0x7f9c601011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.983+0000 7f9c67a00700 1 --2- 172.21.15.99:0/1900823023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c60101f70 0x7f9c601023d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.983+0000 7f9c67a00700 1 -- 172.21.15.99:0/1900823023 >> 172.21.15.99:0/1900823023 conn(0x7f9c600fc2f0 msgr2=0x7f9c600fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:09.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.983+0000 7f9c67a00700 1 -- 172.21.15.99:0/1900823023 shutdown_connections 2024-03-30T00:34:09.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.983+0000 7f9c67a00700 1 -- 172.21.15.99:0/1900823023 wait complete. 2024-03-30T00:34:09.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.984+0000 7f9c67a00700 1 Processor -- start 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.984+0000 7f9c67a00700 1 -- start start 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.984+0000 7f9c67a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c60100d80 0x7f9c60196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.984+0000 7f9c67a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c60101f70 0x7f9c60197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.984+0000 7f9c67a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c60102910 0x7f9c6019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.984+0000 7f9c67a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6019e770 con 0x7f9c60101f70 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.984+0000 7f9c67a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6019e8b0 con 0x7f9c60102910 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.984+0000 7f9c67a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6019ea40 con 0x7f9c60100d80 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.984+0000 7f9c5fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c60101f70 0x7f9c60197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c5fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c60101f70 0x7f9c60197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36448/0 (socket says 172.21.15.99:36448) 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c5fe00700 1 -- 172.21.15.99:0/3045117889 learned_addr learned my addr 172.21.15.99:0/3045117889 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c65000700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c60100d80 0x7f9c60196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c65a00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c60102910 0x7f9c6019e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c5fe00700 1 -- 172.21.15.99:0/3045117889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c60100d80 msgr2=0x7f9c60196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c5fe00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c60100d80 0x7f9c60196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c5fe00700 1 -- 172.21.15.99:0/3045117889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c60102910 msgr2=0x7f9c6019e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:09.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c5fe00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c60102910 0x7f9c6019e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:09.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c5fe00700 1 -- 172.21.15.99:0/3045117889 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0095f0 con 0x7f9c60101f70 2024-03-30T00:34:09.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c5fe00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c60101f70 0x7f9c60197180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c000c00 tx=0x7f9c4c003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:09.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c5d600700 1 -- 172.21.15.99:0/3045117889 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c01d070 con 0x7f9c60101f70 2024-03-30T00:34:09.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c5d600700 1 -- 172.21.15.99:0/3045117889 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c4c00bd30 con 0x7f9c60101f70 2024-03-30T00:34:09.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c67a00700 1 -- 172.21.15.99:0/3045117889 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6019ecc0 con 0x7f9c60101f70 2024-03-30T00:34:09.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c5d600700 1 -- 172.21.15.99:0/3045117889 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c022dc0 con 0x7f9c60101f70 2024-03-30T00:34:09.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.985+0000 7f9c67a00700 1 -- 172.21.15.99:0/3045117889 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6019f0e0 con 0x7f9c60101f70 2024-03-30T00:34:09.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.988+0000 7f9c5d600700 1 -- 172.21.15.99:0/3045117889 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f9c4c003ff0 con 0x7f9c60101f70 2024-03-30T00:34:09.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.989+0000 7f9c7b45fb80 1 Processor -- start 2024-03-30T00:34:09.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c7b45fb80 1 -- start start 2024-03-30T00:34:09.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c5d600700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9c480a11b0 0x7f9c480a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c5d600700 1 -- 172.21.15.99:0/3045117889 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9c4c0cdf80 con 0x7f9c60101f70 2024-03-30T00:34:09.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902e9afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902e9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9c390 0x55b902e9c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:09.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902dee940 con 0x55b902e9b4f0 2024-03-30T00:34:09.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902dec300 con 0x55b902e9c390 2024-03-30T00:34:09.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902de7bf0 con 0x55b902e9ab90 2024-03-30T00:34:09.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c65a00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9c480a11b0 0x7f9c480a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c5fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902e9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:09.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c5fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902e9bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36452/0 (socket says 172.21.15.99:36452) 2024-03-30T00:34:09.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c5fe00700 1 -- 172.21.15.99:0/3955054235 learned_addr learned my addr 172.21.15.99:0/3955054235 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:09.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.990+0000 7f9c65000700 1 --2- 172.21.15.99:0/3955054235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902e9afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.991+0000 7f9c65a00700 1 --2- 172.21.15.99:0/3955054235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9c390 0x55b902e9c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.991+0000 7f9c5fe00700 1 -- 172.21.15.99:0/3955054235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 msgr2=0x55b902e9afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.991+0000 7f9c5fe00700 1 --2- 172.21.15.99:0/3955054235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902e9afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.991+0000 7f9c5fe00700 1 -- 172.21.15.99:0/3955054235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9c390 msgr2=0x55b902e9c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.991+0000 7f9c5fe00700 1 --2- 172.21.15.99:0/3955054235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9c390 0x55b902e9c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.991+0000 7f9c5fe00700 1 -- 172.21.15.99:0/3955054235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b902c69360 con 0x55b902e9b4f0 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.991+0000 7f9c5fe00700 1 --2- 172.21.15.99:0/3955054235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902e9bdb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0977f0 tx=0x7f9c4c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68eaf0288ccfd912 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.991+0000 7f9c65a00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9c480a11b0 0x7f9c480a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9c580075c0 tx=0x7f9c58007980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.991+0000 7f9c52000700 1 -- 172.21.15.99:0/3955054235 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0b0070 con 0x55b902e9b4f0 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.991+0000 7f9c52000700 1 -- 172.21.15.99:0/3955054235 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c4c09b2f0 con 0x55b902e9b4f0 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.991+0000 7f9c52000700 1 -- 172.21.15.99:0/3955054235 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c09a7e0 con 0x55b902e9b4f0 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.992+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3955054235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 msgr2=0x55b902e9bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.992+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/3955054235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902e9bdb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0977f0 tx=0x7f9c4c009310 comp rx=0 tx=0).stop 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.992+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3955054235 shutdown_connections 2024-03-30T00:34:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.992+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/3955054235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9c390 0x55b902e9c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.992+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/3955054235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902e9afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.992+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/3955054235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902e9bdb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.992+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3955054235 >> 172.21.15.99:0/3955054235 conn(0x55b902df5650 msgr2=0x55b902e91950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:10.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.992+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3955054235 shutdown_connections 2024-03-30T00:34:10.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.992+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3955054235 wait complete. 2024-03-30T00:34:10.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.992+0000 7f9c7b45fb80 1 -- start start 2024-03-30T00:34:10.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c7b45fb80 1 Processor -- start 2024-03-30T00:34:10.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9ab90 0x55b902ea46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:10.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9b4f0 0x55b902ea4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:10.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902eabc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:10.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902d83d30 con 0x55b902e9c390 2024-03-30T00:34:10.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902d839d0 con 0x55b902e9ab90 2024-03-30T00:34:10.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902c6b650 con 0x55b902e9b4f0 2024-03-30T00:34:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c7b45fb80 10 client.0 _ll_register_callbacks cb 0x55b902d8d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c65a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902eabc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c65a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902eabc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36466/0 (socket says 172.21.15.99:36466) 2024-03-30T00:34:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c65a00700 1 -- 172.21.15.99:0/578080473 learned_addr learned my addr 172.21.15.99:0/578080473 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c65000700 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9ab90 0x55b902ea46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.993+0000 7f9c5fe00700 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9b4f0 0x55b902ea4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c65a00700 1 -- 172.21.15.99:0/578080473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9b4f0 msgr2=0x55b902ea4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c65a00700 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9b4f0 0x55b902ea4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c65a00700 1 -- 172.21.15.99:0/578080473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9ab90 msgr2=0x55b902ea46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c65a00700 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9ab90 0x55b902ea46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c65a00700 1 -- 172.21.15.99:0/578080473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6019f0e0 con 0x55b902e9c390 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c65a00700 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902eabc20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9c58019ae0 tx=0x7f9c58004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c484b05ab711b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c46a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c46a00700 1 -- 172.21.15.99:0/578080473 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58018c00 con 0x55b902e9c390 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/578080473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b902da8330 con 0x55b902e9c390 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c46a00700 1 -- 172.21.15.99:0/578080473 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9c58018d60 con 0x55b902e9c390 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c46a00700 1 -- 172.21.15.99:0/578080473 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c58024440 con 0x55b902e9c390 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c7b45fb80 10 client.15159 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.994+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/578080473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b902c15b10 con 0x55b902e9c390 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.995+0000 7f9c46a00700 1 -- 172.21.15.99:0/578080473 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c58033080 con 0x55b902e9c390 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.995+0000 7f9c46a00700 1 -- 172.21.15.99:0/578080473 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9c5802f3f0 con 0x55b902e9c390 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.995+0000 7f9c7b45fb80 10 client.15159 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.995+0000 7f9c7b45fb80 10 client.15159 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.995+0000 7f9c7b45fb80 20 client.15159 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.995+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55b902eaf940 0x55b902eb4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:10.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.995+0000 7f9c7b45fb80 4 client.15159 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.995+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/578080473 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b902de06b0 con 0x55b902eaf940 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.996+0000 7f9c65000700 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55b902eaf940 0x55b902eb4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.996+0000 7f9c46a00700 10 client.15159 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.996+0000 7f9c65000700 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55b902eaf940 0x55b902eb4c40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b462ddfcbc2f685b server_cookie=4ca2133516af0186 in_seq=0 out_seq=0 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c46a00700 1 -- 172.21.15.99:0/578080473 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b902de06b0 con 0x55b902eaf940 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c46a00700 10 client.15159 handle_command_reply: tid=0 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c7b45fb80 1 client.15159 shutdown 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/578080473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 msgr2=0x55b902eabc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902eabc20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9c58019ae0 tx=0x7f9c58004b40 comp rx=0 tx=0).stop 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/578080473 shutdown_connections 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55b902eaf940 0x55b902eb4c40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902eabc20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9ab90 0x55b902ea46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/578080473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9b4f0 0x55b902ea4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/578080473 >> 172.21.15.99:0/578080473 conn(0x55b902df5650 msgr2=0x55b902e95120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:10.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.997+0000 7f9c46a00700 0 client.15159 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:10.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.998+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/578080473 shutdown_connections 2024-03-30T00:34:10.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.998+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/578080473 wait complete. 2024-03-30T00:34:10.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:09.998+0000 7f9c7b45fb80 20 client.15159 trim_cache size 0 max 16384 2024-03-30T00:34:10.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.015+0000 7f9c7b45fb80 1 Processor -- start 2024-03-30T00:34:10.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c7b45fb80 1 -- start start 2024-03-30T00:34:10.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902e9af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:10.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902eabbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:10.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9c390 0x55b902e9c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:10.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902de0990 con 0x55b902e9b4f0 2024-03-30T00:34:10.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902de4850 con 0x55b902e9c390 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902e92460 con 0x55b902e9ab90 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c5fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902eabbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c5fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902eabbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36478/0 (socket says 172.21.15.99:36478) 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c5fe00700 1 -- 172.21.15.99:0/1121395083 learned_addr learned my addr 172.21.15.99:0/1121395083 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c5fe00700 1 -- 172.21.15.99:0/1121395083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 msgr2=0x55b902e9af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c5fe00700 1 --2- 172.21.15.99:0/1121395083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902e9af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c65a00700 1 --2- 172.21.15.99:0/1121395083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9c390 0x55b902e9c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c5fe00700 1 -- 172.21.15.99:0/1121395083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9c390 msgr2=0x55b902e9c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c5fe00700 1 --2- 172.21.15.99:0/1121395083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9c390 0x55b902e9c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c5fe00700 1 -- 172.21.15.99:0/1121395083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b902dade50 con 0x55b902e9b4f0 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.016+0000 7f9c65a00700 1 --2- 172.21.15.99:0/1121395083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9c390 0x55b902e9c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:10.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c5fe00700 1 --2- 172.21.15.99:0/1121395083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902eabbe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c004340 tx=0x7f9c4c0cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9aaee594363d013 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c52a00700 1 -- 172.21.15.99:0/1121395083 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0b0070 con 0x55b902e9b4f0 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c52a00700 1 -- 172.21.15.99:0/1121395083 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c4c09f120 con 0x55b902e9b4f0 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c52a00700 1 -- 172.21.15.99:0/1121395083 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c09aa40 con 0x55b902e9b4f0 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/1121395083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 msgr2=0x55b902eabbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/1121395083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902eabbe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c004340 tx=0x7f9c4c0cd040 comp rx=0 tx=0).stop 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/1121395083 shutdown_connections 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/1121395083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9c390 0x55b902e9c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/1121395083 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902e9af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/1121395083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9b4f0 0x55b902eabbe0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/1121395083 >> 172.21.15.99:0/1121395083 conn(0x55b902df5650 msgr2=0x55b902e8a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.017+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/1121395083 shutdown_connections 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.018+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/1121395083 wait complete. 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.018+0000 7f9c7b45fb80 1 -- start start 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.018+0000 7f9c7b45fb80 1 Processor -- start 2024-03-30T00:34:10.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.018+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902ea1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:10.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.018+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9b4f0 0x55b902ea1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:10.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.018+0000 7f9c7b45fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902ea1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:10.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.018+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902eabf20 con 0x55b902e9c390 2024-03-30T00:34:10.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.018+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902eac060 con 0x55b902e9b4f0 2024-03-30T00:34:10.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.018+0000 7f9c7b45fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b902f04280 con 0x55b902e9ab90 2024-03-30T00:34:10.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c7b45fb80 10 client.0 _ll_register_callbacks cb 0x55b902d8d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:10.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c65a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902ea1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c65a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902ea1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36486/0 (socket says 172.21.15.99:36486) 2024-03-30T00:34:10.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c65a00700 1 -- 172.21.15.99:0/3837673622 learned_addr learned my addr 172.21.15.99:0/3837673622 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:10.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c5fe00700 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9b4f0 0x55b902ea1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c65a00700 1 -- 172.21.15.99:0/3837673622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 msgr2=0x55b902ea1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:10.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c65000700 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902ea1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c65a00700 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902ea1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c65a00700 1 -- 172.21.15.99:0/3837673622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9b4f0 msgr2=0x55b902ea1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c65a00700 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9b4f0 0x55b902ea1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c65a00700 1 -- 172.21.15.99:0/3837673622 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b902c69360 con 0x55b902e9c390 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c65000700 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902ea1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c65a00700 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902ea1c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009f60 tx=0x7f9c58009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478bf6d84dc7326c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.019+0000 7f9c46a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c46a00700 1 -- 172.21.15.99:0/3837673622 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58018c00 con 0x55b902e9c390 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3837673622 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c4c09b2f0 con 0x55b902e9c390 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c46a00700 1 -- 172.21.15.99:0/3837673622 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9c58036070 con 0x55b902e9c390 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c46a00700 1 -- 172.21.15.99:0/3837673622 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c58018d60 con 0x55b902e9c390 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c7b45fb80 10 client.15165 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:10.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3837673622 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b902c6dc80 con 0x55b902e9c390 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c46a00700 1 -- 172.21.15.99:0/3837673622 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c4c09b2f0 con 0x55b902e9c390 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c46a00700 1 -- 172.21.15.99:0/3837673622 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9c58045ac0 con 0x55b902e9c390 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c7b45fb80 10 client.15165 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c7b45fb80 10 client.15165 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c7b45fb80 20 client.15165 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55b902f09060 0x55b902f05990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c7b45fb80 4 client.15165 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.020+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3837673622 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b902c36ca0 con 0x55b902f09060 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.021+0000 7f9c65000700 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55b902f09060 0x55b902f05990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.021+0000 7f9c65000700 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55b902f09060 0x55b902f05990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1156677a3baa97 server_cookie=f083c33472fe1385 in_seq=0 out_seq=0 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.021+0000 7f9c46a00700 10 client.15165 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c46a00700 1 -- 172.21.15.99:0/3837673622 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b902c36ca0 con 0x55b902f09060 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c46a00700 10 client.15165 handle_command_reply: tid=0 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 1 client.15165 shutdown 2024-03-30T00:34:10.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3837673622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 msgr2=0x55b902ea1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:10.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902ea1c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009f60 tx=0x7f9c58009200 comp rx=0 tx=0).stop 2024-03-30T00:34:10.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3837673622 shutdown_connections 2024-03-30T00:34:10.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55b902f09060 0x55b902f05990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b902e9c390 0x55b902ea1c80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b902e9ab90 0x55b902ea1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 1 --2- 172.21.15.99:0/3837673622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b902e9b4f0 0x55b902ea1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c46a00700 0 client.15165 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:10.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3837673622 >> 172.21.15.99:0/3837673622 conn(0x55b902df5650 msgr2=0x55b902ea3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:10.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3837673622 shutdown_connections 2024-03-30T00:34:10.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 1 -- 172.21.15.99:0/3837673622 wait complete. 2024-03-30T00:34:10.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.022+0000 7f9c7b45fb80 20 client.15165 trim_cache size 0 max 16384 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "03801b1a-ee2c-11ee-b647-cb9ed24678a4", 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24572, 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 54.369544664999999, 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 84, 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 61.428109224000004 2024-03-30T00:34:10.035 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 -- 172.21.15.99:0/3045117889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9c480a11b0 msgr2=0x7f9c480a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9c480a11b0 0x7f9c480a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9c580075c0 tx=0x7f9c58007980 comp rx=0 tx=0).stop 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 -- 172.21.15.99:0/3045117889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c60101f70 msgr2=0x7f9c60197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c60101f70 0x7f9c60197180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c000c00 tx=0x7f9c4c003b00 comp rx=0 tx=0).stop 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 -- 172.21.15.99:0/3045117889 shutdown_connections 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9c480a11b0 0x7f9c480a3670 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c60102910 0x7f9c6019e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c60100d80 0x7f9c60196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 --2- 172.21.15.99:0/3045117889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c60101f70 0x7f9c60197180 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 -- 172.21.15.99:0/3045117889 >> 172.21.15.99:0/3045117889 conn(0x7f9c600fc2f0 msgr2=0x7f9c60105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 -- 172.21.15.99:0/3045117889 shutdown_connections 2024-03-30T00:34:10.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:10.023+0000 7f9c67a00700 1 -- 172.21.15.99:0/3045117889 wait complete. 2024-03-30T00:34:10.047 INFO:tasks.cephadm:Setup complete, yielding 2024-03-30T00:34:10.048 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-30T00:34:10.057 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-03-30T00:34:10.058 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch status' 2024-03-30T00:34:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:34:10.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-30T00:34:10.489 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-03-30T00:34:10.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-30T00:34:10.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-30T00:34:10.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-30T00:34:10.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-30T00:34:10.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-03-30T00:34:10.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-03-30T00:34:10.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8, 5, 7]}]': finished 2024-03-30T00:34:10.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]': finished 2024-03-30T00:34:10.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: osdmap e84: 12 total, 12 up, 12 in 2024-03-30T00:34:10.490 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:10 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/945578128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:10.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-30T00:34:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-03-30T00:34:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-30T00:34:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-30T00:34:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-30T00:34:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-30T00:34:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-03-30T00:34:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-03-30T00:34:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8, 5, 7]}]': finished 2024-03-30T00:34:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]': finished 2024-03-30T00:34:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: osdmap e84: 12 total, 12 up, 12 in 2024-03-30T00:34:10.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:10 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/945578128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:10.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-30T00:34:10.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-03-30T00:34:10.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-30T00:34:10.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-30T00:34:10.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-30T00:34:10.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-30T00:34:10.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-03-30T00:34:10.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-03-30T00:34:10.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8, 5, 7]}]': finished 2024-03-30T00:34:10.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3d", "id": [1, 8]}]': finished 2024-03-30T00:34:10.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: osdmap e84: 12 total, 12 up, 12 in 2024-03-30T00:34:10.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:10 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/945578128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:11.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:11 smithi142 ceph-mon[41779]: osdmap e85: 12 total, 12 up, 12 in 2024-03-30T00:34:11.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.570+0000 7ff7f2000700 1 -- 172.21.15.99:0/464752110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec104210 msgr2=0x7ff7ec106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:11.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.570+0000 7ff7f2000700 1 --2- 172.21.15.99:0/464752110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec104210 0x7ff7ec106600 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009990 tx=0x7ff7d8009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:11.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.570+0000 7ff7f2000700 1 -- 172.21.15.99:0/464752110 shutdown_connections 2024-03-30T00:34:11.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.570+0000 7ff7f2000700 1 --2- 172.21.15.99:0/464752110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec106b40 0x7ff7ec108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:11.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.570+0000 7ff7f2000700 1 --2- 172.21.15.99:0/464752110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7ec1018e0 0x7ff7ec103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:11.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.570+0000 7ff7f2000700 1 --2- 172.21.15.99:0/464752110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec104210 0x7ff7ec106600 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:11.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.570+0000 7ff7f2000700 1 -- 172.21.15.99:0/464752110 >> 172.21.15.99:0/464752110 conn(0x7ff7ec0fb2b0 msgr2=0x7ff7ec0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:11.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.571+0000 7ff7f2000700 1 -- 172.21.15.99:0/464752110 shutdown_connections 2024-03-30T00:34:11.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.571+0000 7ff7f2000700 1 -- 172.21.15.99:0/464752110 wait complete. 2024-03-30T00:34:11.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.571+0000 7ff7f2000700 1 Processor -- start 2024-03-30T00:34:11.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.572+0000 7ff7f2000700 1 -- start start 2024-03-30T00:34:11.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.572+0000 7ff7f2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec1018e0 0x7ff7ec198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:11.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.572+0000 7ff7f2000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7ec104210 0x7ff7ec1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:11.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.572+0000 7ff7f2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec106b40 0x7ff7ec1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.572+0000 7ff7f2000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec1a09c0 con 0x7ff7ec106b40 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.572+0000 7ff7f2000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec1a0b00 con 0x7ff7ec1018e0 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.572+0000 7ff7f2000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec1a0c90 con 0x7ff7ec104210 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.572+0000 7ff7ebe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec106b40 0x7ff7ec1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.572+0000 7ff7ebe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec106b40 0x7ff7ec1a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36500/0 (socket says 172.21.15.99:36500) 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.572+0000 7ff7ebe00700 1 -- 172.21.15.99:0/133874991 learned_addr learned my addr 172.21.15.99:0/133874991 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.572+0000 7ff7eaa00700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7ec104210 0x7ff7ec1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7eb400700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec1018e0 0x7ff7ec198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7ebe00700 1 -- 172.21.15.99:0/133874991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7ec104210 msgr2=0x7ff7ec1993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7ebe00700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7ec104210 0x7ff7ec1993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7ebe00700 1 -- 172.21.15.99:0/133874991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec1018e0 msgr2=0x7ff7ec198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7ebe00700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec1018e0 0x7ff7ec198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7ebe00700 1 -- 172.21.15.99:0/133874991 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d80095f0 con 0x7ff7ec106b40 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7eb400700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec1018e0 0x7ff7ec198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:11.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7ebe00700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec106b40 0x7ff7ec1a03f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4007d70 tx=0x7ff7e400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:11.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7e3e00700 1 -- 172.21.15.99:0/133874991 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400ad00 con 0x7ff7ec106b40 2024-03-30T00:34:11.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7e3e00700 1 -- 172.21.15.99:0/133874991 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff7e400ae60 con 0x7ff7ec106b40 2024-03-30T00:34:11.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7e3e00700 1 -- 172.21.15.99:0/133874991 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40115e0 con 0x7ff7ec106b40 2024-03-30T00:34:11.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7f2000700 1 -- 172.21.15.99:0/133874991 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ec1a0f70 con 0x7ff7ec106b40 2024-03-30T00:34:11.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.573+0000 7ff7f2000700 1 -- 172.21.15.99:0/133874991 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ec1a1330 con 0x7ff7ec106b40 2024-03-30T00:34:11.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.574+0000 7ff7f2000700 1 -- 172.21.15.99:0/133874991 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ec1086e0 con 0x7ff7ec106b40 2024-03-30T00:34:11.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.576+0000 7ff7e3e00700 1 -- 172.21.15.99:0/133874991 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff7e4012940 con 0x7ff7ec106b40 2024-03-30T00:34:11.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.577+0000 7ff7e3e00700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7d00a11b0 0x7ff7d00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:11.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.578+0000 7ff7e3e00700 1 -- 172.21.15.99:0/133874991 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff7e40ccc10 con 0x7ff7ec106b40 2024-03-30T00:34:11.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.578+0000 7ff7eb400700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7d00a11b0 0x7ff7d00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:11.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.578+0000 7ff7eb400700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7d00a11b0 0x7ff7d00a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc005950 tx=0x7ff7dc005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:11.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.582+0000 7ff7e3e00700 1 -- 172.21.15.99:0/133874991 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff7e40962e0 con 0x7ff7ec106b40 2024-03-30T00:34:11.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:11 smithi161 ceph-mon[42221]: osdmap e85: 12 total, 12 up, 12 in 2024-03-30T00:34:11.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:11 smithi099 ceph-mon[38079]: osdmap e85: 12 total, 12 up, 12 in 2024-03-30T00:34:11.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.807+0000 7ff7f2000700 1 -- 172.21.15.99:0/133874991 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7ec02ce10 con 0x7ff7d00a11b0 2024-03-30T00:34:11.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.809+0000 7ff7e3e00700 1 -- 172.21.15.99:0/133874991 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff7ec02ce10 con 0x7ff7d00a11b0 2024-03-30T00:34:11.816 INFO:teuthology.orchestra.run.smithi099.stdout:Backend: cephadm 2024-03-30T00:34:11.816 INFO:teuthology.orchestra.run.smithi099.stdout:Available: Yes 2024-03-30T00:34:11.816 INFO:teuthology.orchestra.run.smithi099.stdout:Paused: No 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 -- 172.21.15.99:0/133874991 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7d00a11b0 msgr2=0x7ff7d00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7d00a11b0 0x7ff7d00a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc005950 tx=0x7ff7dc005f50 comp rx=0 tx=0).stop 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 -- 172.21.15.99:0/133874991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec106b40 msgr2=0x7ff7ec1a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec106b40 0x7ff7ec1a03f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4007d70 tx=0x7ff7e400a340 comp rx=0 tx=0).stop 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 -- 172.21.15.99:0/133874991 shutdown_connections 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec106b40 0x7ff7ec1a03f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7d00a11b0 0x7ff7d00a3670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec1018e0 0x7ff7ec198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 --2- 172.21.15.99:0/133874991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7ec104210 0x7ff7ec1993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 -- 172.21.15.99:0/133874991 >> 172.21.15.99:0/133874991 conn(0x7ff7ec0fb2b0 msgr2=0x7ff7ec1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 -- 172.21.15.99:0/133874991 shutdown_connections 2024-03-30T00:34:11.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:11.811+0000 7ff7f2000700 1 -- 172.21.15.99:0/133874991 wait complete. 2024-03-30T00:34:12.137 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ps' 2024-03-30T00:34:12.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:12 smithi099 ceph-mon[38079]: pgmap v286: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-30T00:34:12.497 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:34:12.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:12 smithi142 ceph-mon[41779]: pgmap v286: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-30T00:34:12.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:12 smithi161 ceph-mon[42221]: pgmap v286: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-30T00:34:13.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:13 smithi142 ceph-mon[41779]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:13.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.651+0000 7fed2f800700 1 -- 172.21.15.99:0/4038907442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28100e00 msgr2=0x7fed28101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:13.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.651+0000 7fed2f800700 1 --2- 172.21.15.99:0/4038907442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28100e00 0x7fed28101220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009990 tx=0x7fed1c009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:13.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.652+0000 7fed2f800700 1 -- 172.21.15.99:0/4038907442 shutdown_connections 2024-03-30T00:34:13.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.652+0000 7fed2f800700 1 --2- 172.21.15.99:0/4038907442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed28102920 0x7fed28109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:13.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.652+0000 7fed2f800700 1 --2- 172.21.15.99:0/4038907442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28100e00 0x7fed28101220 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:13.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.652+0000 7fed2f800700 1 --2- 172.21.15.99:0/4038907442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed28101f60 0x7fed281023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:13.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.652+0000 7fed2f800700 1 -- 172.21.15.99:0/4038907442 >> 172.21.15.99:0/4038907442 conn(0x7fed280fc3c0 msgr2=0x7fed280fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:13.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.652+0000 7fed2f800700 1 -- 172.21.15.99:0/4038907442 shutdown_connections 2024-03-30T00:34:13.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.652+0000 7fed2f800700 1 -- 172.21.15.99:0/4038907442 wait complete. 2024-03-30T00:34:13.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.653+0000 7fed2f800700 1 Processor -- start 2024-03-30T00:34:13.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.653+0000 7fed2f800700 1 -- start start 2024-03-30T00:34:13.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.653+0000 7fed2f800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed28100e00 0x7fed2819d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:13.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.653+0000 7fed2f800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed28101f60 0x7fed2819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:13.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.653+0000 7fed2f800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28102920 0x7fed281a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.653+0000 7fed2f800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed281a4d60 con 0x7fed28102920 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.653+0000 7fed2f800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed281a4ea0 con 0x7fed28100e00 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.653+0000 7fed2f800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed281a5030 con 0x7fed28101f60 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed2d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28102920 0x7fed281a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed2d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28102920 0x7fed281a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36516/0 (socket says 172.21.15.99:36516) 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed2d800700 1 -- 172.21.15.99:0/2112017753 learned_addr learned my addr 172.21.15.99:0/2112017753 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed2ce00700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed28100e00 0x7fed2819d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed27e00700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed28101f60 0x7fed2819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed2d800700 1 -- 172.21.15.99:0/2112017753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed28101f60 msgr2=0x7fed2819d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed2d800700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed28101f60 0x7fed2819d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed2d800700 1 -- 172.21.15.99:0/2112017753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed28100e00 msgr2=0x7fed2819d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed2d800700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed28100e00 0x7fed2819d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed2d800700 1 -- 172.21.15.99:0/2112017753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c0095f0 con 0x7fed28102920 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed27e00700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed28101f60 0x7fed2819d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:13.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed2d800700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28102920 0x7fed281a4790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fed20007da0 tx=0x7fed2000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:13.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.654+0000 7fed25600700 1 -- 172.21.15.99:0/2112017753 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2000ad00 con 0x7fed28102920 2024-03-30T00:34:13.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.655+0000 7fed25600700 1 -- 172.21.15.99:0/2112017753 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed2000ae60 con 0x7fed28102920 2024-03-30T00:34:13.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.655+0000 7fed2f800700 1 -- 172.21.15.99:0/2112017753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed281a5310 con 0x7fed28102920 2024-03-30T00:34:13.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.655+0000 7fed25600700 1 -- 172.21.15.99:0/2112017753 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed200116f0 con 0x7fed28102920 2024-03-30T00:34:13.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.655+0000 7fed2f800700 1 -- 172.21.15.99:0/2112017753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed281a57c0 con 0x7fed28102920 2024-03-30T00:34:13.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.656+0000 7fed2f800700 1 -- 172.21.15.99:0/2112017753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed281062e0 con 0x7fed28102920 2024-03-30T00:34:13.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.658+0000 7fed25600700 1 -- 172.21.15.99:0/2112017753 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fed20012a40 con 0x7fed28102920 2024-03-30T00:34:13.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.659+0000 7fed25600700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed100a11b0 0x7fed100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:13.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.659+0000 7fed25600700 1 -- 172.21.15.99:0/2112017753 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fed200ccc80 con 0x7fed28102920 2024-03-30T00:34:13.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.659+0000 7fed2ce00700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed100a11b0 0x7fed100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:13.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.660+0000 7fed2ce00700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed100a11b0 0x7fed100a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fed1c006010 tx=0x7fed1c003740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:13.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.663+0000 7fed25600700 1 -- 172.21.15.99:0/2112017753 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fed20096350 con 0x7fed28102920 2024-03-30T00:34:13.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:13 smithi161 ceph-mon[42221]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:13.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:13 smithi099 ceph-mon[38079]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:13.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.877+0000 7fed2f800700 1 -- 172.21.15.99:0/2112017753 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fed280611f0 con 0x7fed100a11b0 2024-03-30T00:34:13.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.891+0000 7fed25600700 1 -- 172.21.15.99:0/2112017753 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fed280611f0 con 0x7fed100a11b0 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.a smithi099 running (67s) 60s ago 67s 13.1M - 18.2.2-1018-g470100a4 e7c444a08cf7 92bf90815d97 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.b smithi142 running (65s) 60s ago 65s 13.0M - 18.2.2-1018-g470100a4 e7c444a08cf7 9c254cab379e 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.c smithi161 running (77s) 60s ago 77s 13.1M - 18.2.2-1018-g470100a4 e7c444a08cf7 5850d9e78d22 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.d smithi099 running (85s) 60s ago 84s 13.1M - 18.2.2-1018-g470100a4 e7c444a08cf7 43ecc6bbd054 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.e smithi142 running (82s) 60s ago 82s 13.0M - 18.2.2-1018-g470100a4 e7c444a08cf7 9f784a44d663 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.f smithi161 running (80s) 60s ago 80s 13.0M - 18.2.2-1018-g470100a4 e7c444a08cf7 24b249e2726f 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.g smithi099 running (70s) 60s ago 70s 13.1M - 18.2.2-1018-g470100a4 e7c444a08cf7 58255d9043ef 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.h smithi142 running (75s) 60s ago 75s 13.0M - 18.2.2-1018-g470100a4 e7c444a08cf7 9bc08616c821 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.i smithi161 running (72s) 60s ago 72s 13.0M - 18.2.2-1018-g470100a4 e7c444a08cf7 6bf8e61fcde8 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.j smithi099 running (87s) 60s ago 87s 13.1M - 18.2.2-1018-g470100a4 e7c444a08cf7 519a94f9796f 2024-03-30T00:34:13.900 INFO:teuthology.orchestra.run.smithi099.stdout:mds.k smithi142 running (89s) 60s ago 89s 13.1M - 18.2.2-1018-g470100a4 e7c444a08cf7 72f5314df4c4 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:mds.l smithi161 running (92s) 60s ago 92s 13.1M - 18.2.2-1018-g470100a4 e7c444a08cf7 e77ed2d3f808 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.x smithi099 *:9283,8765 running (7m) 60s ago 7m 528M - 18.2.2-1018-g470100a4 e7c444a08cf7 417f140602d0 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.y smithi142 *:8443,8765 running (6m) 60s ago 6m 461M - 18.2.2-1018-g470100a4 e7c444a08cf7 86e2af8b711f 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:mgr.z smithi161 *:8443,8765 running (6m) 60s ago 6m 459M - 18.2.2-1018-g470100a4 e7c444a08cf7 3eea1cdabea7 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:mon.a smithi099 running (7m) 60s ago 7m 83.9M 2048M 18.2.2-1018-g470100a4 e7c444a08cf7 aa969bc775f1 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:mon.b smithi142 running (6m) 60s ago 6m 57.2M 2048M 18.2.2-1018-g470100a4 e7c444a08cf7 b775c6a43040 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:mon.c smithi161 running (6m) 60s ago 6m 58.8M 2048M 18.2.2-1018-g470100a4 e7c444a08cf7 7943442a0136 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:osd.0 smithi099 running (5m) 60s ago 5m 63.6M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 57a861f0335b 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:osd.1 smithi142 running (5m) 60s ago 5m 70.3M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 f22b101e9fe0 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:osd.2 smithi161 running (5m) 60s ago 5m 64.5M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 b6b33c969c7f 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:osd.3 smithi099 running (4m) 60s ago 4m 51.6M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 63d8abfdc7bf 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:osd.4 smithi142 running (4m) 60s ago 4m 58.9M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 75f790c1de4a 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:osd.5 smithi161 running (4m) 60s ago 4m 62.1M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 c3e243ecd073 2024-03-30T00:34:13.901 INFO:teuthology.orchestra.run.smithi099.stdout:osd.6 smithi099 running (3m) 60s ago 3m 58.1M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 ff5ea1018f70 2024-03-30T00:34:13.902 INFO:teuthology.orchestra.run.smithi099.stdout:osd.7 smithi142 running (3m) 60s ago 3m 51.8M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 dd8aea7f468f 2024-03-30T00:34:13.902 INFO:teuthology.orchestra.run.smithi099.stdout:osd.8 smithi161 running (2m) 60s ago 2m 49.5M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 04e61493c47a 2024-03-30T00:34:13.902 INFO:teuthology.orchestra.run.smithi099.stdout:osd.9 smithi099 running (2m) 60s ago 2m 46.6M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 1ba7db4c2229 2024-03-30T00:34:13.902 INFO:teuthology.orchestra.run.smithi099.stdout:osd.10 smithi142 running (2m) 60s ago 2m 48.4M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 ad93d37847e9 2024-03-30T00:34:13.902 INFO:teuthology.orchestra.run.smithi099.stdout:osd.11 smithi161 running (109s) 60s ago 109s 54.8M 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 693caa162a97 2024-03-30T00:34:13.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 -- 172.21.15.99:0/2112017753 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed100a11b0 msgr2=0x7fed100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:13.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed100a11b0 0x7fed100a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fed1c006010 tx=0x7fed1c003740 comp rx=0 tx=0).stop 2024-03-30T00:34:13.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 -- 172.21.15.99:0/2112017753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28102920 msgr2=0x7fed281a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:13.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28102920 0x7fed281a4790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fed20007da0 tx=0x7fed2000a340 comp rx=0 tx=0).stop 2024-03-30T00:34:13.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 -- 172.21.15.99:0/2112017753 shutdown_connections 2024-03-30T00:34:13.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed28102920 0x7fed281a4790 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:13.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed100a11b0 0x7fed100a3670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:13.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed28100e00 0x7fed2819d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:13.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 --2- 172.21.15.99:0/2112017753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed28101f60 0x7fed2819d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:13.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 -- 172.21.15.99:0/2112017753 >> 172.21.15.99:0/2112017753 conn(0x7fed280fc3c0 msgr2=0x7fed28103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:13.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 -- 172.21.15.99:0/2112017753 shutdown_connections 2024-03-30T00:34:13.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:13.894+0000 7fed2f800700 1 -- 172.21.15.99:0/2112017753 wait complete. 2024-03-30T00:34:14.244 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ls' 2024-03-30T00:34:14.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:14 smithi099 ceph-mon[38079]: pgmap v287: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-30T00:34:14.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:14 smithi142 ceph-mon[41779]: pgmap v287: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-30T00:34:14.613 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:34:14.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:14 smithi161 ceph-mon[42221]: pgmap v287: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-30T00:34:15.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:15 smithi099 ceph-mon[38079]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:15.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:15 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:34:15.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:15 smithi142 ceph-mon[41779]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:15.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:15 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:34:15.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:15 smithi161 ceph-mon[42221]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:15.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:15 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:34:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.823+0000 7effe9600700 1 -- 172.21.15.99:0/2977192388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe4100e00 msgr2=0x7effe4101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.823+0000 7effe9600700 1 --2- 172.21.15.99:0/2977192388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe4100e00 0x7effe4101220 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7effd4009990 tx=0x7effd4009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.823+0000 7effe9600700 1 -- 172.21.15.99:0/2977192388 shutdown_connections 2024-03-30T00:34:15.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.823+0000 7effe9600700 1 --2- 172.21.15.99:0/2977192388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4102920 0x7effe4109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.823+0000 7effe9600700 1 --2- 172.21.15.99:0/2977192388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe4100e00 0x7effe4101220 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.823+0000 7effe9600700 1 --2- 172.21.15.99:0/2977192388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe4101f60 0x7effe41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.823+0000 7effe9600700 1 -- 172.21.15.99:0/2977192388 >> 172.21.15.99:0/2977192388 conn(0x7effe40fc3c0 msgr2=0x7effe40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.823+0000 7effe9600700 1 -- 172.21.15.99:0/2977192388 shutdown_connections 2024-03-30T00:34:15.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.823+0000 7effe9600700 1 -- 172.21.15.99:0/2977192388 wait complete. 2024-03-30T00:34:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.824+0000 7effe9600700 1 Processor -- start 2024-03-30T00:34:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe9600700 1 -- start start 2024-03-30T00:34:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4100e00 0x7effe4198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe9600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe4101f60 0x7effe4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe9600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe4102920 0x7effe41a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe9600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe41a0a10 con 0x7effe4101f60 2024-03-30T00:34:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe9600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe41a0b50 con 0x7effe4100e00 2024-03-30T00:34:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe9600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe41a0ce0 con 0x7effe4102920 2024-03-30T00:34:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe4101f60 0x7effe4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:15.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe4101f60 0x7effe4199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50262/0 (socket says 172.21.15.99:50262) 2024-03-30T00:34:15.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4100e00 0x7effe4198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:15.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe2000700 1 -- 172.21.15.99:0/653961197 learned_addr learned my addr 172.21.15.99:0/653961197 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:15.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.825+0000 7effe3400700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe4102920 0x7effe41a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.826+0000 7effe2000700 1 -- 172.21.15.99:0/653961197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe4102920 msgr2=0x7effe41a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.826+0000 7effe2000700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe4102920 0x7effe41a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.826+0000 7effe2000700 1 -- 172.21.15.99:0/653961197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4100e00 msgr2=0x7effe4198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.826+0000 7effe2000700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4100e00 0x7effe4198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.826+0000 7effe2000700 1 -- 172.21.15.99:0/653961197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd40095f0 con 0x7effe4101f60 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.826+0000 7effe3400700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe4102920 0x7effe41a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.826+0000 7effe2a00700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4100e00 0x7effe4198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.826+0000 7effe2000700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe4101f60 0x7effe4199460 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7effd000e9f0 tx=0x7effd000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.826+0000 7effdb400700 1 -- 172.21.15.99:0/653961197 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd00097b0 con 0x7effe4101f60 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.826+0000 7effdb400700 1 -- 172.21.15.99:0/653961197 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7effd000ee90 con 0x7effe4101f60 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.826+0000 7effe9600700 1 -- 172.21.15.99:0/653961197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe41a0fc0 con 0x7effe4101f60 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.827+0000 7effdb400700 1 -- 172.21.15.99:0/653961197 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0005450 con 0x7effe4101f60 2024-03-30T00:34:15.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.827+0000 7effe9600700 1 -- 172.21.15.99:0/653961197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe41a1430 con 0x7effe4101f60 2024-03-30T00:34:15.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.828+0000 7effe9600700 1 -- 172.21.15.99:0/653961197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe404ea80 con 0x7effe4101f60 2024-03-30T00:34:15.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.830+0000 7effdb400700 1 -- 172.21.15.99:0/653961197 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7effd0005630 con 0x7effe4101f60 2024-03-30T00:34:15.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.831+0000 7effdb400700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1230 0x7effc80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:15.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.831+0000 7effdb400700 1 -- 172.21.15.99:0/653961197 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7effd0014070 con 0x7effe4101f60 2024-03-30T00:34:15.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.831+0000 7effe2a00700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1230 0x7effc80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:15.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.832+0000 7effe2a00700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1230 0x7effc80a36f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7effd4006010 tx=0x7effd400b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:15.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:15.835+0000 7effdb400700 1 -- 172.21.15.99:0/653961197 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7effd0096200 con 0x7effe4101f60 2024-03-30T00:34:16.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.059+0000 7effe9600700 1 -- 172.21.15.99:0/653961197 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7effe4074700 con 0x7effc80a1230 2024-03-30T00:34:16.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.065+0000 7effdb400700 1 -- 172.21.15.99:0/653961197 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7effe4074700 con 0x7effc80a1230 2024-03-30T00:34:16.073 INFO:teuthology.orchestra.run.smithi099.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-30T00:34:16.073 INFO:teuthology.orchestra.run.smithi099.stdout:mds.all 12/12 62s ago 96s smithi099=a;smithi099=d;smithi099=g;smithi099=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-03-30T00:34:16.073 INFO:teuthology.orchestra.run.smithi099.stdout:mgr 3/3 62s ago 6m smithi099=x;smithi142=y;smithi161=z;count:3 2024-03-30T00:34:16.073 INFO:teuthology.orchestra.run.smithi099.stdout:mon 3/3 62s ago 6m smithi099:172.21.15.99=a;smithi142:172.21.15.142=b;smithi161:172.21.15.161=c;count:3 2024-03-30T00:34:16.073 INFO:teuthology.orchestra.run.smithi099.stdout:osd 12 62s ago - 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 -- 172.21.15.99:0/653961197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1230 msgr2=0x7effc80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1230 0x7effc80a36f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7effd4006010 tx=0x7effd400b5b0 comp rx=0 tx=0).stop 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 -- 172.21.15.99:0/653961197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe4101f60 msgr2=0x7effe4199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe4101f60 0x7effe4199460 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7effd000e9f0 tx=0x7effd000ed00 comp rx=0 tx=0).stop 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 -- 172.21.15.99:0/653961197 shutdown_connections 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe4102920 0x7effe41a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1230 0x7effc80a36f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe4100e00 0x7effe4198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 --2- 172.21.15.99:0/653961197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe4101f60 0x7effe4199460 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 -- 172.21.15.99:0/653961197 >> 172.21.15.99:0/653961197 conn(0x7effe40fc3c0 msgr2=0x7effe4072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 -- 172.21.15.99:0/653961197 shutdown_connections 2024-03-30T00:34:16.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:16.068+0000 7effe9600700 1 -- 172.21.15.99:0/653961197 wait complete. 2024-03-30T00:34:16.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:16 smithi099 ceph-mon[38079]: pgmap v288: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-30T00:34:16.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:16 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.525 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch host ls' 2024-03-30T00:34:16.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:16 smithi142 ceph-mon[41779]: pgmap v288: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-30T00:34:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:16 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:16 smithi161 ceph-mon[42221]: pgmap v288: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-30T00:34:16.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:16 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:16.960 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:34:17.442 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:17 smithi099 ceph-mon[38079]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:17.442 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:17 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:17.442 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:17 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:17.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:17 smithi142 ceph-mon[41779]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:17.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:17 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:17.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:17 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:17.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:17 smithi161 ceph-mon[42221]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:17.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:17 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:17.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:17 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.119+0000 7f873ca00700 1 -- 172.21.15.99:0/3504421132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8738100e00 msgr2=0x7f8738101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.119+0000 7f873ca00700 1 --2- 172.21.15.99:0/3504421132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8738100e00 0x7f8738101220 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8728009990 tx=0x7f8728009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.119+0000 7f873ca00700 1 -- 172.21.15.99:0/3504421132 shutdown_connections 2024-03-30T00:34:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.119+0000 7f873ca00700 1 --2- 172.21.15.99:0/3504421132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8738102920 0x7f8738109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.119+0000 7f873ca00700 1 --2- 172.21.15.99:0/3504421132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8738100e00 0x7f8738101220 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:18.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.119+0000 7f873ca00700 1 --2- 172.21.15.99:0/3504421132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8738101f60 0x7f87381023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:18.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.119+0000 7f873ca00700 1 -- 172.21.15.99:0/3504421132 >> 172.21.15.99:0/3504421132 conn(0x7f87380fc3c0 msgr2=0x7f87380fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:18.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.119+0000 7f873ca00700 1 -- 172.21.15.99:0/3504421132 shutdown_connections 2024-03-30T00:34:18.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.119+0000 7f873ca00700 1 -- 172.21.15.99:0/3504421132 wait complete. 2024-03-30T00:34:18.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.120+0000 7f873ca00700 1 Processor -- start 2024-03-30T00:34:18.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.120+0000 7f873ca00700 1 -- start start 2024-03-30T00:34:18.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f873ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8738100e00 0x7f8738198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:18.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f873ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8738101f60 0x7f87381993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:18.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f873ca00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8738102920 0x7f87381a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:18.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f873ca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87381a09a0 con 0x7f8738100e00 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f873ca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87381a0ae0 con 0x7f8738101f60 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f873ca00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87381a0c70 con 0x7f8738102920 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f8735e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8738100e00 0x7f8738198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f8735e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8738100e00 0x7f8738198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50284/0 (socket says 172.21.15.99:50284) 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f8735e00700 1 -- 172.21.15.99:0/992691308 learned_addr learned my addr 172.21.15.99:0/992691308 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f8735400700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8738101f60 0x7f87381993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f8736800700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8738102920 0x7f87381a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f8735e00700 1 -- 172.21.15.99:0/992691308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8738102920 msgr2=0x7f87381a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f8735e00700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8738102920 0x7f87381a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f8735e00700 1 -- 172.21.15.99:0/992691308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8738101f60 msgr2=0x7f87381993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f8735e00700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8738101f60 0x7f87381993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.121+0000 7f8735e00700 1 -- 172.21.15.99:0/992691308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87280095f0 con 0x7f8738100e00 2024-03-30T00:34:18.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.122+0000 7f8735e00700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8738100e00 0x7f8738198e60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f87280098e0 tx=0x7f8728005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:18.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.122+0000 7f872ea00700 1 -- 172.21.15.99:0/992691308 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872801d070 con 0x7f8738100e00 2024-03-30T00:34:18.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.122+0000 7f872ea00700 1 -- 172.21.15.99:0/992691308 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8728003ec0 con 0x7f8738100e00 2024-03-30T00:34:18.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.122+0000 7f873ca00700 1 -- 172.21.15.99:0/992691308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87381a0ef0 con 0x7f8738100e00 2024-03-30T00:34:18.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.122+0000 7f872ea00700 1 -- 172.21.15.99:0/992691308 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728022b40 con 0x7f8738100e00 2024-03-30T00:34:18.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.122+0000 7f873ca00700 1 -- 172.21.15.99:0/992691308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87381a1250 con 0x7f8738100e00 2024-03-30T00:34:18.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.123+0000 7f873ca00700 1 -- 172.21.15.99:0/992691308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873804ea80 con 0x7f8738100e00 2024-03-30T00:34:18.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.125+0000 7f872ea00700 1 -- 172.21.15.99:0/992691308 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8728004030 con 0x7f8738100e00 2024-03-30T00:34:18.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.126+0000 7f872ea00700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f871c0a1230 0x7f871c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:18.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.126+0000 7f872ea00700 1 -- 172.21.15.99:0/992691308 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f87280ce0a0 con 0x7f8738100e00 2024-03-30T00:34:18.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.127+0000 7f8735400700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f871c0a1230 0x7f871c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:18.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.127+0000 7f8735400700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f871c0a1230 0x7f871c0a36f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8724005950 tx=0x7f872400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:18.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.130+0000 7f872ea00700 1 -- 172.21.15.99:0/992691308 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f87280976f0 con 0x7f8738100e00 2024-03-30T00:34:18.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.339+0000 7f873ca00700 1 -- 172.21.15.99:0/992691308 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8738108ee0 con 0x7f871c0a1230 2024-03-30T00:34:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.341+0000 7f872ea00700 1 -- 172.21.15.99:0/992691308 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f8738108ee0 con 0x7f871c0a1230 2024-03-30T00:34:18.349 INFO:teuthology.orchestra.run.smithi099.stdout:HOST ADDR LABELS STATUS 2024-03-30T00:34:18.349 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099 172.21.15.99 2024-03-30T00:34:18.349 INFO:teuthology.orchestra.run.smithi099.stdout:smithi142 172.21.15.142 2024-03-30T00:34:18.349 INFO:teuthology.orchestra.run.smithi099.stdout:smithi161 172.21.15.161 2024-03-30T00:34:18.349 INFO:teuthology.orchestra.run.smithi099.stdout:3 hosts in cluster 2024-03-30T00:34:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.343+0000 7f873ca00700 1 -- 172.21.15.99:0/992691308 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f871c0a1230 msgr2=0x7f871c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.343+0000 7f873ca00700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f871c0a1230 0x7f871c0a36f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8724005950 tx=0x7f872400b410 comp rx=0 tx=0).stop 2024-03-30T00:34:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.343+0000 7f873ca00700 1 -- 172.21.15.99:0/992691308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8738100e00 msgr2=0x7f8738198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.343+0000 7f873ca00700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8738100e00 0x7f8738198e60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f87280098e0 tx=0x7f8728005740 comp rx=0 tx=0).stop 2024-03-30T00:34:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.343+0000 7f873ca00700 1 -- 172.21.15.99:0/992691308 shutdown_connections 2024-03-30T00:34:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.343+0000 7f873ca00700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8738102920 0x7f87381a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.343+0000 7f873ca00700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8738100e00 0x7f8738198e60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.343+0000 7f873ca00700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f871c0a1230 0x7f871c0a36f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.344+0000 7f873ca00700 1 --2- 172.21.15.99:0/992691308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8738101f60 0x7f87381993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.344+0000 7f873ca00700 1 -- 172.21.15.99:0/992691308 >> 172.21.15.99:0/992691308 conn(0x7f87380fc3c0 msgr2=0x7f87381077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.344+0000 7f873ca00700 1 -- 172.21.15.99:0/992691308 shutdown_connections 2024-03-30T00:34:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:18.344+0000 7f873ca00700 1 -- 172.21.15.99:0/992691308 wait complete. 2024-03-30T00:34:18.553 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:18 smithi099 ceph-mon[38079]: pgmap v289: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-30T00:34:18.553 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:18 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:34:18.554 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:18 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:34:18.554 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:18 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:18.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:18 smithi142 ceph-mon[41779]: pgmap v289: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-30T00:34:18.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:18 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:34:18.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:18 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:34:18.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:18 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:18.712 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch device ls' 2024-03-30T00:34:18.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:18 smithi161 ceph-mon[42221]: pgmap v289: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-30T00:34:18.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:18 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:34:18.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:18 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:34:18.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:18 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:34:19.075 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:34:19.559 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:19 smithi099 ceph-mon[38079]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:19.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:19 smithi142 ceph-mon[41779]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:19.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:19 smithi161 ceph-mon[42221]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:20.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.193+0000 7f1c0da00700 1 -- 172.21.15.99:0/2020525442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c08101f60 msgr2=0x7f1c081023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:20.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.193+0000 7f1c0da00700 1 --2- 172.21.15.99:0/2020525442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c08101f60 0x7f1c081023e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009990 tx=0x7f1bf4009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:20.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.193+0000 7f1c0da00700 1 -- 172.21.15.99:0/2020525442 shutdown_connections 2024-03-30T00:34:20.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.193+0000 7f1c0da00700 1 --2- 172.21.15.99:0/2020525442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c08102920 0x7f1c08109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:20.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.193+0000 7f1c0da00700 1 --2- 172.21.15.99:0/2020525442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c08100e00 0x7f1c08101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:20.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.193+0000 7f1c0da00700 1 --2- 172.21.15.99:0/2020525442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c08101f60 0x7f1c081023e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:20.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.193+0000 7f1c0da00700 1 -- 172.21.15.99:0/2020525442 >> 172.21.15.99:0/2020525442 conn(0x7f1c080fc3c0 msgr2=0x7f1c080fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:20.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.194+0000 7f1c0da00700 1 -- 172.21.15.99:0/2020525442 shutdown_connections 2024-03-30T00:34:20.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.194+0000 7f1c0da00700 1 -- 172.21.15.99:0/2020525442 wait complete. 2024-03-30T00:34:20.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c0da00700 1 Processor -- start 2024-03-30T00:34:20.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c0da00700 1 -- start start 2024-03-30T00:34:20.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c0da00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c08100e00 0x7f1c08198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:20.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c0da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c08101f60 0x7f1c08199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:20.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c0da00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c08102920 0x7f1c081a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c0da00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081a08e0 con 0x7f1c08102920 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c0da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081a0a20 con 0x7f1c08101f60 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c0da00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c081a0be0 con 0x7f1c08100e00 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c07400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c08102920 0x7f1c081a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c07400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c08102920 0x7f1c081a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50290/0 (socket says 172.21.15.99:50290) 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c06000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c08101f60 0x7f1c08199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c07400700 1 -- 172.21.15.99:0/154704702 learned_addr learned my addr 172.21.15.99:0/154704702 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.195+0000 7f1c06a00700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c08100e00 0x7f1c08198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.196+0000 7f1c07400700 1 -- 172.21.15.99:0/154704702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c08100e00 msgr2=0x7f1c08198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.196+0000 7f1c07400700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c08100e00 0x7f1c08198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.196+0000 7f1c07400700 1 -- 172.21.15.99:0/154704702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c08101f60 msgr2=0x7f1c08199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.196+0000 7f1c07400700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c08101f60 0x7f1c08199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.196+0000 7f1c07400700 1 -- 172.21.15.99:0/154704702 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf40095f0 con 0x7f1c08102920 2024-03-30T00:34:20.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.196+0000 7f1c07400700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c08102920 0x7f1c081a0310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1c00007d70 tx=0x7f1c0000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:20.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.196+0000 7f1bff400700 1 -- 172.21.15.99:0/154704702 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000ac90 con 0x7f1c08102920 2024-03-30T00:34:20.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.196+0000 7f1bff400700 1 -- 172.21.15.99:0/154704702 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c0000adf0 con 0x7f1c08102920 2024-03-30T00:34:20.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.196+0000 7f1bff400700 1 -- 172.21.15.99:0/154704702 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c000115e0 con 0x7f1c08102920 2024-03-30T00:34:20.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.196+0000 7f1c0da00700 1 -- 172.21.15.99:0/154704702 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c081a0ec0 con 0x7f1c08102920 2024-03-30T00:34:20.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.197+0000 7f1c0da00700 1 -- 172.21.15.99:0/154704702 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c081a1280 con 0x7f1c08102920 2024-03-30T00:34:20.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.198+0000 7f1c0da00700 1 -- 172.21.15.99:0/154704702 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0804ea80 con 0x7f1c08102920 2024-03-30T00:34:20.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.199+0000 7f1bff400700 1 -- 172.21.15.99:0/154704702 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1c00012420 con 0x7f1c08102920 2024-03-30T00:34:20.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.201+0000 7f1bff400700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bec0a11b0 0x7f1bec0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:20.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.201+0000 7f1bff400700 1 -- 172.21.15.99:0/154704702 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1c000ccb60 con 0x7f1c08102920 2024-03-30T00:34:20.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.201+0000 7f1c06a00700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bec0a11b0 0x7f1bec0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:20.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.201+0000 7f1c06a00700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bec0a11b0 0x7f1bec0a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1c0819a270 tx=0x7f1bf8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:20.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.205+0000 7f1bff400700 1 -- 172.21.15.99:0/154704702 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1c00096230 con 0x7f1c08102920 2024-03-30T00:34:20.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.417+0000 7f1c0da00700 1 -- 172.21.15.99:0/154704702 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c08074700 con 0x7f1bec0a11b0 2024-03-30T00:34:20.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.419+0000 7f1bff400700 1 -- 172.21.15.99:0/154704702 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f1c08074700 con 0x7f1bec0a11b0 2024-03-30T00:34:20.427 INFO:teuthology.orchestra.run.smithi099.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-30T00:34:20.427 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040101400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-30T00:34:20.427 INFO:teuthology.orchestra.run.smithi099.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-30T00:34:20.428 INFO:teuthology.orchestra.run.smithi099.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 109s ago Has a FileSystem, LVM detected 2024-03-30T00:34:20.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 -- 172.21.15.99:0/154704702 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bec0a11b0 msgr2=0x7f1bec0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:20.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bec0a11b0 0x7f1bec0a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1c0819a270 tx=0x7f1bf8009260 comp rx=0 tx=0).stop 2024-03-30T00:34:20.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 -- 172.21.15.99:0/154704702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c08102920 msgr2=0x7f1c081a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:20.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c08102920 0x7f1c081a0310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1c00007d70 tx=0x7f1c0000a340 comp rx=0 tx=0).stop 2024-03-30T00:34:20.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 -- 172.21.15.99:0/154704702 shutdown_connections 2024-03-30T00:34:20.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c08102920 0x7f1c081a0310 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:20.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c08100e00 0x7f1c08198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:20.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1bec0a11b0 0x7f1bec0a3670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:20.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 --2- 172.21.15.99:0/154704702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c08101f60 0x7f1c08199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:20.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 -- 172.21.15.99:0/154704702 >> 172.21.15.99:0/154704702 conn(0x7f1c080fc3c0 msgr2=0x7f1c08072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:20.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 -- 172.21.15.99:0/154704702 shutdown_connections 2024-03-30T00:34:20.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:20.422+0000 7f1c0da00700 1 -- 172.21.15.99:0/154704702 wait complete. 2024-03-30T00:34:20.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:20 smithi142 ceph-mon[41779]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-30T00:34:20.632 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:20 smithi099 ceph-mon[38079]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-30T00:34:20.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:20 smithi161 ceph-mon[42221]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-30T00:34:20.822 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-30T00:34:20.830 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-03-30T00:34:20.831 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-03-30T00:34:21.190 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:34:21.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:21 smithi142 ceph-mon[41779]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:21.683 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:21 smithi099 ceph-mon[38079]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:21.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:21 smithi161 ceph-mon[42221]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T00:34:22.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.299+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2076585264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3ac100e00 msgr2=0x7fe3ac101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:22.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.299+0000 7fe3b3e00700 1 --2- 172.21.15.99:0/2076585264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3ac100e00 0x7fe3ac101220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009990 tx=0x7fe3a0009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:22.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.299+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2076585264 shutdown_connections 2024-03-30T00:34:22.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.299+0000 7fe3b3e00700 1 --2- 172.21.15.99:0/2076585264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3ac102920 0x7fe3ac109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:22.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.299+0000 7fe3b3e00700 1 --2- 172.21.15.99:0/2076585264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3ac100e00 0x7fe3ac101220 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:22.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.299+0000 7fe3b3e00700 1 --2- 172.21.15.99:0/2076585264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3ac101f60 0x7fe3ac1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:22.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.299+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2076585264 >> 172.21.15.99:0/2076585264 conn(0x7fe3ac0fc3c0 msgr2=0x7fe3ac0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:22.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.300+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2076585264 shutdown_connections 2024-03-30T00:34:22.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.300+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2076585264 wait complete. 2024-03-30T00:34:22.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b3e00700 1 Processor -- start 2024-03-30T00:34:22.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b3e00700 1 -- start start 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3ac100e00 0x7fe3ac196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3ac101f60 0x7fe3ac197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b3e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3ac102920 0x7fe3ac19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b3e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac19e700 con 0x7fe3ac100e00 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b3e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac19e840 con 0x7fe3ac101f60 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b3e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac19e9d0 con 0x7fe3ac102920 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3ac100e00 0x7fe3ac196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3ac100e00 0x7fe3ac196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50300/0 (socket says 172.21.15.99:50300) 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b1400700 1 -- 172.21.15.99:0/2481410224 learned_addr learned my addr 172.21.15.99:0/2481410224 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.301+0000 7fe3b1e00700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3ac102920 0x7fe3ac19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3b0a00700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3ac101f60 0x7fe3ac197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3b1400700 1 -- 172.21.15.99:0/2481410224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3ac102920 msgr2=0x7fe3ac19e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3b1400700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3ac102920 0x7fe3ac19e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3b1400700 1 -- 172.21.15.99:0/2481410224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3ac101f60 msgr2=0x7fe3ac197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3b1400700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3ac101f60 0x7fe3ac197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3b1400700 1 -- 172.21.15.99:0/2481410224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00095f0 con 0x7fe3ac100e00 2024-03-30T00:34:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3b0a00700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3ac101f60 0x7fe3ac197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3b1400700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3ac100e00 0x7fe3ac196bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0005910 tx=0x7fe3a0003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3aa000700 1 -- 172.21.15.99:0/2481410224 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a001d070 con 0x7fe3ac100e00 2024-03-30T00:34:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3aa000700 1 -- 172.21.15.99:0/2481410224 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3a000bd10 con 0x7fe3ac100e00 2024-03-30T00:34:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2481410224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ac19ec50 con 0x7fe3ac100e00 2024-03-30T00:34:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3aa000700 1 -- 172.21.15.99:0/2481410224 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0022da0 con 0x7fe3ac100e00 2024-03-30T00:34:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.302+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2481410224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ac19f060 con 0x7fe3ac100e00 2024-03-30T00:34:22.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.303+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2481410224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ac108230 con 0x7fe3ac100e00 2024-03-30T00:34:22.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.305+0000 7fe3aa000700 1 -- 172.21.15.99:0/2481410224 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fe3a0022570 con 0x7fe3ac100e00 2024-03-30T00:34:22.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.307+0000 7fe3aa000700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe3940a11b0 0x7fe3940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:22.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.307+0000 7fe3aa000700 1 -- 172.21.15.99:0/2481410224 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe3a00ce5a0 con 0x7fe3ac100e00 2024-03-30T00:34:22.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.307+0000 7fe3b0a00700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe3940a11b0 0x7fe3940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:22.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.307+0000 7fe3b0a00700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe3940a11b0 0x7fe3940a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe398005fd0 tx=0x7fe398005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:22.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.309+0000 7fe3aa000700 1 -- 172.21.15.99:0/2481410224 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe3a0097bf0 con 0x7fe3ac100e00 2024-03-30T00:34:22.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.554+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2481410224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe3ac04ea80 con 0x7fe3ac100e00 2024-03-30T00:34:22.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.554+0000 7fe3aa000700 1 -- 172.21.15.99:0/2481410224 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2893 (secure 0 0 0) 0x7fe3a009b210 con 0x7fe3ac100e00 2024-03-30T00:34:22.562 INFO:teuthology.orchestra.run.smithi099.stdout:e23 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:legacy client fscid: 1 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:Filesystem 'cephfs' (1) 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:fs_name cephfs 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 23 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:created 2024-03-30T00:33:15.648899+0000 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:modified 2024-03-30T00:33:21.119410+0000 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:tableserver 0 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:root 0 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:session_timeout 300 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:session_autoclose 300 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:max_file_size 1099511627776 2024-03-30T00:34:22.563 INFO:teuthology.orchestra.run.smithi099.stdout:max_xattr_size 65536 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:required_client_features {} 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:last_failure 0 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:last_failure_osd_epoch 0 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:max_mds 3 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:in 0,1,2 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:up {0=24572,1=24559,2=24502} 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:failed 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:damaged 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:stopped 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:data_pools [3] 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:metadata_pool 2 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:inline_data disabled 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:balancer 2024-03-30T00:34:22.564 INFO:teuthology.orchestra.run.smithi099.stdout:bal_rank_mask -1 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:standby_count_wanted 0 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b{0:24572} state up:active seq 3 addr [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.i{0:24544} state up:standby-replay seq 1 addr [v2:172.21.15.161:6838/1486341215,v1:172.21.15.161:6839/1486341215] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a{1:24559} state up:active seq 5 addr [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.e{1:24503} state up:standby-replay seq 1 addr [v2:172.21.15.142:6834/1641167461,v1:172.21.15.142:6835/1641167461] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.f{2:24502} state up:active seq 8 addr [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.k{2:24473} state up:standby-replay seq 1 addr [v2:172.21.15.142:6832/780858221,v1:172.21.15.142:6833/780858221] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:Standby daemons: 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.j{-1:14607} state up:standby seq 1 addr [v2:172.21.15.99:6834/380315573,v1:172.21.15.99:6835/380315573] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d{-1:14619} state up:standby seq 1 addr [v2:172.21.15.99:6836/4277780379,v1:172.21.15.99:6837/4277780379] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c{-1:14655} state up:standby seq 1 addr [v2:172.21.15.161:6836/2189606703,v1:172.21.15.161:6837/2189606703] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.h{-1:14667} state up:standby seq 1 addr [v2:172.21.15.142:6836/364916855,v1:172.21.15.142:6837/364916855] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.565 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.g{-1:14685} state up:standby seq 1 addr [v2:172.21.15.99:6838/757502672,v1:172.21.15.99:6839/757502672] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.566 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.l{-1:24454} state up:standby seq 1 addr [v2:172.21.15.161:6832/3055946850,v1:172.21.15.161:6833/3055946850] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T00:34:22.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2481410224 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe3940a11b0 msgr2=0x7fe3940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:22.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe3940a11b0 0x7fe3940a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe398005fd0 tx=0x7fe398005e20 comp rx=0 tx=0).stop 2024-03-30T00:34:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2481410224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3ac100e00 msgr2=0x7fe3ac196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3ac100e00 0x7fe3ac196bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0005910 tx=0x7fe3a0003a50 comp rx=0 tx=0).stop 2024-03-30T00:34:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2481410224 shutdown_connections 2024-03-30T00:34:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3ac102920 0x7fe3ac19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3ac100e00 0x7fe3ac196bc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe3940a11b0 0x7fe3940a3670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 --2- 172.21.15.99:0/2481410224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3ac101f60 0x7fe3ac197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2481410224 >> 172.21.15.99:0/2481410224 conn(0x7fe3ac0fc3c0 msgr2=0x7fe3ac105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2481410224 shutdown_connections 2024-03-30T00:34:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:22.557+0000 7fe3b3e00700 1 -- 172.21.15.99:0/2481410224 wait complete. 2024-03-30T00:34:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2024-03-30T00:34:22.587 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:22 smithi099 ceph-mon[38079]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-30T00:34:22.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:22 smithi161 ceph-mon[42221]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-30T00:34:22.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:22 smithi142 ceph-mon[41779]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-30T00:34:22.948 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- bash -c 'ceph osd dump' 2024-03-30T00:34:23.308 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:34:23.610 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:23 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2481410224' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-30T00:34:23.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:23 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2481410224' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-30T00:34:23.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:23 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2481410224' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-30T00:34:24.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.548+0000 7faa78a00700 1 -- 172.21.15.99:0/1555670068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa74101f60 msgr2=0x7faa741023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:24.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.548+0000 7faa78a00700 1 --2- 172.21.15.99:0/1555670068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa74101f60 0x7faa741023e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa60009990 tx=0x7faa60009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:24.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.548+0000 7faa78a00700 1 -- 172.21.15.99:0/1555670068 shutdown_connections 2024-03-30T00:34:24.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.548+0000 7faa78a00700 1 --2- 172.21.15.99:0/1555670068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa74102920 0x7faa74109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.548+0000 7faa78a00700 1 --2- 172.21.15.99:0/1555670068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa74100e00 0x7faa74101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.548+0000 7faa78a00700 1 --2- 172.21.15.99:0/1555670068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa74101f60 0x7faa741023e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.548+0000 7faa78a00700 1 -- 172.21.15.99:0/1555670068 >> 172.21.15.99:0/1555670068 conn(0x7faa740fc3c0 msgr2=0x7faa740fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.548+0000 7faa78a00700 1 -- 172.21.15.99:0/1555670068 shutdown_connections 2024-03-30T00:34:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.548+0000 7faa78a00700 1 -- 172.21.15.99:0/1555670068 wait complete. 2024-03-30T00:34:24.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.549+0000 7faa78a00700 1 Processor -- start 2024-03-30T00:34:24.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.549+0000 7faa78a00700 1 -- start start 2024-03-30T00:34:24.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.549+0000 7faa78a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa74100e00 0x7faa74074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:24.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.549+0000 7faa78a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa74101f60 0x7faa740731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:24.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.549+0000 7faa78a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa74102920 0x7faa740736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:24.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.549+0000 7faa78a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa74073e80 con 0x7faa74102920 2024-03-30T00:34:24.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.549+0000 7faa78a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa74073ff0 con 0x7faa74101f60 2024-03-30T00:34:24.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.549+0000 7faa78a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa741a0d20 con 0x7faa74100e00 2024-03-30T00:34:24.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.550+0000 7faa72800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa74102920 0x7faa740736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:24.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.550+0000 7faa72800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa74102920 0x7faa740736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50312/0 (socket says 172.21.15.99:50312) 2024-03-30T00:34:24.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.550+0000 7faa72800700 1 -- 172.21.15.99:0/2135500934 learned_addr learned my addr 172.21.15.99:0/2135500934 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.550+0000 7faa71400700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa74101f60 0x7faa740731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.550+0000 7faa71e00700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa74100e00 0x7faa74074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.550+0000 7faa72800700 1 -- 172.21.15.99:0/2135500934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa74100e00 msgr2=0x7faa74074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.550+0000 7faa72800700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa74100e00 0x7faa74074b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.550+0000 7faa72800700 1 -- 172.21.15.99:0/2135500934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa74101f60 msgr2=0x7faa740731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.550+0000 7faa72800700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa74101f60 0x7faa740731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.550+0000 7faa72800700 1 -- 172.21.15.99:0/2135500934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa600095f0 con 0x7faa74102920 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.550+0000 7faa72800700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa74102920 0x7faa740736e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa6c007ee0 tx=0x7faa6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.551+0000 7faa6aa00700 1 -- 172.21.15.99:0/2135500934 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00dd20 con 0x7faa74102920 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.551+0000 7faa6aa00700 1 -- 172.21.15.99:0/2135500934 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa6c00f040 con 0x7faa74102920 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.551+0000 7faa78a00700 1 -- 172.21.15.99:0/2135500934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa741a0fb0 con 0x7faa74102920 2024-03-30T00:34:24.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.551+0000 7faa6aa00700 1 -- 172.21.15.99:0/2135500934 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c004750 con 0x7faa74102920 2024-03-30T00:34:24.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.551+0000 7faa78a00700 1 -- 172.21.15.99:0/2135500934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa741a1420 con 0x7faa74102920 2024-03-30T00:34:24.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.552+0000 7faa78a00700 1 -- 172.21.15.99:0/2135500934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa741083c0 con 0x7faa74102920 2024-03-30T00:34:24.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.554+0000 7faa6aa00700 1 -- 172.21.15.99:0/2135500934 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7faa6c003a00 con 0x7faa74102920 2024-03-30T00:34:24.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.555+0000 7faa6aa00700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faa580a11b0 0x7faa580a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:24.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.555+0000 7faa6aa00700 1 -- 172.21.15.99:0/2135500934 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7faa6c0cc2b0 con 0x7faa74102920 2024-03-30T00:34:24.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.555+0000 7faa71e00700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faa580a11b0 0x7faa580a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:24.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.556+0000 7faa71e00700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faa580a11b0 0x7faa580a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa64009c60 tx=0x7faa64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:24.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.559+0000 7faa6aa00700 1 -- 172.21.15.99:0/2135500934 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7faa6c0958d0 con 0x7faa74102920 2024-03-30T00:34:24.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:24 smithi161 ceph-mon[42221]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-30T00:34:24.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:24 smithi099 ceph-mon[38079]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-30T00:34:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.766+0000 7faa78a00700 1 -- 172.21.15.99:0/2135500934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7faa7410c6a0 con 0x7faa74102920 2024-03-30T00:34:24.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.767+0000 7faa6aa00700 1 -- 172.21.15.99:0/2135500934 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) v1 ==== 56+0+5310 (secure 0 0 0) 0x7faa6c0048b0 con 0x7faa74102920 2024-03-30T00:34:24.775 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 85 2024-03-30T00:34:24.775 INFO:teuthology.orchestra.run.smithi099.stdout:fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:34:24.775 INFO:teuthology.orchestra.run.smithi099.stdout:created 2024-03-30T00:26:14.505850+0000 2024-03-30T00:34:24.775 INFO:teuthology.orchestra.run.smithi099.stdout:modified 2024-03-30T00:34:10.248126+0000 2024-03-30T00:34:24.775 INFO:teuthology.orchestra.run.smithi099.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-30T00:34:24.775 INFO:teuthology.orchestra.run.smithi099.stdout:crush_version 26 2024-03-30T00:34:24.775 INFO:teuthology.orchestra.run.smithi099.stdout:full_ratio 0.95 2024-03-30T00:34:24.775 INFO:teuthology.orchestra.run.smithi099.stdout:backfillfull_ratio 0.9 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:nearfull_ratio 0.85 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:require_min_compat_client luminous 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:min_compat_client luminous 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:require_osd_release reef 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:stretch_mode_enabled false 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.69 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:max_osd 12 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6802/3911742857,v1:172.21.15.99:6803/3911742857] [v2:172.21.15.99:6804/3911742857,v1:172.21.15.99:6805/3911742857] exists,up c244d549-0aff-48af-9523-f7096beda3e4 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/912538361,v1:172.21.15.142:6801/912538361] [v2:172.21.15.142:6802/912538361,v1:172.21.15.142:6803/912538361] exists,up 0b541e0e-4b00-4af5-92c5-adb75a359bbc 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/1904456041,v1:172.21.15.161:6801/1904456041] [v2:172.21.15.161:6802/1904456041,v1:172.21.15.161:6803/1904456041] exists,up e0faf760-1f43-4799-bc8d-bdc2f8ba5386 2024-03-30T00:34:24.776 INFO:teuthology.orchestra.run.smithi099.stdout:osd.3 up in weight 1 up_from 24 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6810/1432967840,v1:172.21.15.99:6811/1432967840] [v2:172.21.15.99:6812/1432967840,v1:172.21.15.99:6813/1432967840] exists,up 5a75bf9a-8f16-4d28-9685-cdbc9a5a9325 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:osd.4 up in weight 1 up_from 31 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6808/1812504396,v1:172.21.15.142:6809/1812504396] [v2:172.21.15.142:6810/1812504396,v1:172.21.15.142:6811/1812504396] exists,up 68d47704-9d21-440d-a85a-46cd077d50d4 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:osd.5 up in weight 1 up_from 37 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/235239367,v1:172.21.15.161:6809/235239367] [v2:172.21.15.161:6810/235239367,v1:172.21.15.161:6811/235239367] exists,up 64cdf569-3dfe-4792-bcb0-f0031e0dbf7d 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:osd.6 up in weight 1 up_from 45 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6818/2055935080,v1:172.21.15.99:6819/2055935080] [v2:172.21.15.99:6820/2055935080,v1:172.21.15.99:6821/2055935080] exists,up ce11a006-0d8d-45d4-a9a8-e465307c6025 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:osd.7 up in weight 1 up_from 50 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/445950815,v1:172.21.15.142:6817/445950815] [v2:172.21.15.142:6818/445950815,v1:172.21.15.142:6819/445950815] exists,up f762698a-4112-4f4e-9658-3f91393ef1a1 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:osd.8 up in weight 1 up_from 58 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/2321115120,v1:172.21.15.161:6817/2321115120] [v2:172.21.15.161:6818/2321115120,v1:172.21.15.161:6819/2321115120] exists,up 9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:osd.9 up in weight 1 up_from 66 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6826/3086664896,v1:172.21.15.99:6827/3086664896] [v2:172.21.15.99:6828/3086664896,v1:172.21.15.99:6829/3086664896] exists,up 3cb6f263-248a-417e-b8a4-37afed246071 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:osd.10 up in weight 1 up_from 71 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/3224161760,v1:172.21.15.142:6825/3224161760] [v2:172.21.15.142:6826/3224161760,v1:172.21.15.142:6827/3224161760] exists,up c2d0d43f-b79c-4830-926f-8d51fb41eccd 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:osd.11 up in weight 1 up_from 78 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/3737118097,v1:172.21.15.161:6825/3737118097] [v2:172.21.15.161:6826/3737118097,v1:172.21.15.161:6827/3737118097] exists,up 2d7a3b48-85d3-41d2-8a24-288b8250e803 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.0 [4,6] 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.1 [4,9] 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.6 [3,7] 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.e [4,6] 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.10 [5,9] 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.11 [4,8] 2024-03-30T00:34:24.777 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.1b [5,9] 2024-03-30T00:34:24.778 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.28 [3,8,5,7] 2024-03-30T00:34:24.778 INFO:teuthology.orchestra.run.smithi099.stdout:pg_upmap_items 3.3d [1,8] 2024-03-30T00:34:24.778 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6801/1748928762 expires 2024-03-31T00:27:08.932957+0000 2024-03-30T00:34:24.778 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/2943078453 expires 2024-03-31T00:27:08.932957+0000 2024-03-30T00:34:24.778 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6801/1133935143 expires 2024-03-31T00:26:45.301121+0000 2024-03-30T00:34:24.779 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6800/1748928762 expires 2024-03-31T00:27:08.932957+0000 2024-03-30T00:34:24.779 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:6800/1133935143 expires 2024-03-31T00:26:45.301121+0000 2024-03-30T00:34:24.779 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/479443591 expires 2024-03-31T00:26:45.301121+0000 2024-03-30T00:34:24.779 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/4006319857 expires 2024-03-31T00:26:45.301121+0000 2024-03-30T00:34:24.779 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/2253836899 expires 2024-03-31T00:27:08.932957+0000 2024-03-30T00:34:24.779 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/2565551826 expires 2024-03-31T00:26:45.301121+0000 2024-03-30T00:34:24.779 INFO:teuthology.orchestra.run.smithi099.stdout:blocklist 172.21.15.99:0/1147023397 expires 2024-03-31T00:27:08.932957+0000 2024-03-30T00:34:24.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.769+0000 7faa78a00700 1 -- 172.21.15.99:0/2135500934 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faa580a11b0 msgr2=0x7faa580a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:24.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.769+0000 7faa78a00700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faa580a11b0 0x7faa580a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa64009c60 tx=0x7faa64009260 comp rx=0 tx=0).stop 2024-03-30T00:34:24.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.769+0000 7faa78a00700 1 -- 172.21.15.99:0/2135500934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa74102920 msgr2=0x7faa740736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:24.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.769+0000 7faa78a00700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa74102920 0x7faa740736e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa6c007ee0 tx=0x7faa6c00d3b0 comp rx=0 tx=0).stop 2024-03-30T00:34:24.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.769+0000 7faa78a00700 1 -- 172.21.15.99:0/2135500934 shutdown_connections 2024-03-30T00:34:24.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.769+0000 7faa78a00700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa74102920 0x7faa740736e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:24.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.769+0000 7faa78a00700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa74100e00 0x7faa74074b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:24.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.769+0000 7faa78a00700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7faa580a11b0 0x7faa580a3670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:24.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.769+0000 7faa78a00700 1 --2- 172.21.15.99:0/2135500934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa74101f60 0x7faa740731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:24.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.769+0000 7faa78a00700 1 -- 172.21.15.99:0/2135500934 >> 172.21.15.99:0/2135500934 conn(0x7faa740fc3c0 msgr2=0x7faa74105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:24.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.770+0000 7faa78a00700 1 -- 172.21.15.99:0/2135500934 shutdown_connections 2024-03-30T00:34:24.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:24.770+0000 7faa78a00700 1 -- 172.21.15.99:0/2135500934 wait complete. 2024-03-30T00:34:24.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:24 smithi142 ceph-mon[41779]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-30T00:34:25.143 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-30T00:34:25.184 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:34:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.375+0000 7f891ee00700 1 -- 172.21.15.99:0/1630187492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89181016e0 msgr2=0x7f8918103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.375+0000 7f891ee00700 1 --2- 172.21.15.99:0/1630187492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89181016e0 0x7f8918103ad0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8908009990 tx=0x7f8908009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.375+0000 7f891ee00700 1 -- 172.21.15.99:0/1630187492 shutdown_connections 2024-03-30T00:34:25.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.375+0000 7f891ee00700 1 --2- 172.21.15.99:0/1630187492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8918106940 0x7f8918108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.375+0000 7f891ee00700 1 --2- 172.21.15.99:0/1630187492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89181016e0 0x7f8918103ad0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.375+0000 7f891ee00700 1 --2- 172.21.15.99:0/1630187492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8918104010 0x7f8918106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.375+0000 7f891ee00700 1 -- 172.21.15.99:0/1630187492 >> 172.21.15.99:0/1630187492 conn(0x7f89180fb0d0 msgr2=0x7f89180fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:25.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.375+0000 7f891ee00700 1 -- 172.21.15.99:0/1630187492 shutdown_connections 2024-03-30T00:34:25.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.375+0000 7f891ee00700 1 -- 172.21.15.99:0/1630187492 wait complete. 2024-03-30T00:34:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.376+0000 7f891ee00700 1 Processor -- start 2024-03-30T00:34:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.376+0000 7f891ee00700 1 -- start start 2024-03-30T00:34:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ee00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89181016e0 0x7f8918101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ee00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8918104010 0x7f89180ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8918106940 0x7f89180ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8918100370 con 0x7f8918106940 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ee00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89181004e0 con 0x7f89181016e0 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ee00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89181a0b40 con 0x7f8918104010 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8918106940 0x7f89180ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8918106940 0x7f89180ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38400/0 (socket says 172.21.15.99:38400) 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ce00700 1 -- 172.21.15.99:0/2400825276 learned_addr learned my addr 172.21.15.99:0/2400825276 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f8917e00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89181016e0 0x7f8918101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f8917400700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8918104010 0x7f89180ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ce00700 1 -- 172.21.15.99:0/2400825276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8918104010 msgr2=0x7f89180ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ce00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8918104010 0x7f89180ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ce00700 1 -- 172.21.15.99:0/2400825276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89181016e0 msgr2=0x7f8918101000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ce00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89181016e0 0x7f8918101000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.377+0000 7f891ce00700 1 -- 172.21.15.99:0/2400825276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89080095f0 con 0x7f8918106940 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.378+0000 7f891ce00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8918106940 0x7f89180ffbd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8910007d70 tx=0x7f891000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.378+0000 7f8914c00700 1 -- 172.21.15.99:0/2400825276 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891000ad40 con 0x7f8918106940 2024-03-30T00:34:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.378+0000 7f8914c00700 1 -- 172.21.15.99:0/2400825276 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8910007e60 con 0x7f8918106940 2024-03-30T00:34:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.378+0000 7f891ee00700 1 -- 172.21.15.99:0/2400825276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89181a0dd0 con 0x7f8918106940 2024-03-30T00:34:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.378+0000 7f8914c00700 1 -- 172.21.15.99:0/2400825276 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89100115e0 con 0x7f8918106940 2024-03-30T00:34:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.378+0000 7f891ee00700 1 -- 172.21.15.99:0/2400825276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89181a1280 con 0x7f8918106940 2024-03-30T00:34:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.379+0000 7f891ee00700 1 -- 172.21.15.99:0/2400825276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89180fcc50 con 0x7f8918106940 2024-03-30T00:34:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.381+0000 7f8914c00700 1 -- 172.21.15.99:0/2400825276 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8910012940 con 0x7f8918106940 2024-03-30T00:34:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.382+0000 7f8914c00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88fc0a11b0 0x7f88fc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.382+0000 7f8914c00700 1 -- 172.21.15.99:0/2400825276 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f89100ccc30 con 0x7f8918106940 2024-03-30T00:34:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.382+0000 7f8917e00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88fc0a11b0 0x7f88fc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.383+0000 7f8917e00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88fc0a11b0 0x7f88fc0a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8908003780 tx=0x7f8908005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:25.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.386+0000 7f8914c00700 1 -- 172.21.15.99:0/2400825276 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8910096300 con 0x7f8918106940 2024-03-30T00:34:25.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:25 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2135500934' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-30T00:34:25.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.630+0000 7f891ee00700 1 -- 172.21.15.99:0/2400825276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f891810c340 con 0x7f8918106940 2024-03-30T00:34:25.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.632+0000 7f8914c00700 1 -- 172.21.15.99:0/2400825276 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22615 (secure 0 0 0) 0x7f8910099920 con 0x7f8918106940 2024-03-30T00:34:25.640 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:25.640 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:34:10.248126+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8},{"from":5,"to":7}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:34:25.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 -- 172.21.15.99:0/2400825276 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88fc0a11b0 msgr2=0x7f88fc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:25.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88fc0a11b0 0x7f88fc0a3670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8908003780 tx=0x7f8908005fb0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 -- 172.21.15.99:0/2400825276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8918106940 msgr2=0x7f89180ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:25.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8918106940 0x7f89180ffbd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8910007d70 tx=0x7f891000a340 comp rx=0 tx=0).stop 2024-03-30T00:34:25.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 -- 172.21.15.99:0/2400825276 shutdown_connections 2024-03-30T00:34:25.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8918106940 0x7f89180ffbd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88fc0a11b0 0x7f88fc0a3670 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89181016e0 0x7f8918101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 --2- 172.21.15.99:0/2400825276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8918104010 0x7f89180ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 -- 172.21.15.99:0/2400825276 >> 172.21.15.99:0/2400825276 conn(0x7f89180fb0d0 msgr2=0x7f8918102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:25.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 -- 172.21.15.99:0/2400825276 shutdown_connections 2024-03-30T00:34:25.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.634+0000 7f891ee00700 1 -- 172.21.15.99:0/2400825276 wait complete. 2024-03-30T00:34:25.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:33:14.085184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:33:15.090699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:34:25.662 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:34:25.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:25 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2135500934' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-30T00:34:25.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:25 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2135500934' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-30T00:34:25.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.848+0000 7fcfe0e00700 1 -- 172.21.15.99:0/1239106719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfdc102910 msgr2=0x7fcfdc1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:25.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.848+0000 7fcfe0e00700 1 --2- 172.21.15.99:0/1239106719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfdc102910 0x7fcfdc1091c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400b0d0 tx=0x7fcfd400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.849+0000 7fcfe0e00700 1 -- 172.21.15.99:0/1239106719 shutdown_connections 2024-03-30T00:34:25.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.849+0000 7fcfe0e00700 1 --2- 172.21.15.99:0/1239106719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfdc102910 0x7fcfdc1091c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.849+0000 7fcfe0e00700 1 --2- 172.21.15.99:0/1239106719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfdc100d80 0x7fcfdc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.849+0000 7fcfe0e00700 1 --2- 172.21.15.99:0/1239106719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfdc101f70 0x7fcfdc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.849+0000 7fcfe0e00700 1 -- 172.21.15.99:0/1239106719 >> 172.21.15.99:0/1239106719 conn(0x7fcfdc0fc2f0 msgr2=0x7fcfdc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:25.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.849+0000 7fcfe0e00700 1 -- 172.21.15.99:0/1239106719 shutdown_connections 2024-03-30T00:34:25.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.849+0000 7fcfe0e00700 1 -- 172.21.15.99:0/1239106719 wait complete. 2024-03-30T00:34:25.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.850+0000 7fcfe0e00700 1 Processor -- start 2024-03-30T00:34:25.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.850+0000 7fcfe0e00700 1 -- start start 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.850+0000 7fcfe0e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfdc100d80 0x7fcfdc196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.850+0000 7fcfe0e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfdc101f70 0x7fcfdc1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.850+0000 7fcfe0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfdc102910 0x7fcfdc19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.850+0000 7fcfe0e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc19e830 con 0x7fcfdc101f70 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.850+0000 7fcfe0e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc19e9a0 con 0x7fcfdc102910 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.850+0000 7fcfe0e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc19eb60 con 0x7fcfdc100d80 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.850+0000 7fcfd9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfdc101f70 0x7fcfdc1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.850+0000 7fcfd9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfdc101f70 0x7fcfdc1972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38424/0 (socket says 172.21.15.99:38424) 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.850+0000 7fcfd9400700 1 -- 172.21.15.99:0/713752609 learned_addr learned my addr 172.21.15.99:0/713752609 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfda800700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfdc102910 0x7fcfdc19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfd9e00700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfdc100d80 0x7fcfdc196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfd9400700 1 -- 172.21.15.99:0/713752609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfdc100d80 msgr2=0x7fcfdc196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:25.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfd9400700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfdc100d80 0x7fcfdc196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfd9400700 1 -- 172.21.15.99:0/713752609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfdc102910 msgr2=0x7fcfdc19e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfd9400700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfdc102910 0x7fcfdc19e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfd9400700 1 -- 172.21.15.99:0/713752609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd4009d00 con 0x7fcfdc101f70 2024-03-30T00:34:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfda800700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfdc102910 0x7fcfdc19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfd9400700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfdc101f70 0x7fcfdc1972d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800e9a0 tx=0x7fcfc800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfd2a00700 1 -- 172.21.15.99:0/713752609 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8009790 con 0x7fcfdc101f70 2024-03-30T00:34:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfd2a00700 1 -- 172.21.15.99:0/713752609 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfc800cbd0 con 0x7fcfdc101f70 2024-03-30T00:34:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfe0e00700 1 -- 172.21.15.99:0/713752609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfdc19ee40 con 0x7fcfdc101f70 2024-03-30T00:34:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfd2a00700 1 -- 172.21.15.99:0/713752609 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8018680 con 0x7fcfdc101f70 2024-03-30T00:34:25.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.851+0000 7fcfe0e00700 1 -- 172.21.15.99:0/713752609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc19f2c0 con 0x7fcfdc101f70 2024-03-30T00:34:25.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.852+0000 7fcfe0e00700 1 -- 172.21.15.99:0/713752609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfdc108250 con 0x7fcfdc101f70 2024-03-30T00:34:25.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.854+0000 7fcfd2a00700 1 -- 172.21.15.99:0/713752609 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fcfc800cd40 con 0x7fcfdc101f70 2024-03-30T00:34:25.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.855+0000 7fcfd2a00700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcfc00a1230 0x7fcfc00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:25.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.856+0000 7fcfd2a00700 1 -- 172.21.15.99:0/713752609 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fcfc8014070 con 0x7fcfdc101f70 2024-03-30T00:34:25.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.856+0000 7fcfd9e00700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcfc00a1230 0x7fcfc00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:25.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.856+0000 7fcfd9e00700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcfc00a1230 0x7fcfc00a36f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc1981c0 tx=0x7fcfcc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:25.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:25.858+0000 7fcfd2a00700 1 -- 172.21.15.99:0/713752609 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcfc8096520 con 0x7fcfdc101f70 2024-03-30T00:34:26.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.063+0000 7fcfe0e00700 1 -- 172.21.15.99:0/713752609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcfdc10c560 con 0x7fcfdc101f70 2024-03-30T00:34:26.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.064+0000 7fcfd2a00700 1 -- 172.21.15.99:0/713752609 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcfc80260a0 con 0x7fcfdc101f70 2024-03-30T00:34:26.072 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:34:26.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.066+0000 7fcfe0e00700 1 -- 172.21.15.99:0/713752609 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcfc00a1230 msgr2=0x7fcfc00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.067+0000 7fcfe0e00700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcfc00a1230 0x7fcfc00a36f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc1981c0 tx=0x7fcfcc009260 comp rx=0 tx=0).stop 2024-03-30T00:34:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.067+0000 7fcfe0e00700 1 -- 172.21.15.99:0/713752609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfdc101f70 msgr2=0x7fcfdc1972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.067+0000 7fcfe0e00700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfdc101f70 0x7fcfdc1972d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800e9a0 tx=0x7fcfc800ed60 comp rx=0 tx=0).stop 2024-03-30T00:34:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.067+0000 7fcfe0e00700 1 -- 172.21.15.99:0/713752609 shutdown_connections 2024-03-30T00:34:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.067+0000 7fcfe0e00700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcfc00a1230 0x7fcfc00a36f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.067+0000 7fcfe0e00700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfdc102910 0x7fcfdc19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.067+0000 7fcfe0e00700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfdc100d80 0x7fcfdc196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.067+0000 7fcfe0e00700 1 --2- 172.21.15.99:0/713752609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfdc101f70 0x7fcfdc1972d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.067+0000 7fcfe0e00700 1 -- 172.21.15.99:0/713752609 >> 172.21.15.99:0/713752609 conn(0x7fcfdc0fc2f0 msgr2=0x7fcfdc105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.067+0000 7fcfe0e00700 1 -- 172.21.15.99:0/713752609 shutdown_connections 2024-03-30T00:34:26.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.067+0000 7fcfe0e00700 1 -- 172.21.15.99:0/713752609 wait complete. 2024-03-30T00:34:26.093 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:34:26.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.281+0000 7f92cb800700 1 -- 172.21.15.99:0/487000953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c4100d80 msgr2=0x7f92c41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.281+0000 7f92cb800700 1 --2- 172.21.15.99:0/487000953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c4100d80 0x7f92c41011a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f92b80099c0 tx=0x7f92b8009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.281+0000 7f92cb800700 1 -- 172.21.15.99:0/487000953 shutdown_connections 2024-03-30T00:34:26.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.281+0000 7f92cb800700 1 --2- 172.21.15.99:0/487000953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92c4102910 0x7f92c41091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.281+0000 7f92cb800700 1 --2- 172.21.15.99:0/487000953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c4100d80 0x7f92c41011a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.281+0000 7f92cb800700 1 --2- 172.21.15.99:0/487000953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c4101f70 0x7f92c41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.281+0000 7f92cb800700 1 -- 172.21.15.99:0/487000953 >> 172.21.15.99:0/487000953 conn(0x7f92c40fc2f0 msgr2=0x7f92c40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:26.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.281+0000 7f92cb800700 1 -- 172.21.15.99:0/487000953 shutdown_connections 2024-03-30T00:34:26.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.281+0000 7f92cb800700 1 -- 172.21.15.99:0/487000953 wait complete. 2024-03-30T00:34:26.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.283+0000 7f92cb800700 1 Processor -- start 2024-03-30T00:34:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.283+0000 7f92cb800700 1 -- start start 2024-03-30T00:34:26.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.283+0000 7f92cb800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c4100d80 0x7f92c4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.283+0000 7f92cb800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c4101f70 0x7f92c4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.283+0000 7f92cb800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92c4102910 0x7f92c41a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92cb800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41a0a00 con 0x7f92c4101f70 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92cb800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41a0b40 con 0x7f92c4102910 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92cb800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c41a0cd0 con 0x7f92c4100d80 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c4101f70 0x7f92c4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c4101f70 0x7f92c4199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38448/0 (socket says 172.21.15.99:38448) 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c3e00700 1 -- 172.21.15.99:0/3102684410 learned_addr learned my addr 172.21.15.99:0/3102684410 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c8e00700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c4100d80 0x7f92c4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c9800700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92c4102910 0x7f92c41a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c3e00700 1 -- 172.21.15.99:0/3102684410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c4100d80 msgr2=0x7f92c4198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c3e00700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c4100d80 0x7f92c4198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c3e00700 1 -- 172.21.15.99:0/3102684410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92c4102910 msgr2=0x7f92c41a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c3e00700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92c4102910 0x7f92c41a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c3e00700 1 -- 172.21.15.99:0/3102684410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b80095f0 con 0x7f92c4101f70 2024-03-30T00:34:26.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c8e00700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c4100d80 0x7f92c4198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:26.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.284+0000 7f92c3e00700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c4101f70 0x7f92c4199410 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f92b000e9a0 tx=0x7f92b000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:26.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.285+0000 7f92c1600700 1 -- 172.21.15.99:0/3102684410 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0009790 con 0x7f92c4101f70 2024-03-30T00:34:26.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.285+0000 7f92c1600700 1 -- 172.21.15.99:0/3102684410 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92b000cbd0 con 0x7f92c4101f70 2024-03-30T00:34:26.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.285+0000 7f92cb800700 1 -- 172.21.15.99:0/3102684410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c41a0fb0 con 0x7f92c4101f70 2024-03-30T00:34:26.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.285+0000 7f92c1600700 1 -- 172.21.15.99:0/3102684410 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0018680 con 0x7f92c4101f70 2024-03-30T00:34:26.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.285+0000 7f92cb800700 1 -- 172.21.15.99:0/3102684410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c41a1430 con 0x7f92c4101f70 2024-03-30T00:34:26.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.286+0000 7f92cb800700 1 -- 172.21.15.99:0/3102684410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c4067af0 con 0x7f92c4101f70 2024-03-30T00:34:26.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.287+0000 7f92c1600700 1 -- 172.21.15.99:0/3102684410 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f92b000cd40 con 0x7f92c4101f70 2024-03-30T00:34:26.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.290+0000 7f92c1600700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f92ac0a11b0 0x7f92ac0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:26.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.290+0000 7f92c1600700 1 -- 172.21.15.99:0/3102684410 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f92b0014070 con 0x7f92c4101f70 2024-03-30T00:34:26.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.290+0000 7f92c8e00700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f92ac0a11b0 0x7f92ac0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:26.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.290+0000 7f92c8e00700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f92ac0a11b0 0x7f92ac0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009990 tx=0x7f92b801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:26.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.294+0000 7f92c1600700 1 -- 172.21.15.99:0/3102684410 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f92b00529f0 con 0x7f92c4101f70 2024-03-30T00:34:26.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:26 smithi099 ceph-mon[38079]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:34:26.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:26 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2400825276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:26.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:26 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/713752609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:26.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.505+0000 7f92cb800700 1 -- 172.21.15.99:0/3102684410 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92c41a16d0 con 0x7f92c4101f70 2024-03-30T00:34:26.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.505+0000 7f92c1600700 1 -- 172.21.15.99:0/3102684410 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f92b0099e20 con 0x7f92c4101f70 2024-03-30T00:34:26.513 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 -- 172.21.15.99:0/3102684410 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f92ac0a11b0 msgr2=0x7f92ac0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f92ac0a11b0 0x7f92ac0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009990 tx=0x7f92b801a040 comp rx=0 tx=0).stop 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 -- 172.21.15.99:0/3102684410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c4101f70 msgr2=0x7f92c4199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c4101f70 0x7f92c4199410 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f92b000e9a0 tx=0x7f92b000ed60 comp rx=0 tx=0).stop 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 -- 172.21.15.99:0/3102684410 shutdown_connections 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f92ac0a11b0 0x7f92ac0a3670 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92c4102910 0x7f92c41a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92c4100d80 0x7f92c4198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 --2- 172.21.15.99:0/3102684410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92c4101f70 0x7f92c4199410 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 -- 172.21.15.99:0/3102684410 >> 172.21.15.99:0/3102684410 conn(0x7f92c40fc2f0 msgr2=0x7f92c4107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 -- 172.21.15.99:0/3102684410 shutdown_connections 2024-03-30T00:34:26.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.508+0000 7f92cb800700 1 -- 172.21.15.99:0/3102684410 wait complete. 2024-03-30T00:34:26.535 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:34:26.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:26 smithi161 ceph-mon[42221]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:34:26.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:26 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2400825276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:26.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:26 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/713752609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:26.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.723+0000 7f999fe00700 1 -- 172.21.15.99:0/3531345808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a0100bc0 msgr2=0x7f99a0100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.723+0000 7f999fe00700 1 --2- 172.21.15.99:0/3531345808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a0100bc0 0x7f99a0100fe0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f99900099c0 tx=0x7f9990009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.723+0000 7f999fe00700 1 -- 172.21.15.99:0/3531345808 shutdown_connections 2024-03-30T00:34:26.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.723+0000 7f999fe00700 1 --2- 172.21.15.99:0/3531345808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0102750 0x7f99a0109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.723+0000 7f999fe00700 1 --2- 172.21.15.99:0/3531345808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a0100bc0 0x7f99a0100fe0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.723+0000 7f999fe00700 1 --2- 172.21.15.99:0/3531345808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99a0101db0 0x7f99a0102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.723+0000 7f999fe00700 1 -- 172.21.15.99:0/3531345808 >> 172.21.15.99:0/3531345808 conn(0x7f99a00fc110 msgr2=0x7f99a00fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:26.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.723+0000 7f999fe00700 1 -- 172.21.15.99:0/3531345808 shutdown_connections 2024-03-30T00:34:26.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.723+0000 7f999fe00700 1 -- 172.21.15.99:0/3531345808 wait complete. 2024-03-30T00:34:26.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.724+0000 7f999fe00700 1 Processor -- start 2024-03-30T00:34:26.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.724+0000 7f999fe00700 1 -- start start 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99a0100bc0 0x7f99a0196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0101db0 0x7f99a0197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a0102750 0x7f99a019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999fe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a019e620 con 0x7f99a0102750 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999fe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a019e760 con 0x7f99a0101db0 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999fe00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a019e8f0 con 0x7f99a0100bc0 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a0102750 0x7f99a019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a0102750 0x7f99a019e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38456/0 (socket says 172.21.15.99:38456) 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999f400700 1 -- 172.21.15.99:0/3177989591 learned_addr learned my addr 172.21.15.99:0/3177989591 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999ea00700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99a0100bc0 0x7f99a0196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999e000700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0101db0 0x7f99a0197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999f400700 1 -- 172.21.15.99:0/3177989591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99a0100bc0 msgr2=0x7f99a0196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999f400700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99a0100bc0 0x7f99a0196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999f400700 1 -- 172.21.15.99:0/3177989591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0101db0 msgr2=0x7f99a0197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999f400700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0101db0 0x7f99a0197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999f400700 1 -- 172.21.15.99:0/3177989591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99900095f0 con 0x7f99a0102750 2024-03-30T00:34:26.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999ea00700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99a0100bc0 0x7f99a0196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:26.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.725+0000 7f999e000700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0101db0 0x7f99a0197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:26.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.726+0000 7f999f400700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a0102750 0x7f99a019e050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f999800ada0 tx=0x7f999800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:26.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.726+0000 7f9997400700 1 -- 172.21.15.99:0/3177989591 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999800f070 con 0x7f99a0102750 2024-03-30T00:34:26.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.726+0000 7f9997400700 1 -- 172.21.15.99:0/3177989591 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9998007b30 con 0x7f99a0102750 2024-03-30T00:34:26.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.726+0000 7f9997400700 1 -- 172.21.15.99:0/3177989591 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99980053e0 con 0x7f99a0102750 2024-03-30T00:34:26.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.726+0000 7f999fe00700 1 -- 172.21.15.99:0/3177989591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a019ebd0 con 0x7f99a0102750 2024-03-30T00:34:26.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.726+0000 7f999fe00700 1 -- 172.21.15.99:0/3177989591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a019f0e0 con 0x7f99a0102750 2024-03-30T00:34:26.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.727+0000 7f999fe00700 1 -- 172.21.15.99:0/3177989591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a0108130 con 0x7f99a0102750 2024-03-30T00:34:26.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.729+0000 7f9997400700 1 -- 172.21.15.99:0/3177989591 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f999800c980 con 0x7f99a0102750 2024-03-30T00:34:26.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.730+0000 7f9997400700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f99840a1230 0x7f99840a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:26.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.730+0000 7f9997400700 1 -- 172.21.15.99:0/3177989591 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f99980cc140 con 0x7f99a0102750 2024-03-30T00:34:26.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.730+0000 7f999ea00700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f99840a1230 0x7f99840a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:26.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.731+0000 7f999ea00700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f99840a1230 0x7f99840a36f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9990000c00 tx=0x7f9990005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:26.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.734+0000 7f9997400700 1 -- 172.21.15.99:0/3177989591 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9998095790 con 0x7f99a0102750 2024-03-30T00:34:26.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:26 smithi142 ceph-mon[41779]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:34:26.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:26 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2400825276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:26.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:26 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/713752609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:26.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.939+0000 7f999fe00700 1 -- 172.21.15.99:0/3177989591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f99a010c2d0 con 0x7f99a0102750 2024-03-30T00:34:26.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.939+0000 7f9997400700 1 -- 172.21.15.99:0/3177989591 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f999800cc30 con 0x7f99a0102750 2024-03-30T00:34:26.947 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-30T00:34:26.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 -- 172.21.15.99:0/3177989591 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f99840a1230 msgr2=0x7f99840a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f99840a1230 0x7f99840a36f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9990000c00 tx=0x7f9990005fb0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 -- 172.21.15.99:0/3177989591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a0102750 msgr2=0x7f99a019e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a0102750 0x7f99a019e050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f999800ada0 tx=0x7f999800c330 comp rx=0 tx=0).stop 2024-03-30T00:34:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 -- 172.21.15.99:0/3177989591 shutdown_connections 2024-03-30T00:34:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99a0102750 0x7f99a019e050 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99a0100bc0 0x7f99a0196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f99840a1230 0x7f99840a36f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 --2- 172.21.15.99:0/3177989591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0101db0 0x7f99a0197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 -- 172.21.15.99:0/3177989591 >> 172.21.15.99:0/3177989591 conn(0x7f99a00fc110 msgr2=0x7f99a0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 -- 172.21.15.99:0/3177989591 shutdown_connections 2024-03-30T00:34:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:26.942+0000 7f999fe00700 1 -- 172.21.15.99:0/3177989591 wait complete. 2024-03-30T00:34:26.968 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:34:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.153+0000 7ff220a00700 1 -- 172.21.15.99:0/15744130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff21c102810 msgr2=0x7ff21c1090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.153+0000 7ff220a00700 1 --2- 172.21.15.99:0/15744130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff21c102810 0x7ff21c1090c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff21400b0d0 tx=0x7ff21400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.153+0000 7ff220a00700 1 -- 172.21.15.99:0/15744130 shutdown_connections 2024-03-30T00:34:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.153+0000 7ff220a00700 1 --2- 172.21.15.99:0/15744130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff21c102810 0x7ff21c1090c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.153+0000 7ff220a00700 1 --2- 172.21.15.99:0/15744130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff21c100c80 0x7ff21c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.153+0000 7ff220a00700 1 --2- 172.21.15.99:0/15744130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff21c101e70 0x7ff21c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.153+0000 7ff220a00700 1 -- 172.21.15.99:0/15744130 >> 172.21.15.99:0/15744130 conn(0x7ff21c0fc1f0 msgr2=0x7ff21c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.153+0000 7ff220a00700 1 -- 172.21.15.99:0/15744130 shutdown_connections 2024-03-30T00:34:27.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.153+0000 7ff220a00700 1 -- 172.21.15.99:0/15744130 wait complete. 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.154+0000 7ff220a00700 1 Processor -- start 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.154+0000 7ff220a00700 1 -- start start 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.154+0000 7ff220a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff21c100c80 0x7ff21c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.154+0000 7ff220a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff21c101e70 0x7ff21c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.154+0000 7ff220a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff21c102810 0x7ff21c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.154+0000 7ff220a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c073e80 con 0x7ff21c100c80 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.154+0000 7ff220a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c073ff0 con 0x7ff21c101e70 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.154+0000 7ff220a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c1a0c90 con 0x7ff21c102810 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff219e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff21c100c80 0x7ff21c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff219e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff21c100c80 0x7ff21c074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38470/0 (socket says 172.21.15.99:38470) 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff219e00700 1 -- 172.21.15.99:0/3528968215 learned_addr learned my addr 172.21.15.99:0/3528968215 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:27.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff219400700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff21c101e70 0x7ff21c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff21a800700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff21c102810 0x7ff21c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff219e00700 1 -- 172.21.15.99:0/3528968215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff21c102810 msgr2=0x7ff21c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff219e00700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff21c102810 0x7ff21c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff219e00700 1 -- 172.21.15.99:0/3528968215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff21c101e70 msgr2=0x7ff21c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff219e00700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff21c101e70 0x7ff21c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff219e00700 1 -- 172.21.15.99:0/3528968215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff214009d00 con 0x7ff21c100c80 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff219400700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff21c101e70 0x7ff21c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff219e00700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff21c100c80 0x7ff21c074b10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00d980 tx=0x7ff20c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.155+0000 7ff212a00700 1 -- 172.21.15.99:0/3528968215 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c004020 con 0x7ff21c100c80 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.156+0000 7ff212a00700 1 -- 172.21.15.99:0/3528968215 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff20c0096d0 con 0x7ff21c100c80 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.156+0000 7ff220a00700 1 -- 172.21.15.99:0/3528968215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21c1a0f20 con 0x7ff21c100c80 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.156+0000 7ff212a00700 1 -- 172.21.15.99:0/3528968215 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c003a00 con 0x7ff21c100c80 2024-03-30T00:34:27.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.156+0000 7ff220a00700 1 -- 172.21.15.99:0/3528968215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21c1a1340 con 0x7ff21c100c80 2024-03-30T00:34:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.157+0000 7ff220a00700 1 -- 172.21.15.99:0/3528968215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21c1080b0 con 0x7ff21c100c80 2024-03-30T00:34:27.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.159+0000 7ff212a00700 1 -- 172.21.15.99:0/3528968215 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff20c009840 con 0x7ff21c100c80 2024-03-30T00:34:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.160+0000 7ff212a00700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2000a11b0 0x7ff2000a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.160+0000 7ff212a00700 1 -- 172.21.15.99:0/3528968215 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff20c0cc730 con 0x7ff21c100c80 2024-03-30T00:34:27.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.160+0000 7ff219400700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2000a11b0 0x7ff2000a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:27.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.160+0000 7ff219400700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2000a11b0 0x7ff2000a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff208005950 tx=0x7ff2080058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:27.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.164+0000 7ff212a00700 1 -- 172.21.15.99:0/3528968215 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff20c095cf0 con 0x7ff21c100c80 2024-03-30T00:34:27.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.424+0000 7ff220a00700 1 -- 172.21.15.99:0/3528968215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff21c067a10 con 0x7ff21c100c80 2024-03-30T00:34:27.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.425+0000 7ff212a00700 1 -- 172.21.15.99:0/3528968215 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10860 (secure 0 0 0) 0x7ff20c099310 con 0x7ff21c100c80 2024-03-30T00:34:27.433 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:27.433 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:33:21.119410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:34:27.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 -- 172.21.15.99:0/3528968215 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2000a11b0 msgr2=0x7ff2000a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:27.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2000a11b0 0x7ff2000a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff208005950 tx=0x7ff2080058e0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 -- 172.21.15.99:0/3528968215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff21c100c80 msgr2=0x7ff21c074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:27.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff21c100c80 0x7ff21c074b10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00d980 tx=0x7ff20c00dc90 comp rx=0 tx=0).stop 2024-03-30T00:34:27.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 -- 172.21.15.99:0/3528968215 shutdown_connections 2024-03-30T00:34:27.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff21c102810 0x7ff21c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff21c100c80 0x7ff21c074b10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2000a11b0 0x7ff2000a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 --2- 172.21.15.99:0/3528968215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff21c101e70 0x7ff21c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 -- 172.21.15.99:0/3528968215 >> 172.21.15.99:0/3528968215 conn(0x7ff21c0fc1f0 msgr2=0x7ff21c1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:27.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 -- 172.21.15.99:0/3528968215 shutdown_connections 2024-03-30T00:34:27.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.427+0000 7ff220a00700 1 -- 172.21.15.99:0/3528968215 wait complete. 2024-03-30T00:34:27.436 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2024-03-30T00:34:27.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:34:27.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.637+0000 7fc0d5400700 1 -- 172.21.15.99:0/2901713688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0d0100c80 msgr2=0x7fc0d01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:27.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.637+0000 7fc0d5400700 1 --2- 172.21.15.99:0/2901713688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0d0100c80 0x7fc0d01010a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00099c0 tx=0x7fc0c0009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.637+0000 7fc0d5400700 1 -- 172.21.15.99:0/2901713688 shutdown_connections 2024-03-30T00:34:27.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.637+0000 7fc0d5400700 1 --2- 172.21.15.99:0/2901713688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d0102810 0x7fc0d01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.637+0000 7fc0d5400700 1 --2- 172.21.15.99:0/2901713688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0d0100c80 0x7fc0d01010a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.637+0000 7fc0d5400700 1 --2- 172.21.15.99:0/2901713688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0d0101e70 0x7fc0d01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.637+0000 7fc0d5400700 1 -- 172.21.15.99:0/2901713688 >> 172.21.15.99:0/2901713688 conn(0x7fc0d00fc1f0 msgr2=0x7fc0d00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:27.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.638+0000 7fc0d5400700 1 -- 172.21.15.99:0/2901713688 shutdown_connections 2024-03-30T00:34:27.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.638+0000 7fc0d5400700 1 -- 172.21.15.99:0/2901713688 wait complete. 2024-03-30T00:34:27.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0d5400700 1 Processor -- start 2024-03-30T00:34:27.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0d5400700 1 -- start start 2024-03-30T00:34:27.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0d5400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0d0100c80 0x7fc0d0196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:27.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0d5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d0101e70 0x7fc0d0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:27.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0d5400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0d0102810 0x7fc0d019e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:27.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0d5400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d019e700 con 0x7fc0d0100c80 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0d5400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d019e840 con 0x7fc0d0101e70 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0d5400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d019e9d0 con 0x7fc0d0102810 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0ce800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0d0100c80 0x7fc0d0196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0ce800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0d0100c80 0x7fc0d0196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38490/0 (socket says 172.21.15.99:38490) 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0ce800700 1 -- 172.21.15.99:0/3849496146 learned_addr learned my addr 172.21.15.99:0/3849496146 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.639+0000 7fc0cde00700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d0101e70 0x7fc0d0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0cf200700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0d0102810 0x7fc0d019e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0ce800700 1 -- 172.21.15.99:0/3849496146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0d0102810 msgr2=0x7fc0d019e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0ce800700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0d0102810 0x7fc0d019e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0ce800700 1 -- 172.21.15.99:0/3849496146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d0101e70 msgr2=0x7fc0d0197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0ce800700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d0101e70 0x7fc0d0197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0ce800700 1 -- 172.21.15.99:0/3849496146 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c00095f0 con 0x7fc0d0100c80 2024-03-30T00:34:27.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0ce800700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0d0100c80 0x7fc0d0196bd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009f50 tx=0x7fc0c0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:27.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0c7400700 1 -- 172.21.15.99:0/3849496146 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c001d070 con 0x7fc0d0100c80 2024-03-30T00:34:27.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0d5400700 1 -- 172.21.15.99:0/3849496146 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d019ec50 con 0x7fc0d0100c80 2024-03-30T00:34:27.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0c7400700 1 -- 172.21.15.99:0/3849496146 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc0c000bda0 con 0x7fc0d0100c80 2024-03-30T00:34:27.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0c7400700 1 -- 172.21.15.99:0/3849496146 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0022dd0 con 0x7fc0d0100c80 2024-03-30T00:34:27.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.640+0000 7fc0d5400700 1 -- 172.21.15.99:0/3849496146 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d019f080 con 0x7fc0d0100c80 2024-03-30T00:34:27.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.641+0000 7fc0d5400700 1 -- 172.21.15.99:0/3849496146 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d01081c0 con 0x7fc0d0100c80 2024-03-30T00:34:27.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.643+0000 7fc0c7400700 1 -- 172.21.15.99:0/3849496146 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc0c0022570 con 0x7fc0d0100c80 2024-03-30T00:34:27.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.645+0000 7fc0c7400700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc0b40a11b0 0x7fc0b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:27.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.645+0000 7fc0c7400700 1 -- 172.21.15.99:0/3849496146 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fc0c00cdee0 con 0x7fc0d0100c80 2024-03-30T00:34:27.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.645+0000 7fc0cde00700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc0b40a11b0 0x7fc0b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:27.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.646+0000 7fc0cde00700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc0b40a11b0 0x7fc0b40a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc005950 tx=0x7fc0bc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:27.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.649+0000 7fc0c7400700 1 -- 172.21.15.99:0/3849496146 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc0c0097530 con 0x7fc0d0100c80 2024-03-30T00:34:27.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:27 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3102684410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:27.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:27 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3177989591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:27.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:27 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3102684410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:27.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:27 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3177989591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:27.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:27 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3102684410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:27.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:27 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3177989591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:27.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.896+0000 7fc0d5400700 1 -- 172.21.15.99:0/3849496146 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0d0067a10 con 0x7fc0d0100c80 2024-03-30T00:34:27.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.897+0000 7fc0c7400700 1 -- 172.21.15.99:0/3849496146 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22615 (secure 0 0 0) 0x7fc0c009ab50 con 0x7fc0d0100c80 2024-03-30T00:34:27.905 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:27.905 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:34:10.248126+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8},{"from":5,"to":7}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 -- 172.21.15.99:0/3849496146 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc0b40a11b0 msgr2=0x7fc0b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc0b40a11b0 0x7fc0b40a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc005950 tx=0x7fc0bc00b410 comp rx=0 tx=0).stop 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 -- 172.21.15.99:0/3849496146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0d0100c80 msgr2=0x7fc0d0196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0d0100c80 0x7fc0d0196bd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009f50 tx=0x7fc0c0005b00 comp rx=0 tx=0).stop 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 -- 172.21.15.99:0/3849496146 shutdown_connections 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0d0102810 0x7fc0d019e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0d0100c80 0x7fc0d0196bd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc0b40a11b0 0x7fc0b40a3670 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 --2- 172.21.15.99:0/3849496146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d0101e70 0x7fc0d0197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 -- 172.21.15.99:0/3849496146 >> 172.21.15.99:0/3849496146 conn(0x7fc0d00fc1f0 msgr2=0x7fc0d0105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 -- 172.21.15.99:0/3849496146 shutdown_connections 2024-03-30T00:34:27.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:27.900+0000 7fc0d5400700 1 -- 172.21.15.99:0/3849496146 wait complete. 2024-03-30T00:34:27.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:33:14.085184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:33:15.090699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:34:27.927 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:34:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.114+0000 7f9000a00700 1 -- 172.21.15.99:0/420829751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ffc074df0 msgr2=0x7f8ffc073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.114+0000 7f9000a00700 1 --2- 172.21.15.99:0/420829751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ffc074df0 0x7f8ffc073240 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009990 tx=0x7f8fec009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.114+0000 7f9000a00700 1 -- 172.21.15.99:0/420829751 shutdown_connections 2024-03-30T00:34:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.114+0000 7f9000a00700 1 --2- 172.21.15.99:0/420829751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc108c40 0x7f8ffc10b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.114+0000 7f9000a00700 1 --2- 172.21.15.99:0/420829751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ffc074df0 0x7f8ffc073240 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.114+0000 7f9000a00700 1 --2- 172.21.15.99:0/420829751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ffc073810 0x7f8ffc073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.114+0000 7f9000a00700 1 -- 172.21.15.99:0/420829751 >> 172.21.15.99:0/420829751 conn(0x7f8ffc0fc2f0 msgr2=0x7f8ffc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.114+0000 7f9000a00700 1 -- 172.21.15.99:0/420829751 shutdown_connections 2024-03-30T00:34:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.114+0000 7f9000a00700 1 -- 172.21.15.99:0/420829751 wait complete. 2024-03-30T00:34:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.115+0000 7f9000a00700 1 Processor -- start 2024-03-30T00:34:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.115+0000 7f9000a00700 1 -- start start 2024-03-30T00:34:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f9000a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ffc073810 0x7f8ffc198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f9000a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ffc074df0 0x7f8ffc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f9000a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc108c40 0x7f8ffc1a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f9000a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1a0940 con 0x7f8ffc074df0 2024-03-30T00:34:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f9000a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1a0a80 con 0x7f8ffc073810 2024-03-30T00:34:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f9000a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1a0c10 con 0x7f8ffc108c40 2024-03-30T00:34:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ff9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ffc074df0 0x7f8ffc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ff9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ffc074df0 0x7f8ffc199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38514/0 (socket says 172.21.15.99:38514) 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ff9400700 1 -- 172.21.15.99:0/2324436225 learned_addr learned my addr 172.21.15.99:0/2324436225 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ffa800700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc108c40 0x7f8ffc1a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ff9400700 1 -- 172.21.15.99:0/2324436225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc108c40 msgr2=0x7f8ffc1a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ff9e00700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ffc073810 0x7f8ffc198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ff9400700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc108c40 0x7f8ffc1a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ff9400700 1 -- 172.21.15.99:0/2324436225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ffc073810 msgr2=0x7f8ffc198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ff9400700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ffc073810 0x7f8ffc198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ff9400700 1 -- 172.21.15.99:0/2324436225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec0095f0 con 0x7f8ffc074df0 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ffa800700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc108c40 0x7f8ffc1a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.116+0000 7f8ff9400700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ffc074df0 0x7f8ffc199350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c970 tx=0x7f8fe800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.117+0000 7f8ff2a00700 1 -- 172.21.15.99:0/2324436225 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8004220 con 0x7f8ffc074df0 2024-03-30T00:34:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.117+0000 7f8ff2a00700 1 -- 172.21.15.99:0/2324436225 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fe8007d30 con 0x7f8ffc074df0 2024-03-30T00:34:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.117+0000 7f9000a00700 1 -- 172.21.15.99:0/2324436225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc1a0ef0 con 0x7f8ffc074df0 2024-03-30T00:34:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.117+0000 7f8ff2a00700 1 -- 172.21.15.99:0/2324436225 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8003bf0 con 0x7f8ffc074df0 2024-03-30T00:34:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.117+0000 7f9000a00700 1 -- 172.21.15.99:0/2324436225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc1a13c0 con 0x7f8ffc074df0 2024-03-30T00:34:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.118+0000 7f9000a00700 1 -- 172.21.15.99:0/2324436225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ffc107a00 con 0x7f8ffc074df0 2024-03-30T00:34:28.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.121+0000 7f8ff2a00700 1 -- 172.21.15.99:0/2324436225 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8fe8003d70 con 0x7f8ffc074df0 2024-03-30T00:34:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.122+0000 7f8ff2a00700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fe00a1230 0x7f8fe00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.122+0000 7f8ff2a00700 1 -- 172.21.15.99:0/2324436225 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f8fe80cca70 con 0x7f8ffc074df0 2024-03-30T00:34:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.123+0000 7f8ff9e00700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fe00a1230 0x7f8fe00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.123+0000 7f8ff9e00700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fe00a1230 0x7f8fe00a36f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8fec003780 tx=0x7f8fec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:28.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.125+0000 7f8ff2a00700 1 -- 172.21.15.99:0/2324436225 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8fe80960b0 con 0x7f8ffc074df0 2024-03-30T00:34:28.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.330+0000 7f9000a00700 1 -- 172.21.15.99:0/2324436225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ffc10e670 con 0x7f8ffc074df0 2024-03-30T00:34:28.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.330+0000 7f8ff2a00700 1 -- 172.21.15.99:0/2324436225 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8fe801d020 con 0x7f8ffc074df0 2024-03-30T00:34:28.338 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:34:28.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 -- 172.21.15.99:0/2324436225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fe00a1230 msgr2=0x7f8fe00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fe00a1230 0x7f8fe00a36f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8fec003780 tx=0x7f8fec000bc0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 -- 172.21.15.99:0/2324436225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ffc074df0 msgr2=0x7f8ffc199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ffc074df0 0x7f8ffc199350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c970 tx=0x7f8fe800cd30 comp rx=0 tx=0).stop 2024-03-30T00:34:28.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 -- 172.21.15.99:0/2324436225 shutdown_connections 2024-03-30T00:34:28.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ffc108c40 0x7f8ffc1a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fe00a1230 0x7f8fe00a36f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ffc073810 0x7f8ffc198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 --2- 172.21.15.99:0/2324436225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ffc074df0 0x7f8ffc199350 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 -- 172.21.15.99:0/2324436225 >> 172.21.15.99:0/2324436225 conn(0x7f8ffc0fc2f0 msgr2=0x7f8ffc104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:28.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 -- 172.21.15.99:0/2324436225 shutdown_connections 2024-03-30T00:34:28.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.333+0000 7f9000a00700 1 -- 172.21.15.99:0/2324436225 wait complete. 2024-03-30T00:34:28.359 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:34:28.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.546+0000 7fed55600700 1 -- 172.21.15.99:0/1205130681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed50101f70 msgr2=0x7fed501023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.546+0000 7fed55600700 1 --2- 172.21.15.99:0/1205130681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed50101f70 0x7fed501023d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0099c0 tx=0x7fed3c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.546+0000 7fed55600700 1 -- 172.21.15.99:0/1205130681 shutdown_connections 2024-03-30T00:34:28.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.546+0000 7fed55600700 1 --2- 172.21.15.99:0/1205130681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50102910 0x7fed501091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.546+0000 7fed55600700 1 --2- 172.21.15.99:0/1205130681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed50100d80 0x7fed501011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.546+0000 7fed55600700 1 --2- 172.21.15.99:0/1205130681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed50101f70 0x7fed501023d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.546+0000 7fed55600700 1 -- 172.21.15.99:0/1205130681 >> 172.21.15.99:0/1205130681 conn(0x7fed500fc2f0 msgr2=0x7fed500fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:28.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.546+0000 7fed55600700 1 -- 172.21.15.99:0/1205130681 shutdown_connections 2024-03-30T00:34:28.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.546+0000 7fed55600700 1 -- 172.21.15.99:0/1205130681 wait complete. 2024-03-30T00:34:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.547+0000 7fed55600700 1 Processor -- start 2024-03-30T00:34:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.547+0000 7fed55600700 1 -- start start 2024-03-30T00:34:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.548+0000 7fed55600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed50100d80 0x7fed50074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.548+0000 7fed55600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50101f70 0x7fed500731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.548+0000 7fed55600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed50102910 0x7fed500736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.548+0000 7fed55600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed50073e80 con 0x7fed50102910 2024-03-30T00:34:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.548+0000 7fed55600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed50073ff0 con 0x7fed50101f70 2024-03-30T00:34:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.548+0000 7fed55600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed501a0c40 con 0x7fed50100d80 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.548+0000 7fed4f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed50102910 0x7fed500736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.548+0000 7fed4f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed50102910 0x7fed500736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38542/0 (socket says 172.21.15.99:38542) 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.548+0000 7fed4f400700 1 -- 172.21.15.99:0/2035968229 learned_addr learned my addr 172.21.15.99:0/2035968229 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.548+0000 7fed4e000700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50101f70 0x7fed500731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.548+0000 7fed4ea00700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed50100d80 0x7fed50074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.549+0000 7fed4f400700 1 -- 172.21.15.99:0/2035968229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed50100d80 msgr2=0x7fed50074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.549+0000 7fed4f400700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed50100d80 0x7fed50074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.549+0000 7fed4f400700 1 -- 172.21.15.99:0/2035968229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50101f70 msgr2=0x7fed500731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.549+0000 7fed4f400700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50101f70 0x7fed500731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.549+0000 7fed4f400700 1 -- 172.21.15.99:0/2035968229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c0095f0 con 0x7fed50102910 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.549+0000 7fed4f400700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed50102910 0x7fed500736e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fed48007ee0 tx=0x7fed4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.549+0000 7fed47400700 1 -- 172.21.15.99:0/2035968229 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4800dcc0 con 0x7fed50102910 2024-03-30T00:34:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.549+0000 7fed47400700 1 -- 172.21.15.99:0/2035968229 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed4800f040 con 0x7fed50102910 2024-03-30T00:34:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.549+0000 7fed55600700 1 -- 172.21.15.99:0/2035968229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed501a0ed0 con 0x7fed50102910 2024-03-30T00:34:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.549+0000 7fed47400700 1 -- 172.21.15.99:0/2035968229 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed480046e0 con 0x7fed50102910 2024-03-30T00:34:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.549+0000 7fed55600700 1 -- 172.21.15.99:0/2035968229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed501a1350 con 0x7fed50102910 2024-03-30T00:34:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.550+0000 7fed55600700 1 -- 172.21.15.99:0/2035968229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed501081f0 con 0x7fed50102910 2024-03-30T00:34:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.552+0000 7fed47400700 1 -- 172.21.15.99:0/2035968229 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fed48003a00 con 0x7fed50102910 2024-03-30T00:34:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.553+0000 7fed47400700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed340a11b0 0x7fed340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.553+0000 7fed47400700 1 -- 172.21.15.99:0/2035968229 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fed480cc2b0 con 0x7fed50102910 2024-03-30T00:34:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.554+0000 7fed4ea00700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed340a11b0 0x7fed340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.554+0000 7fed4ea00700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed340a11b0 0x7fed340a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed50101dd0 tx=0x7fed4000a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:28.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.557+0000 7fed47400700 1 -- 172.21.15.99:0/2035968229 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fed48095870 con 0x7fed50102910 2024-03-30T00:34:28.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:28 smithi161 ceph-mon[42221]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:34:28.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:28 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3528968215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:28.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:28 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3849496146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:28.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:28 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2324436225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:28.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:28 smithi099 ceph-mon[38079]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:34:28.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:28 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3528968215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:28.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:28 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3849496146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:28.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:28 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2324436225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:28.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.764+0000 7fed55600700 1 -- 172.21.15.99:0/2035968229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed5010c490 con 0x7fed50102910 2024-03-30T00:34:28.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.765+0000 7fed47400700 1 -- 172.21.15.99:0/2035968229 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed48098e90 con 0x7fed50102910 2024-03-30T00:34:28.772 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-30T00:34:28.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 -- 172.21.15.99:0/2035968229 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed340a11b0 msgr2=0x7fed340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed340a11b0 0x7fed340a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed50101dd0 tx=0x7fed4000a3b0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 -- 172.21.15.99:0/2035968229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed50102910 msgr2=0x7fed500736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed50102910 0x7fed500736e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fed48007ee0 tx=0x7fed4800d3b0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 -- 172.21.15.99:0/2035968229 shutdown_connections 2024-03-30T00:34:28.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed50102910 0x7fed500736e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed50100d80 0x7fed50074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fed340a11b0 0x7fed340a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 --2- 172.21.15.99:0/2035968229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed50101f70 0x7fed500731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 -- 172.21.15.99:0/2035968229 >> 172.21.15.99:0/2035968229 conn(0x7fed500fc2f0 msgr2=0x7fed50105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:28.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 -- 172.21.15.99:0/2035968229 shutdown_connections 2024-03-30T00:34:28.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.768+0000 7fed55600700 1 -- 172.21.15.99:0/2035968229 wait complete. 2024-03-30T00:34:28.793 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:34:28.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:28 smithi142 ceph-mon[41779]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:34:28.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:28 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3528968215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:28.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:28 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3849496146' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:28.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:28 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2324436225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.981+0000 7fec54200700 1 -- 172.21.15.99:0/579649729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec4c100d80 msgr2=0x7fec4c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.981+0000 7fec54200700 1 --2- 172.21.15.99:0/579649729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec4c100d80 0x7fec4c1011a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fec400099c0 tx=0x7fec40009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.981+0000 7fec54200700 1 -- 172.21.15.99:0/579649729 shutdown_connections 2024-03-30T00:34:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.981+0000 7fec54200700 1 --2- 172.21.15.99:0/579649729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec4c102910 0x7fec4c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.981+0000 7fec54200700 1 --2- 172.21.15.99:0/579649729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec4c100d80 0x7fec4c1011a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.981+0000 7fec54200700 1 --2- 172.21.15.99:0/579649729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c101f70 0x7fec4c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.981+0000 7fec54200700 1 -- 172.21.15.99:0/579649729 >> 172.21.15.99:0/579649729 conn(0x7fec4c0fc2f0 msgr2=0x7fec4c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:28.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.981+0000 7fec54200700 1 -- 172.21.15.99:0/579649729 shutdown_connections 2024-03-30T00:34:28.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.981+0000 7fec54200700 1 -- 172.21.15.99:0/579649729 wait complete. 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.982+0000 7fec54200700 1 Processor -- start 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.982+0000 7fec54200700 1 -- start start 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.982+0000 7fec54200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c100d80 0x7fec4c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.982+0000 7fec54200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec4c101f70 0x7fec4c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec54200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec4c102910 0x7fec4c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec54200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c073e80 con 0x7fec4c101f70 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec54200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c073ff0 con 0x7fec4c100d80 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec54200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c1a0c70 con 0x7fec4c102910 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec50e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec4c101f70 0x7fec4c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec50e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec4c101f70 0x7fec4c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38572/0 (socket says 172.21.15.99:38572) 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec50e00700 1 -- 172.21.15.99:0/1567873627 learned_addr learned my addr 172.21.15.99:0/1567873627 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec50e00700 1 -- 172.21.15.99:0/1567873627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec4c102910 msgr2=0x7fec4c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec51800700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c100d80 0x7fec4c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec52200700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec4c102910 0x7fec4c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec50e00700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec4c102910 0x7fec4c0736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec50e00700 1 -- 172.21.15.99:0/1567873627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c100d80 msgr2=0x7fec4c074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec50e00700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c100d80 0x7fec4c074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec50e00700 1 -- 172.21.15.99:0/1567873627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400095f0 con 0x7fec4c101f70 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.983+0000 7fec50e00700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec4c101f70 0x7fec4c0731a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c9b0 tx=0x7fec3800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.984+0000 7fec4a000700 1 -- 172.21.15.99:0/1567873627 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38007aa0 con 0x7fec4c101f70 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.984+0000 7fec4a000700 1 -- 172.21.15.99:0/1567873627 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec38007c00 con 0x7fec4c101f70 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.984+0000 7fec4a000700 1 -- 172.21.15.99:0/1567873627 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38018660 con 0x7fec4c101f70 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.984+0000 7fec54200700 1 -- 172.21.15.99:0/1567873627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4c1a0f00 con 0x7fec4c101f70 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.984+0000 7fec54200700 1 -- 172.21.15.99:0/1567873627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c1a1380 con 0x7fec4c101f70 2024-03-30T00:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.985+0000 7fec54200700 1 -- 172.21.15.99:0/1567873627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4c108250 con 0x7fec4c101f70 2024-03-30T00:34:28.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.986+0000 7fec4a000700 1 -- 172.21.15.99:0/1567873627 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fec3800f450 con 0x7fec4c101f70 2024-03-30T00:34:28.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.988+0000 7fec4a000700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fec340a1230 0x7fec340a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:28.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.988+0000 7fec4a000700 1 -- 172.21.15.99:0/1567873627 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fec380cc9d0 con 0x7fec4c101f70 2024-03-30T00:34:28.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.988+0000 7fec51800700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fec340a1230 0x7fec340a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:28.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.988+0000 7fec51800700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fec340a1230 0x7fec340a36f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fec40006010 tx=0x7fec40009510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:28.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:28.992+0000 7fec4a000700 1 -- 172.21.15.99:0/1567873627 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fec38095ff0 con 0x7fec4c101f70 2024-03-30T00:34:29.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.200+0000 7fec54200700 1 -- 172.21.15.99:0/1567873627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec4c02ccb0 con 0x7fec4c101f70 2024-03-30T00:34:29.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.201+0000 7fec4a000700 1 -- 172.21.15.99:0/1567873627 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec38099610 con 0x7fec4c101f70 2024-03-30T00:34:29.208 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-30T00:34:29.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 -- 172.21.15.99:0/1567873627 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fec340a1230 msgr2=0x7fec340a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:29.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fec340a1230 0x7fec340a36f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fec40006010 tx=0x7fec40009510 comp rx=0 tx=0).stop 2024-03-30T00:34:29.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 -- 172.21.15.99:0/1567873627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec4c101f70 msgr2=0x7fec4c0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:29.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec4c101f70 0x7fec4c0731a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c9b0 tx=0x7fec3800cd70 comp rx=0 tx=0).stop 2024-03-30T00:34:29.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 -- 172.21.15.99:0/1567873627 shutdown_connections 2024-03-30T00:34:29.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec4c102910 0x7fec4c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fec340a1230 0x7fec340a36f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c100d80 0x7fec4c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 --2- 172.21.15.99:0/1567873627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec4c101f70 0x7fec4c0731a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 -- 172.21.15.99:0/1567873627 >> 172.21.15.99:0/1567873627 conn(0x7fec4c0fc2f0 msgr2=0x7fec4c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:29.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 -- 172.21.15.99:0/1567873627 shutdown_connections 2024-03-30T00:34:29.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.203+0000 7fec54200700 1 -- 172.21.15.99:0/1567873627 wait complete. 2024-03-30T00:34:29.229 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:34:29.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.415+0000 7ff294c00700 1 -- 172.21.15.99:0/2165577449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290100c80 msgr2=0x7ff2901010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:29.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.415+0000 7ff294c00700 1 --2- 172.21.15.99:0/2165577449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290100c80 0x7ff2901010a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff2800099c0 tx=0x7ff280009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.416+0000 7ff294c00700 1 -- 172.21.15.99:0/2165577449 shutdown_connections 2024-03-30T00:34:29.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.416+0000 7ff294c00700 1 --2- 172.21.15.99:0/2165577449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff290102810 0x7ff2901090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.416+0000 7ff294c00700 1 --2- 172.21.15.99:0/2165577449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290100c80 0x7ff2901010a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.416+0000 7ff294c00700 1 --2- 172.21.15.99:0/2165577449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff290101e70 0x7ff2901022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.416+0000 7ff294c00700 1 -- 172.21.15.99:0/2165577449 >> 172.21.15.99:0/2165577449 conn(0x7ff2900fc210 msgr2=0x7ff2900fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:29.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.416+0000 7ff294c00700 1 -- 172.21.15.99:0/2165577449 shutdown_connections 2024-03-30T00:34:29.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.416+0000 7ff294c00700 1 -- 172.21.15.99:0/2165577449 wait complete. 2024-03-30T00:34:29.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff294c00700 1 Processor -- start 2024-03-30T00:34:29.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff294c00700 1 -- start start 2024-03-30T00:34:29.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff294c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290100c80 0x7ff290198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff294c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff290101e70 0x7ff2901992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff294c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff290102810 0x7ff2901a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff294c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2901a0890 con 0x7ff290100c80 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff294c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2901a0a00 con 0x7ff290102810 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff294c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2901a0bc0 con 0x7ff290101e70 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff28de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290100c80 0x7ff290198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff28de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290100c80 0x7ff290198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38592/0 (socket says 172.21.15.99:38592) 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff28de00700 1 -- 172.21.15.99:0/1194669218 learned_addr learned my addr 172.21.15.99:0/1194669218 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.417+0000 7ff28e800700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff290102810 0x7ff2901a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff28d400700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff290101e70 0x7ff2901992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff28de00700 1 -- 172.21.15.99:0/1194669218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff290101e70 msgr2=0x7ff2901992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:29.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff28de00700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff290101e70 0x7ff2901992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff28de00700 1 -- 172.21.15.99:0/1194669218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff290102810 msgr2=0x7ff2901a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:29.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff28de00700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff290102810 0x7ff2901a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff28de00700 1 -- 172.21.15.99:0/1194669218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2800095f0 con 0x7ff290100c80 2024-03-30T00:34:29.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff28d400700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff290101e70 0x7ff2901992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:29.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff28e800700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff290102810 0x7ff2901a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:29.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff28de00700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290100c80 0x7ff290198da0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff280009f50 tx=0x7ff280003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:29.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff286a00700 1 -- 172.21.15.99:0/1194669218 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28001d070 con 0x7ff290100c80 2024-03-30T00:34:29.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff286a00700 1 -- 172.21.15.99:0/1194669218 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff280003f70 con 0x7ff290100c80 2024-03-30T00:34:29.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff294c00700 1 -- 172.21.15.99:0/1194669218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2901a0e40 con 0x7ff290100c80 2024-03-30T00:34:29.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff286a00700 1 -- 172.21.15.99:0/1194669218 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28000f460 con 0x7ff290100c80 2024-03-30T00:34:29.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.418+0000 7ff294c00700 1 -- 172.21.15.99:0/1194669218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2901a11b0 con 0x7ff290100c80 2024-03-30T00:34:29.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.419+0000 7ff294c00700 1 -- 172.21.15.99:0/1194669218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff290067180 con 0x7ff290100c80 2024-03-30T00:34:29.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.421+0000 7ff286a00700 1 -- 172.21.15.99:0/1194669218 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff280022470 con 0x7ff290100c80 2024-03-30T00:34:29.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.422+0000 7ff286a00700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2740a11b0 0x7ff2740a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:29.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.422+0000 7ff286a00700 1 -- 172.21.15.99:0/1194669218 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff2800cdf30 con 0x7ff290100c80 2024-03-30T00:34:29.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.423+0000 7ff28d400700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2740a11b0 0x7ff2740a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:29.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.423+0000 7ff28d400700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2740a11b0 0x7ff2740a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff27c007900 tx=0x7ff27c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:29.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.426+0000 7ff286a00700 1 -- 172.21.15.99:0/1194669218 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff28005d9f0 con 0x7ff290100c80 2024-03-30T00:34:29.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:29 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2035968229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:29.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:29 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1567873627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:29.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.680+0000 7ff294c00700 1 -- 172.21.15.99:0/1194669218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff29010c410 con 0x7ff290100c80 2024-03-30T00:34:29.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.681+0000 7ff286a00700 1 -- 172.21.15.99:0/1194669218 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10860 (secure 0 0 0) 0x7ff28009b010 con 0x7ff290100c80 2024-03-30T00:34:29.688 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:29.688 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:33:21.119410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:34:29.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.683+0000 7ff294c00700 1 -- 172.21.15.99:0/1194669218 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2740a11b0 msgr2=0x7ff2740a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:29.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.683+0000 7ff294c00700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2740a11b0 0x7ff2740a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff27c007900 tx=0x7ff27c008040 comp rx=0 tx=0).stop 2024-03-30T00:34:29.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.683+0000 7ff294c00700 1 -- 172.21.15.99:0/1194669218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290100c80 msgr2=0x7ff290198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:29.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.683+0000 7ff294c00700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290100c80 0x7ff290198da0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff280009f50 tx=0x7ff280003b00 comp rx=0 tx=0).stop 2024-03-30T00:34:29.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.684+0000 7ff294c00700 1 -- 172.21.15.99:0/1194669218 shutdown_connections 2024-03-30T00:34:29.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.684+0000 7ff294c00700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff2740a11b0 0x7ff2740a3670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.684+0000 7ff294c00700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff290102810 0x7ff2901a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.684+0000 7ff294c00700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290100c80 0x7ff290198da0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.684+0000 7ff294c00700 1 --2- 172.21.15.99:0/1194669218 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff290101e70 0x7ff2901992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.684+0000 7ff294c00700 1 -- 172.21.15.99:0/1194669218 >> 172.21.15.99:0/1194669218 conn(0x7ff2900fc210 msgr2=0x7ff290072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:29.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.684+0000 7ff294c00700 1 -- 172.21.15.99:0/1194669218 shutdown_connections 2024-03-30T00:34:29.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.684+0000 7ff294c00700 1 -- 172.21.15.99:0/1194669218 wait complete. 2024-03-30T00:34:29.692 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2024-03-30T00:34:29.709 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:34:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:29 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2035968229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:29 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1567873627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:29.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:29 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2035968229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:29.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:29 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1567873627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:29.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.894+0000 7ff639c00700 1 -- 172.21.15.99:0/2266844170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff634102810 msgr2=0x7ff6341090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:29.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.894+0000 7ff639c00700 1 --2- 172.21.15.99:0/2266844170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff634102810 0x7ff6341090c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00b0d0 tx=0x7ff62c00b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.895+0000 7ff639c00700 1 -- 172.21.15.99:0/2266844170 shutdown_connections 2024-03-30T00:34:29.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.895+0000 7ff639c00700 1 --2- 172.21.15.99:0/2266844170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff634102810 0x7ff6341090c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.895+0000 7ff639c00700 1 --2- 172.21.15.99:0/2266844170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634100c80 0x7ff6341010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.895+0000 7ff639c00700 1 --2- 172.21.15.99:0/2266844170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff634101e70 0x7ff6341022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.895+0000 7ff639c00700 1 -- 172.21.15.99:0/2266844170 >> 172.21.15.99:0/2266844170 conn(0x7ff6340fc1f0 msgr2=0x7ff6340fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:29.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.895+0000 7ff639c00700 1 -- 172.21.15.99:0/2266844170 shutdown_connections 2024-03-30T00:34:29.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.895+0000 7ff639c00700 1 -- 172.21.15.99:0/2266844170 wait complete. 2024-03-30T00:34:29.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.896+0000 7ff639c00700 1 Processor -- start 2024-03-30T00:34:29.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.896+0000 7ff639c00700 1 -- start start 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.896+0000 7ff639c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634100c80 0x7ff634198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.896+0000 7ff639c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff634101e70 0x7ff6341993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.896+0000 7ff639c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff634102810 0x7ff6341a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.896+0000 7ff639c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6341a0970 con 0x7ff634101e70 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.896+0000 7ff639c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6341a0ab0 con 0x7ff634102810 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.896+0000 7ff639c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6341a0c40 con 0x7ff634100c80 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff632000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff634101e70 0x7ff6341993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff632000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff634101e70 0x7ff6341993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38618/0 (socket says 172.21.15.99:38618) 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff632000700 1 -- 172.21.15.99:0/224847653 learned_addr learned my addr 172.21.15.99:0/224847653 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff633400700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff634102810 0x7ff6341a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff632a00700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634100c80 0x7ff634198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff632000700 1 -- 172.21.15.99:0/224847653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634100c80 msgr2=0x7ff634198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff632000700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634100c80 0x7ff634198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff632000700 1 -- 172.21.15.99:0/224847653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff634102810 msgr2=0x7ff6341a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:29.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff632000700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff634102810 0x7ff6341a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:29.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff632000700 1 -- 172.21.15.99:0/224847653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c009d00 con 0x7ff634101e70 2024-03-30T00:34:29.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff632000700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff634101e70 0x7ff6341993d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff62000e9a0 tx=0x7ff62000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:29.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.897+0000 7ff62b400700 1 -- 172.21.15.99:0/224847653 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620009790 con 0x7ff634101e70 2024-03-30T00:34:29.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.898+0000 7ff62b400700 1 -- 172.21.15.99:0/224847653 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff62000cbd0 con 0x7ff634101e70 2024-03-30T00:34:29.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.898+0000 7ff639c00700 1 -- 172.21.15.99:0/224847653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6341a0f20 con 0x7ff634101e70 2024-03-30T00:34:29.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.898+0000 7ff62b400700 1 -- 172.21.15.99:0/224847653 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620018680 con 0x7ff634101e70 2024-03-30T00:34:29.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.898+0000 7ff639c00700 1 -- 172.21.15.99:0/224847653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6341a13a0 con 0x7ff634101e70 2024-03-30T00:34:29.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.899+0000 7ff639c00700 1 -- 172.21.15.99:0/224847653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff634067a10 con 0x7ff634101e70 2024-03-30T00:34:29.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.901+0000 7ff62b400700 1 -- 172.21.15.99:0/224847653 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff62000cd40 con 0x7ff634101e70 2024-03-30T00:34:29.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.902+0000 7ff62b400700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff6180a11b0 0x7ff6180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:29.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.902+0000 7ff62b400700 1 -- 172.21.15.99:0/224847653 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff620014070 con 0x7ff634101e70 2024-03-30T00:34:29.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.902+0000 7ff632a00700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff6180a11b0 0x7ff6180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:29.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.903+0000 7ff632a00700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff6180a11b0 0x7ff6180a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff624005950 tx=0x7ff6240058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:29.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:29.905+0000 7ff62b400700 1 -- 172.21.15.99:0/224847653 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff6200529f0 con 0x7ff634101e70 2024-03-30T00:34:30.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.144+0000 7ff639c00700 1 -- 172.21.15.99:0/224847653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff63410c350 con 0x7ff634101e70 2024-03-30T00:34:30.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.146+0000 7ff62b400700 1 -- 172.21.15.99:0/224847653 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22615 (secure 0 0 0) 0x7ff6200962f0 con 0x7ff634101e70 2024-03-30T00:34:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:30.153 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:34:10.248126+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8},{"from":5,"to":7}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:34:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.148+0000 7ff639c00700 1 -- 172.21.15.99:0/224847653 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff6180a11b0 msgr2=0x7ff6180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.148+0000 7ff639c00700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff6180a11b0 0x7ff6180a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff624005950 tx=0x7ff6240058e0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.148+0000 7ff639c00700 1 -- 172.21.15.99:0/224847653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff634101e70 msgr2=0x7ff6341993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.148+0000 7ff639c00700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff634101e70 0x7ff6341993d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff62000e9a0 tx=0x7ff62000ecb0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.148+0000 7ff639c00700 1 -- 172.21.15.99:0/224847653 shutdown_connections 2024-03-30T00:34:30.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.148+0000 7ff639c00700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff6180a11b0 0x7ff6180a3670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.148+0000 7ff639c00700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff634102810 0x7ff6341a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.148+0000 7ff639c00700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634100c80 0x7ff634198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.148+0000 7ff639c00700 1 --2- 172.21.15.99:0/224847653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff634101e70 0x7ff6341993d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.149+0000 7ff639c00700 1 -- 172.21.15.99:0/224847653 >> 172.21.15.99:0/224847653 conn(0x7ff6340fc1f0 msgr2=0x7ff634072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.149+0000 7ff639c00700 1 -- 172.21.15.99:0/224847653 shutdown_connections 2024-03-30T00:34:30.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.149+0000 7ff639c00700 1 -- 172.21.15.99:0/224847653 wait complete. 2024-03-30T00:34:30.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-30T00:33:15.648899+0000', 'modified': '2024-03-30T00:33:21.119410+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24572, 'mds_1': 24559, 'mds_2': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24473': {'gid': 24473, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6833/780858221', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 780858221}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 780858221}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24502': {'gid': 24502, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.161:6835/80094727', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 80094727}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 80094727}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24503': {'gid': 24503, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/1641167461', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 1641167461}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 1641167461}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24544': {'gid': 24544, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6839/1486341215', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 1486341215}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 1486341215}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24559': {'gid': 24559, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.99:6841/1391823621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6840', 'nonce': 1391823621}, {'type': 'v1', 'addr': '172.21.15.99:6841', 'nonce': 1391823621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6839/2862512313', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 2862512313}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 2862512313}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T00:34:30.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-30T00:34:30.175 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-30T00:34:30.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.363+0000 7f23c2000700 1 -- 172.21.15.99:0/3092500408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc101e80 msgr2=0x7f23bc1022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.363+0000 7f23c2000700 1 --2- 172.21.15.99:0/3092500408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc101e80 0x7f23bc1022e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f23a80099c0 tx=0x7f23a8009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.363+0000 7f23c2000700 1 -- 172.21.15.99:0/3092500408 shutdown_connections 2024-03-30T00:34:30.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.363+0000 7f23c2000700 1 --2- 172.21.15.99:0/3092500408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23bc102820 0x7f23bc1090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.363+0000 7f23c2000700 1 --2- 172.21.15.99:0/3092500408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23bc100c90 0x7f23bc1010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.363+0000 7f23c2000700 1 --2- 172.21.15.99:0/3092500408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc101e80 0x7f23bc1022e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.363+0000 7f23c2000700 1 -- 172.21.15.99:0/3092500408 >> 172.21.15.99:0/3092500408 conn(0x7f23bc0fc220 msgr2=0x7f23bc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.363+0000 7f23c2000700 1 -- 172.21.15.99:0/3092500408 shutdown_connections 2024-03-30T00:34:30.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.363+0000 7f23c2000700 1 -- 172.21.15.99:0/3092500408 wait complete. 2024-03-30T00:34:30.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.364+0000 7f23c2000700 1 Processor -- start 2024-03-30T00:34:30.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.364+0000 7f23c2000700 1 -- start start 2024-03-30T00:34:30.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.364+0000 7f23c2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23bc100c90 0x7f23bc198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.364+0000 7f23c2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc101e80 0x7f23bc199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.364+0000 7f23c2000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23bc102820 0x7f23bc1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.364+0000 7f23c2000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1a0800 con 0x7f23bc101e80 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.364+0000 7f23c2000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1a0970 con 0x7f23bc100c90 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.364+0000 7f23c2000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1a0b30 con 0x7f23bc102820 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23baa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc101e80 0x7f23bc199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23baa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc101e80 0x7f23bc199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38634/0 (socket says 172.21.15.99:38634) 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23baa00700 1 -- 172.21.15.99:0/327900198 learned_addr learned my addr 172.21.15.99:0/327900198 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23bb400700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23bc100c90 0x7f23bc198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23bbe00700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23bc102820 0x7f23bc1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23baa00700 1 -- 172.21.15.99:0/327900198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23bc102820 msgr2=0x7f23bc1a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23baa00700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23bc102820 0x7f23bc1a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23baa00700 1 -- 172.21.15.99:0/327900198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23bc100c90 msgr2=0x7f23bc198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23baa00700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23bc100c90 0x7f23bc198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23baa00700 1 -- 172.21.15.99:0/327900198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a80095f0 con 0x7f23bc101e80 2024-03-30T00:34:30.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23bbe00700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23bc102820 0x7f23bc1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:30.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.365+0000 7f23baa00700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc101e80 0x7f23bc199220 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009990 tx=0x7f23a800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.366+0000 7f23b3e00700 1 -- 172.21.15.99:0/327900198 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a801d070 con 0x7f23bc101e80 2024-03-30T00:34:30.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.366+0000 7f23b3e00700 1 -- 172.21.15.99:0/327900198 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23a800bd10 con 0x7f23bc101e80 2024-03-30T00:34:30.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.366+0000 7f23c2000700 1 -- 172.21.15.99:0/327900198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc1a0db0 con 0x7f23bc101e80 2024-03-30T00:34:30.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.366+0000 7f23b3e00700 1 -- 172.21.15.99:0/327900198 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8017650 con 0x7f23bc101e80 2024-03-30T00:34:30.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.366+0000 7f23c2000700 1 -- 172.21.15.99:0/327900198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc1a11d0 con 0x7f23bc101e80 2024-03-30T00:34:30.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.370+0000 7f23b3e00700 1 -- 172.21.15.99:0/327900198 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f23a8022470 con 0x7f23bc101e80 2024-03-30T00:34:30.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.371+0000 7f23d5ad2b80 1 Processor -- start 2024-03-30T00:34:30.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.371+0000 7f23d5ad2b80 1 -- start start 2024-03-30T00:34:30.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.371+0000 7f23b3e00700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23a00a1220 0x7f23a00a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.371+0000 7f23b3e00700 1 -- 172.21.15.99:0/327900198 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f23a80cdd60 con 0x7f23bc101e80 2024-03-30T00:34:30.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.371+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338c9780 0x5605338c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.371+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338ca0e0 0x5605338ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.371+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 0x5605338cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.371+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533821940 con 0x5605338ca0e0 2024-03-30T00:34:30.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.371+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56053381f300 con 0x5605338caf80 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.371+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56053381abf0 con 0x5605338c9780 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23bbe00700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23a00a1220 0x7f23a00a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23baa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338ca0e0 0x5605338ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23baa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338ca0e0 0x5605338ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38636/0 (socket says 172.21.15.99:38636) 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23baa00700 1 -- 172.21.15.99:0/4070381569 learned_addr learned my addr 172.21.15.99:0/4070381569 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23bb400700 1 --2- 172.21.15.99:0/4070381569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338c9780 0x5605338c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23baa00700 1 -- 172.21.15.99:0/4070381569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338c9780 msgr2=0x5605338c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23baa00700 1 --2- 172.21.15.99:0/4070381569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338c9780 0x5605338c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23baa00700 1 -- 172.21.15.99:0/4070381569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 msgr2=0x5605338cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23baa00700 1 --2- 172.21.15.99:0/4070381569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 0x5605338cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23baa00700 1 -- 172.21.15.99:0/4070381569 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56053369c360 con 0x5605338ca0e0 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23bbe00700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23a00a1220 0x7f23a00a36e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f23b400e290 tx=0x7f23b400e650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.372+0000 7f23baa00700 1 --2- 172.21.15.99:0/4070381569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338ca0e0 0x5605338ca9a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f23a809b000 tx=0x7f23a80974e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26dbda81bc87e5d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23a7e00700 1 -- 172.21.15.99:0/4070381569 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a809b070 con 0x5605338ca0e0 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23a7e00700 1 -- 172.21.15.99:0/4070381569 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23a809eea0 con 0x5605338ca0e0 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23a7e00700 1 -- 172.21.15.99:0/4070381569 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a809b2d0 con 0x5605338ca0e0 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/4070381569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338ca0e0 msgr2=0x5605338ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/4070381569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338ca0e0 0x5605338ca9a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f23a809b000 tx=0x7f23a80974e0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/4070381569 shutdown_connections 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/4070381569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 0x5605338cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/4070381569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338c9780 0x5605338c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/4070381569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338ca0e0 0x5605338ca9a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/4070381569 >> 172.21.15.99:0/4070381569 conn(0x560533828650 msgr2=0x5605338c4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/4070381569 shutdown_connections 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.373+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/4070381569 wait complete. 2024-03-30T00:34:30.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.374+0000 7f23d5ad2b80 1 -- start start 2024-03-30T00:34:30.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.374+0000 7f23d5ad2b80 1 Processor -- start 2024-03-30T00:34:30.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.374+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338c9780 0x5605338d32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.374+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338caf80 0x5605338da830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605337b6d30 con 0x5605338caf80 2024-03-30T00:34:30.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605337b69d0 con 0x5605338c9780 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56053369e650 con 0x5605338ca0e0 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23d5ad2b80 10 client.0 _ll_register_callbacks cb 0x5605337c0420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23bbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338caf80 0x5605338da830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23bbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338caf80 0x5605338da830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38648/0 (socket says 172.21.15.99:38648) 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23bbe00700 1 -- 172.21.15.99:0/3889704324 learned_addr learned my addr 172.21.15.99:0/3889704324 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23baa00700 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23bb400700 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338c9780 0x5605338d32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23bbe00700 1 -- 172.21.15.99:0/3889704324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 msgr2=0x5605338d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23bbe00700 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338d37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23bbe00700 1 -- 172.21.15.99:0/3889704324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338c9780 msgr2=0x5605338d32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23bbe00700 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338c9780 0x5605338d32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23bbe00700 1 -- 172.21.15.99:0/3889704324 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc1a11d0 con 0x5605338caf80 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23bbe00700 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338caf80 0x5605338da830 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f23b4018e30 tx=0x7f23b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bb5d6930f91ac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.375+0000 7f23a4c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.376+0000 7f23a4c00700 1 -- 172.21.15.99:0/3889704324 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b401d070 con 0x5605338caf80 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.376+0000 7f23a4c00700 1 -- 172.21.15.99:0/3889704324 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f23b4015ba0 con 0x5605338caf80 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.376+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/3889704324 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605337db330 con 0x5605338caf80 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.376+0000 7f23a4c00700 1 -- 172.21.15.99:0/3889704324 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b4015da0 con 0x5605338caf80 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.376+0000 7f23d5ad2b80 10 client.15285 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.376+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/3889704324 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560533648b10 con 0x5605338caf80 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.376+0000 7f23a4c00700 1 -- 172.21.15.99:0/3889704324 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23b4031080 con 0x5605338caf80 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.376+0000 7f23a4c00700 1 -- 172.21.15.99:0/3889704324 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f23b40163b0 con 0x5605338caf80 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.376+0000 7f23d5ad2b80 10 client.15285 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.377+0000 7f23d5ad2b80 10 client.15285 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.377+0000 7f23d5ad2b80 20 client.15285 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.377+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5605338de500 0x5605338e37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.377+0000 7f23d5ad2b80 4 client.15285 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:34:30.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.377+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/3889704324 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605338136b0 con 0x5605338de500 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.377+0000 7f23bb400700 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5605338de500 0x5605338e37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.378+0000 7f23bb400700 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5605338de500 0x5605338e37e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cccb6cf94b944d88 server_cookie=d7cd4c8def8838e5 in_seq=0 out_seq=0 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.378+0000 7f23a4c00700 10 client.15285 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.378+0000 7f23a4c00700 1 -- 172.21.15.99:0/3889704324 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5605338136b0 con 0x5605338de500 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.378+0000 7f23a4c00700 10 client.15285 handle_command_reply: tid=0 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.378+0000 7f23d5ad2b80 1 client.15285 shutdown 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/3889704324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338caf80 msgr2=0x5605338da830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338caf80 0x5605338da830 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f23b4018e30 tx=0x7f23b4004ad0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/3889704324 shutdown_connections 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5605338de500 0x5605338e37e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23a4c00700 0 client.15285 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:34:30.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338caf80 0x5605338da830 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338c9780 0x5605338d32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/3889704324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/3889704324 >> 172.21.15.99:0/3889704324 conn(0x560533828650 msgr2=0x5605338c8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/3889704324 shutdown_connections 2024-03-30T00:34:30.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/3889704324 wait complete. 2024-03-30T00:34:30.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.379+0000 7f23d5ad2b80 20 client.15285 trim_cache size 0 max 16384 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23d5ad2b80 1 Processor -- start 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23d5ad2b80 1 -- start start 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338daee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 0x5605338cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533813990 con 0x5605338c9780 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533817850 con 0x5605338caf80 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605337b9de0 con 0x5605338ca0e0 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23bb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23bb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38652/0 (socket says 172.21.15.99:38652) 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23bb400700 1 -- 172.21.15.99:0/4058204562 learned_addr learned my addr 172.21.15.99:0/4058204562 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23baa00700 1 --2- 172.21.15.99:0/4058204562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338daee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23bb400700 1 -- 172.21.15.99:0/4058204562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 msgr2=0x5605338daee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.397+0000 7f23bb400700 1 --2- 172.21.15.99:0/4058204562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338daee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.398+0000 7f23bb400700 1 -- 172.21.15.99:0/4058204562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 msgr2=0x5605338cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.398+0000 7f23bb400700 1 --2- 172.21.15.99:0/4058204562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 0x5605338cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.398+0000 7f23bb400700 1 -- 172.21.15.99:0/4058204562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605337e0e50 con 0x5605338c9780 2024-03-30T00:34:30.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.398+0000 7f23bb400700 1 --2- 172.21.15.99:0/4058204562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338c9b60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5605338dfa60 tx=0x7f23ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e8332bb14f0a57 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.398+0000 7f23b1200700 1 -- 172.21.15.99:0/4058204562 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac01f050 con 0x5605338c9780 2024-03-30T00:34:30.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.398+0000 7f23b1200700 1 -- 172.21.15.99:0/4058204562 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23ac018040 con 0x5605338c9780 2024-03-30T00:34:30.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.398+0000 7f23b1200700 1 -- 172.21.15.99:0/4058204562 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac01e680 con 0x5605338c9780 2024-03-30T00:34:30.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.399+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/4058204562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 msgr2=0x5605338c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.399+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/4058204562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338c9b60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5605338dfa60 tx=0x7f23ac009260 comp rx=0 tx=0).stop 2024-03-30T00:34:30.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.399+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/4058204562 shutdown_connections 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.399+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/4058204562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 0x5605338cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.399+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/4058204562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338c9b60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.399+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/4058204562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338daee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.399+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/4058204562 >> 172.21.15.99:0/4058204562 conn(0x560533828650 msgr2=0x5605338d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.399+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/4058204562 shutdown_connections 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.399+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/4058204562 wait complete. 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.399+0000 7f23d5ad2b80 1 -- start start 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.399+0000 7f23d5ad2b80 1 Processor -- start 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338d7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23d5ad2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 0x560533932dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605338daaf0 con 0x5605338c9780 2024-03-30T00:34:30.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605338dac60 con 0x5605338caf80 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23d5ad2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560533933310 con 0x5605338ca0e0 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23d5ad2b80 10 client.0 _ll_register_callbacks cb 0x5605337c0420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23bb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23bb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338d79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38668/0 (socket says 172.21.15.99:38668) 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23bb400700 1 -- 172.21.15.99:0/2870214404 learned_addr learned my addr 172.21.15.99:0/2870214404 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23baa00700 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338d7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23bbe00700 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 0x560533932dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23bb400700 1 -- 172.21.15.99:0/2870214404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 msgr2=0x5605338d7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23bb400700 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338d7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23bb400700 1 -- 172.21.15.99:0/2870214404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 msgr2=0x560533932dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23bb400700 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 0x560533932dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.400+0000 7f23bb400700 1 -- 172.21.15.99:0/2870214404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ac009dc0 con 0x5605338c9780 2024-03-30T00:34:30.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.401+0000 7f23bb400700 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338d79d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f23ac000c00 tx=0x7f23ac0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb69db5f3817f9b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.401+0000 7f23a4c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.401+0000 7f23a4c00700 1 -- 172.21.15.99:0/2870214404 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac01f460 con 0x5605338c9780 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.401+0000 7f23a4c00700 1 -- 172.21.15.99:0/2870214404 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f23ac004040 con 0x5605338c9780 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.401+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/2870214404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23a809eea0 con 0x5605338c9780 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.401+0000 7f23a4c00700 1 -- 172.21.15.99:0/2870214404 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23ac011960 con 0x5605338c9780 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.401+0000 7f23d5ad2b80 10 client.15291 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.401+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/2870214404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5605336a0c80 con 0x5605338c9780 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.401+0000 7f23a4c00700 1 -- 172.21.15.99:0/2870214404 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23a809eea0 con 0x5605338c9780 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.402+0000 7f23a4c00700 1 -- 172.21.15.99:0/2870214404 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f23ac028430 con 0x5605338c9780 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.402+0000 7f23d5ad2b80 10 client.15291 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.402+0000 7f23d5ad2b80 10 client.15291 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.402+0000 7f23d5ad2b80 20 client.15291 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.402+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560533937f10 0x5605339348c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.402+0000 7f23d5ad2b80 4 client.15291 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.402+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/2870214404 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560533669ca0 con 0x560533937f10 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.402+0000 7f23baa00700 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560533937f10 0x5605339348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23a4c00700 10 client.15291 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23baa00700 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560533937f10 0x5605339348c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9258bbe9b5f9b42 server_cookie=bf19da49782f86f5 in_seq=0 out_seq=0 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23a4c00700 1 -- 172.21.15.99:0/2870214404 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560533669ca0 con 0x560533937f10 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23a4c00700 10 client.15291 handle_command_reply: tid=0 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23d5ad2b80 1 client.15291 shutdown 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/2870214404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 msgr2=0x5605338d79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338d79d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f23ac000c00 tx=0x7f23ac0057d0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/2870214404 shutdown_connections 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x560533937f10 0x5605339348c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5605338caf80 0x560533932dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605338c9780 0x5605338d79d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23a4c00700 0 client.15291 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:34:30.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23d5ad2b80 1 --2- 172.21.15.99:0/2870214404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605338ca0e0 0x5605338d7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.403+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/2870214404 >> 172.21.15.99:0/2870214404 conn(0x560533828650 msgr2=0x5605338ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/2870214404 shutdown_connections 2024-03-30T00:34:30.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23d5ad2b80 1 -- 172.21.15.99:0/2870214404 wait complete. 2024-03-30T00:34:30.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23d5ad2b80 20 client.15291 trim_cache size 0 max 16384 2024-03-30T00:34:30.415 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:34:30.415 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "03801b1a-ee2c-11ee-b647-cb9ed24678a4", 2024-03-30T00:34:30.415 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 2, 2024-03-30T00:34:30.415 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24502, 2024-03-30T00:34:30.416 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-30T00:34:30.416 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-30T00:34:30.416 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-30T00:34:30.416 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 70.300486785000004, 2024-03-30T00:34:30.416 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2024-03-30T00:34:30.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 85, 2024-03-30T00:34:30.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T00:34:30.416 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 96.609583345000004 2024-03-30T00:34:30.416 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:34:30.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23c2000700 1 -- 172.21.15.99:0/327900198 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23a00a1220 msgr2=0x7f23a00a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23c2000700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23a00a1220 0x7f23a00a36e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f23b400e290 tx=0x7f23b400e650 comp rx=0 tx=0).stop 2024-03-30T00:34:30.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23c2000700 1 -- 172.21.15.99:0/327900198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc101e80 msgr2=0x7f23bc199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23c2000700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc101e80 0x7f23bc199220 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009990 tx=0x7f23a800fa00 comp rx=0 tx=0).stop 2024-03-30T00:34:30.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23c2000700 1 -- 172.21.15.99:0/327900198 shutdown_connections 2024-03-30T00:34:30.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23c2000700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23bc102820 0x7f23bc1a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23c2000700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f23a00a1220 0x7f23a00a36e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23c2000700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23bc100c90 0x7f23bc198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23c2000700 1 --2- 172.21.15.99:0/327900198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23bc101e80 0x7f23bc199220 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23c2000700 1 -- 172.21.15.99:0/327900198 >> 172.21.15.99:0/327900198 conn(0x7f23bc0fc220 msgr2=0x7f23bc072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.404+0000 7f23c2000700 1 -- 172.21.15.99:0/327900198 shutdown_connections 2024-03-30T00:34:30.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.405+0000 7f23c2000700 1 -- 172.21.15.99:0/327900198 wait complete. 2024-03-30T00:34:30.426 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:30 smithi099 ceph-mon[38079]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:34:30.426 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:30 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1194669218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:30.426 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:30 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/224847653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:30.429 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-30T00:34:30.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.615+0000 7f0a7e600700 1 -- 172.21.15.99:0/808670576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78101f70 msgr2=0x7f0a781023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.615+0000 7f0a7e600700 1 --2- 172.21.15.99:0/808670576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78101f70 0x7f0a781023d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0a640099c0 tx=0x7f0a64009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.615+0000 7f0a7e600700 1 -- 172.21.15.99:0/808670576 shutdown_connections 2024-03-30T00:34:30.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.615+0000 7f0a7e600700 1 --2- 172.21.15.99:0/808670576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a78102910 0x7f0a781091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.615+0000 7f0a7e600700 1 --2- 172.21.15.99:0/808670576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a78100d80 0x7f0a781011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.615+0000 7f0a7e600700 1 --2- 172.21.15.99:0/808670576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78101f70 0x7f0a781023d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.615+0000 7f0a7e600700 1 -- 172.21.15.99:0/808670576 >> 172.21.15.99:0/808670576 conn(0x7f0a780fc2f0 msgr2=0x7f0a780fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.615+0000 7f0a7e600700 1 -- 172.21.15.99:0/808670576 shutdown_connections 2024-03-30T00:34:30.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.615+0000 7f0a7e600700 1 -- 172.21.15.99:0/808670576 wait complete. 2024-03-30T00:34:30.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.616+0000 7f0a7e600700 1 Processor -- start 2024-03-30T00:34:30.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.616+0000 7f0a7e600700 1 -- start start 2024-03-30T00:34:30.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a7e600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a78100d80 0x7f0a78198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a7e600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a78101f70 0x7f0a78199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a7e600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78102910 0x7f0a781a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a7e600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a781a0940 con 0x7f0a78102910 2024-03-30T00:34:30.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a7e600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a781a0a80 con 0x7f0a78100d80 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a7e600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a781a0c10 con 0x7f0a78101f70 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a77e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78102910 0x7f0a781a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a77e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78102910 0x7f0a781a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38686/0 (socket says 172.21.15.99:38686) 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a77e00700 1 -- 172.21.15.99:0/3900827942 learned_addr learned my addr 172.21.15.99:0/3900827942 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a76a00700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a78101f70 0x7f0a78199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a77400700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a78100d80 0x7f0a78198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a77e00700 1 -- 172.21.15.99:0/3900827942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a78101f70 msgr2=0x7f0a78199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a77e00700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a78101f70 0x7f0a78199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a77e00700 1 -- 172.21.15.99:0/3900827942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a78100d80 msgr2=0x7f0a78198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a77e00700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a78100d80 0x7f0a78198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a77e00700 1 -- 172.21.15.99:0/3900827942 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a640095f0 con 0x7f0a78102910 2024-03-30T00:34:30.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.617+0000 7f0a77400700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a78100d80 0x7f0a78198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:30.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.618+0000 7f0a77e00700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78102910 0x7f0a781a0370 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0a70007d70 tx=0x7f0a7000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.618+0000 7f0a6fe00700 1 -- 172.21.15.99:0/3900827942 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000ad40 con 0x7f0a78102910 2024-03-30T00:34:30.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.618+0000 7f0a6fe00700 1 -- 172.21.15.99:0/3900827942 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a700128b0 con 0x7f0a78102910 2024-03-30T00:34:30.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.618+0000 7f0a7e600700 1 -- 172.21.15.99:0/3900827942 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a781a0ef0 con 0x7f0a78102910 2024-03-30T00:34:30.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.618+0000 7f0a6fe00700 1 -- 172.21.15.99:0/3900827942 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a700115a0 con 0x7f0a78102910 2024-03-30T00:34:30.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.618+0000 7f0a7e600700 1 -- 172.21.15.99:0/3900827942 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a781a13c0 con 0x7f0a78102910 2024-03-30T00:34:30.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.622+0000 7f0a6fe00700 1 -- 172.21.15.99:0/3900827942 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f0a70012a20 con 0x7f0a78102910 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.623+0000 7f0a921dfb80 1 Processor -- start 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.623+0000 7f0a921dfb80 1 -- start start 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a6fe00700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0a5c0a1230 0x7f0a5c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41462780 0x55aa41462ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 0x55aa414639a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41463f80 0x55aa41464400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa413ba940 con 0x55aa414630e0 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa413b8300 con 0x55aa41463f80 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa413b3bf0 con 0x55aa41462780 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a6fe00700 1 -- 172.21.15.99:0/3900827942 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f0a700ccb60 con 0x7f0a78102910 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a77400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41462780 0x55aa41462ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a77400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41462780 0x55aa41462ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:56076/0 (socket says 172.21.15.99:56076) 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a77400700 1 -- 172.21.15.99:0/751892726 learned_addr learned my addr 172.21.15.99:0/751892726 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a76a00700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0a5c0a1230 0x7f0a5c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a77e00700 1 --2- 172.21.15.99:0/751892726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41463f80 0x55aa41464400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.624+0000 7f0a76a00700 1 --2- 172.21.15.99:0/751892726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 0x55aa414639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.625+0000 7f0a77e00700 1 -- 172.21.15.99:0/751892726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41462780 msgr2=0x55aa41462ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.625+0000 7f0a77e00700 1 --2- 172.21.15.99:0/751892726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41462780 0x55aa41462ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.625+0000 7f0a77e00700 1 -- 172.21.15.99:0/751892726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 msgr2=0x55aa414639a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.625+0000 7f0a77e00700 1 --2- 172.21.15.99:0/751892726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 0x55aa414639a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.625+0000 7f0a77e00700 1 -- 172.21.15.99:0/751892726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa41235360 con 0x55aa41463f80 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.625+0000 7f0a77400700 1 --2- 172.21.15.99:0/751892726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41462780 0x55aa41462ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.625+0000 7f0a76a00700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0a5c0a1230 0x7f0a5c0a36f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009990 tx=0x7f0a64000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.625+0000 7f0a77e00700 1 --2- 172.21.15.99:0/751892726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41463f80 0x55aa41464400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0a700ca5b0 tx=0x7f0a70099b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9172673e15e8f2a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.625+0000 7f0a6cc00700 1 -- 172.21.15.99:0/751892726 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70099ef0 con 0x55aa41463f80 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.625+0000 7f0a6cc00700 1 -- 172.21.15.99:0/751892726 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a7009dd20 con 0x55aa41463f80 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.625+0000 7f0a6cc00700 1 -- 172.21.15.99:0/751892726 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a700a1ee0 con 0x55aa41463f80 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.626+0000 7f0a921dfb80 1 -- 172.21.15.99:0/751892726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41463f80 msgr2=0x55aa41464400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.626+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/751892726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41463f80 0x55aa41464400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0a700ca5b0 tx=0x7f0a70099b60 comp rx=0 tx=0).stop 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.626+0000 7f0a921dfb80 1 -- 172.21.15.99:0/751892726 shutdown_connections 2024-03-30T00:34:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.626+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/751892726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41463f80 0x55aa41464400 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.626+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/751892726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41462780 0x55aa41462ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.626+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/751892726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 0x55aa414639a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.626+0000 7f0a921dfb80 1 -- 172.21.15.99:0/751892726 >> 172.21.15.99:0/751892726 conn(0x55aa413c1650 msgr2=0x55aa4145d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.626+0000 7f0a921dfb80 1 -- 172.21.15.99:0/751892726 shutdown_connections 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.626+0000 7f0a921dfb80 1 -- 172.21.15.99:0/751892726 wait complete. 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a921dfb80 1 -- start start 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a921dfb80 1 Processor -- start 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa414630e0 0x55aa4146cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41463f80 0x55aa41473bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4134fd30 con 0x55aa41462780 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4134f9d0 con 0x55aa41463f80 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41237650 con 0x55aa414630e0 2024-03-30T00:34:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a921dfb80 10 client.0 _ll_register_callbacks cb 0x55aa41359420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a77400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a77400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38706/0 (socket says 172.21.15.99:38706) 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.627+0000 7f0a77400700 1 -- 172.21.15.99:0/2472294866 learned_addr learned my addr 172.21.15.99:0/2472294866 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a77e00700 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41463f80 0x55aa41473bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a76a00700 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa414630e0 0x55aa4146cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a77400700 1 -- 172.21.15.99:0/2472294866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa414630e0 msgr2=0x55aa4146cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a77400700 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa414630e0 0x55aa4146cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a77400700 1 -- 172.21.15.99:0/2472294866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41463f80 msgr2=0x55aa41473bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a77400700 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41463f80 0x55aa41473bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a77400700 1 -- 172.21.15.99:0/2472294866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6800b9a0 con 0x55aa41462780 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a77400700 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146c690 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0a68012700 tx=0x7f0a68012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b9fefc2342617e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a61600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a61600700 1 -- 172.21.15.99:0/2472294866 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68004220 con 0x55aa41462780 2024-03-30T00:34:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2472294866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa41374330 con 0x55aa41462780 2024-03-30T00:34:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.628+0000 7f0a61600700 1 -- 172.21.15.99:0/2472294866 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f0a68004380 con 0x55aa41462780 2024-03-30T00:34:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a61600700 1 -- 172.21.15.99:0/2472294866 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a68004540 con 0x55aa41462780 2024-03-30T00:34:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a921dfb80 10 client.15303 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2472294866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f0a781a13c0 con 0x55aa41462780 2024-03-30T00:34:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a61600700 1 -- 172.21.15.99:0/2472294866 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aa41374330 con 0x55aa41462780 2024-03-30T00:34:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a61600700 1 -- 172.21.15.99:0/2472294866 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0a68023070 con 0x55aa41462780 2024-03-30T00:34:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a921dfb80 10 client.15303 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a921dfb80 10 client.15303 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a921dfb80 20 client.15303 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55aa414778b0 0x55aa4147cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a921dfb80 4 client.15303 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2472294866 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa413ac6b0 con 0x55aa414778b0 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.629+0000 7f0a76a00700 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55aa414778b0 0x55aa4147cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.630+0000 7f0a61600700 10 client.15303 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.630+0000 7f0a76a00700 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55aa414778b0 0x55aa4147cb90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ffa7c51f4061f1fb server_cookie=f9fbde07256fd73a in_seq=0 out_seq=0 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a61600700 1 -- 172.21.15.99:0/2472294866 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa413ac6b0 con 0x55aa414778b0 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a61600700 10 client.15303 handle_command_reply: tid=0 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 1 client.15303 shutdown 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2472294866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 msgr2=0x55aa4146c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146c690 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0a68012700 tx=0x7f0a68012a10 comp rx=0 tx=0).stop 2024-03-30T00:34:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2472294866 shutdown_connections 2024-03-30T00:34:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55aa414778b0 0x55aa4147cb90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41463f80 0x55aa41473bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146c690 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a61600700 0 client.15303 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:34:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2472294866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa414630e0 0x55aa4146cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2472294866 >> 172.21.15.99:0/2472294866 conn(0x55aa413c1650 msgr2=0x55aa41461120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2472294866 shutdown_connections 2024-03-30T00:34:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2472294866 wait complete. 2024-03-30T00:34:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.631+0000 7f0a921dfb80 20 client.15303 trim_cache size 0 max 16384 2024-03-30T00:34:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.649+0000 7f0a921dfb80 1 Processor -- start 2024-03-30T00:34:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.649+0000 7f0a921dfb80 1 -- start start 2024-03-30T00:34:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.649+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.649+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa414630e0 0x55aa4146bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.649+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41463f80 0x55aa41473b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.649+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41374330 con 0x55aa41462780 2024-03-30T00:34:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.649+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41477010 con 0x55aa414630e0 2024-03-30T00:34:30.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.649+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41371ad0 con 0x55aa41463f80 2024-03-30T00:34:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.649+0000 7f0a77400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.649+0000 7f0a77400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38718/0 (socket says 172.21.15.99:38718) 2024-03-30T00:34:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.650+0000 7f0a77400700 1 -- 172.21.15.99:0/443631442 learned_addr learned my addr 172.21.15.99:0/443631442 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.650+0000 7f0a77e00700 1 --2- 172.21.15.99:0/443631442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41463f80 0x55aa41473b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.650+0000 7f0a77400700 1 -- 172.21.15.99:0/443631442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41463f80 msgr2=0x55aa41473b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.650+0000 7f0a77400700 1 --2- 172.21.15.99:0/443631442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41463f80 0x55aa41473b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.650+0000 7f0a77400700 1 -- 172.21.15.99:0/443631442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa414630e0 msgr2=0x55aa4146bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.650+0000 7f0a77400700 1 --2- 172.21.15.99:0/443631442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa414630e0 0x55aa4146bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.650+0000 7f0a77400700 1 -- 172.21.15.99:0/443631442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa411e1b10 con 0x55aa41462780 2024-03-30T00:34:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.650+0000 7f0a77400700 1 --2- 172.21.15.99:0/443631442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146b8b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a68012f20 tx=0x7f0a6800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b438c807270d6b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.650+0000 7f0a6d600700 1 -- 172.21.15.99:0/443631442 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68010c80 con 0x55aa41462780 2024-03-30T00:34:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.650+0000 7f0a6d600700 1 -- 172.21.15.99:0/443631442 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a68010e40 con 0x55aa41462780 2024-03-30T00:34:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.650+0000 7f0a6d600700 1 -- 172.21.15.99:0/443631442 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68005290 con 0x55aa41462780 2024-03-30T00:34:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.651+0000 7f0a921dfb80 1 -- 172.21.15.99:0/443631442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 msgr2=0x55aa4146b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.651+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/443631442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146b8b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a68012f20 tx=0x7f0a6800b900 comp rx=0 tx=0).stop 2024-03-30T00:34:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.651+0000 7f0a921dfb80 1 -- 172.21.15.99:0/443631442 shutdown_connections 2024-03-30T00:34:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.651+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/443631442 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41463f80 0x55aa41473b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.651+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/443631442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa41462780 0x55aa4146b8b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.651+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/443631442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa414630e0 0x55aa4146bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.651+0000 7f0a921dfb80 1 -- 172.21.15.99:0/443631442 >> 172.21.15.99:0/443631442 conn(0x55aa413c1650 msgr2=0x55aa4146a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.651+0000 7f0a921dfb80 1 -- 172.21.15.99:0/443631442 shutdown_connections 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.651+0000 7f0a921dfb80 1 -- 172.21.15.99:0/443631442 wait complete. 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.651+0000 7f0a921dfb80 1 -- start start 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a921dfb80 1 Processor -- start 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41462780 0x55aa414708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 0x55aa41470de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a921dfb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41463f80 0x55aa414cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41471320 con 0x55aa414630e0 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa41471460 con 0x55aa41462780 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a921dfb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa414cc6c0 con 0x55aa41463f80 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a921dfb80 10 client.0 _ll_register_callbacks cb 0x55aa41359420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a76a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 0x55aa41470de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a76a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 0x55aa41470de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38732/0 (socket says 172.21.15.99:38732) 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a76a00700 1 -- 172.21.15.99:0/2484738563 learned_addr learned my addr 172.21.15.99:0/2484738563 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a77400700 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41462780 0x55aa414708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.652+0000 7f0a77e00700 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41463f80 0x55aa414cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a76a00700 1 -- 172.21.15.99:0/2484738563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41463f80 msgr2=0x55aa414cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a76a00700 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41463f80 0x55aa414cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a76a00700 1 -- 172.21.15.99:0/2484738563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41462780 msgr2=0x55aa414708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a76a00700 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41462780 0x55aa414708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a76a00700 1 -- 172.21.15.99:0/2484738563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a781a13c0 con 0x55aa414630e0 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a76a00700 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 0x55aa41470de0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0a640098e0 tx=0x7f0a64003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcbeadb2221fd2e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a61600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a61600700 1 -- 172.21.15.99:0/2484738563 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6402f040 con 0x55aa414630e0 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a61600700 1 -- 172.21.15.99:0/2484738563 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f0a64034470 con 0x55aa414630e0 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2484738563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a7009dd20 con 0x55aa414630e0 2024-03-30T00:34:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a61600700 1 -- 172.21.15.99:0/2484738563 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a64003de0 con 0x55aa414630e0 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a921dfb80 10 client.15309 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.653+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2484738563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55aa41379e50 con 0x55aa414630e0 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.654+0000 7f0a61600700 1 -- 172.21.15.99:0/2484738563 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a7009dd20 con 0x55aa414630e0 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.654+0000 7f0a61600700 1 -- 172.21.15.99:0/2484738563 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0a64029470 con 0x55aa414630e0 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.654+0000 7f0a921dfb80 10 client.15309 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.654+0000 7f0a921dfb80 10 client.15309 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.654+0000 7f0a921dfb80 20 client.15309 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.654+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55aa414d13f0 0x55aa414cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.654+0000 7f0a921dfb80 4 client.15309 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.654+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2484738563 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aa41202ca0 con 0x55aa414d13f0 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.654+0000 7f0a77400700 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55aa414d13f0 0x55aa414cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.654+0000 7f0a61600700 10 client.15309 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:34:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.654+0000 7f0a77400700 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55aa414d13f0 0x55aa414cdd80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d91c324e2fe82562 server_cookie=91b23a2bd14bb5b6 in_seq=0 out_seq=0 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a61600700 1 -- 172.21.15.99:0/2484738563 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55aa41202ca0 con 0x55aa414d13f0 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a61600700 10 client.15309 handle_command_reply: tid=0 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 1 client.15309 shutdown 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2484738563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 msgr2=0x55aa41470de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 0x55aa41470de0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0a640098e0 tx=0x7f0a64003740 comp rx=0 tx=0).stop 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2484738563 shutdown_connections 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55aa414d13f0 0x55aa414cdd80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55aa41463f80 0x55aa414cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa41462780 0x55aa414708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 1 --2- 172.21.15.99:0/2484738563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa414630e0 0x55aa41470de0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a61600700 0 client.15309 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2484738563 >> 172.21.15.99:0/2484738563 conn(0x55aa413c1650 msgr2=0x55aa4146b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2484738563 shutdown_connections 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 1 -- 172.21.15.99:0/2484738563 wait complete. 2024-03-30T00:34:30.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.655+0000 7f0a921dfb80 20 client.15309 trim_cache size 0 max 16384 2024-03-30T00:34:30.668 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:34:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "03801b1a-ee2c-11ee-b647-cb9ed24678a4", 2024-03-30T00:34:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 1, 2024-03-30T00:34:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24559, 2024-03-30T00:34:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-30T00:34:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-30T00:34:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-30T00:34:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 71.535504552999996, 2024-03-30T00:34:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2024-03-30T00:34:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 85, 2024-03-30T00:34:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T00:34:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 84.458229278999994 2024-03-30T00:34:30.670 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 -- 172.21.15.99:0/3900827942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0a5c0a1230 msgr2=0x7f0a5c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0a5c0a1230 0x7f0a5c0a36f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009990 tx=0x7f0a64000bc0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 -- 172.21.15.99:0/3900827942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78102910 msgr2=0x7f0a781a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78102910 0x7f0a781a0370 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0a70007d70 tx=0x7f0a7000a340 comp rx=0 tx=0).stop 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 -- 172.21.15.99:0/3900827942 shutdown_connections 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78102910 0x7f0a781a0370 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0a5c0a1230 0x7f0a5c0a36f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a78100d80 0x7f0a78198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 --2- 172.21.15.99:0/3900827942 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a78101f70 0x7f0a78199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 -- 172.21.15.99:0/3900827942 >> 172.21.15.99:0/3900827942 conn(0x7f0a780fc2f0 msgr2=0x7f0a78107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 -- 172.21.15.99:0/3900827942 shutdown_connections 2024-03-30T00:34:30.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.656+0000 7f0a7e600700 1 -- 172.21.15.99:0/3900827942 wait complete. 2024-03-30T00:34:30.681 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-30T00:34:30.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:30 smithi161 ceph-mon[42221]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:34:30.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:30 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1194669218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:30.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:30 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/224847653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:30.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:30 smithi142 ceph-mon[41779]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:34:30.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:30 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1194669218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:30.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:30 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/224847653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:30.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.868+0000 7f1ce2600700 1 -- 172.21.15.99:0/326798932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cdc101f70 msgr2=0x7f1cdc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.868+0000 7f1ce2600700 1 --2- 172.21.15.99:0/326798932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cdc101f70 0x7f1cdc1023d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80099c0 tx=0x7f1cc8009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.868+0000 7f1ce2600700 1 -- 172.21.15.99:0/326798932 shutdown_connections 2024-03-30T00:34:30.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.868+0000 7f1ce2600700 1 --2- 172.21.15.99:0/326798932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cdc102910 0x7f1cdc1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.868+0000 7f1ce2600700 1 --2- 172.21.15.99:0/326798932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cdc100d80 0x7f1cdc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.868+0000 7f1ce2600700 1 --2- 172.21.15.99:0/326798932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cdc101f70 0x7f1cdc1023d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.868+0000 7f1ce2600700 1 -- 172.21.15.99:0/326798932 >> 172.21.15.99:0/326798932 conn(0x7f1cdc0fc2f0 msgr2=0x7f1cdc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.868+0000 7f1ce2600700 1 -- 172.21.15.99:0/326798932 shutdown_connections 2024-03-30T00:34:30.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.868+0000 7f1ce2600700 1 -- 172.21.15.99:0/326798932 wait complete. 2024-03-30T00:34:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1ce2600700 1 Processor -- start 2024-03-30T00:34:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1ce2600700 1 -- start start 2024-03-30T00:34:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1ce2600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cdc100d80 0x7f1cdc198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1ce2600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cdc101f70 0x7f1cdc199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1ce2600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cdc102910 0x7f1cdc1a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1ce2600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc1a0960 con 0x7f1cdc101f70 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1ce2600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc1a0aa0 con 0x7f1cdc102910 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1ce2600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc1a0c30 con 0x7f1cdc100d80 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1cdaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cdc101f70 0x7f1cdc199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1cdaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cdc101f70 0x7f1cdc199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38764/0 (socket says 172.21.15.99:38764) 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1cdaa00700 1 -- 172.21.15.99:0/1183794856 learned_addr learned my addr 172.21.15.99:0/1183794856 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.869+0000 7f1cdb400700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cdc100d80 0x7f1cdc198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.870+0000 7f1cdbe00700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cdc102910 0x7f1cdc1a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.870+0000 7f1cdaa00700 1 -- 172.21.15.99:0/1183794856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cdc100d80 msgr2=0x7f1cdc198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.870+0000 7f1cdaa00700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cdc100d80 0x7f1cdc198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.870+0000 7f1cdaa00700 1 -- 172.21.15.99:0/1183794856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cdc102910 msgr2=0x7f1cdc1a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.870+0000 7f1cdaa00700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cdc102910 0x7f1cdc1a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.870+0000 7f1cdaa00700 1 -- 172.21.15.99:0/1183794856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80095f0 con 0x7f1cdc101f70 2024-03-30T00:34:30.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.870+0000 7f1cdaa00700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cdc101f70 0x7f1cdc199370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8000c00 tx=0x7f1cc8003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.870+0000 7f1cd3e00700 1 -- 172.21.15.99:0/1183794856 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc801d070 con 0x7f1cdc101f70 2024-03-30T00:34:30.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.870+0000 7f1cd3e00700 1 -- 172.21.15.99:0/1183794856 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1cc800bd30 con 0x7f1cdc101f70 2024-03-30T00:34:30.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.870+0000 7f1ce2600700 1 -- 172.21.15.99:0/1183794856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc1a0eb0 con 0x7f1cdc101f70 2024-03-30T00:34:30.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.870+0000 7f1cd3e00700 1 -- 172.21.15.99:0/1183794856 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8022d20 con 0x7f1cdc101f70 2024-03-30T00:34:30.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.871+0000 7f1ce2600700 1 -- 172.21.15.99:0/1183794856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc1a1270 con 0x7f1cdc101f70 2024-03-30T00:34:30.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.874+0000 7f1cd3e00700 1 -- 172.21.15.99:0/1183794856 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1cc8003f40 con 0x7f1cdc101f70 2024-03-30T00:34:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.875+0000 7f1cd3e00700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1cc00a11b0 0x7f1cc00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.875+0000 7f1cf60e8b80 1 Processor -- start 2024-03-30T00:34:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.875+0000 7f1cd3e00700 1 -- 172.21.15.99:0/1183794856 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1cc80cdff0 con 0x7f1cdc101f70 2024-03-30T00:34:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.875+0000 7f1cdb400700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1cc00a11b0 0x7f1cc00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.875+0000 7f1cf60e8b80 1 -- start start 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.875+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ec9780 0x55e7f2ec9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.875+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2eca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.875+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 0x55e7f2ecb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.875+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2e21940 con 0x55e7f2eca0e0 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.875+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2e1f300 con 0x55e7f2ecaf80 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.875+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2e1abf0 con 0x55e7f2ec9780 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2eca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2eca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38772/0 (socket says 172.21.15.99:38772) 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdaa00700 1 -- 172.21.15.99:0/4251865996 learned_addr learned my addr 172.21.15.99:0/4251865996 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdb400700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1cc00a11b0 0x7f1cc00a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc101dd0 tx=0x7f1ccc007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdbe00700 1 --2- 172.21.15.99:0/4251865996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 0x55e7f2ecb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdb400700 1 --2- 172.21.15.99:0/4251865996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ec9780 0x55e7f2ec9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdaa00700 1 -- 172.21.15.99:0/4251865996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ec9780 msgr2=0x55e7f2ec9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdaa00700 1 --2- 172.21.15.99:0/4251865996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ec9780 0x55e7f2ec9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdaa00700 1 -- 172.21.15.99:0/4251865996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 msgr2=0x55e7f2ecb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdaa00700 1 --2- 172.21.15.99:0/4251865996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 0x55e7f2ecb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdaa00700 1 -- 172.21.15.99:0/4251865996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7f2c9c360 con 0x55e7f2eca0e0 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdbe00700 1 --2- 172.21.15.99:0/4251865996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 0x55e7f2ecb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdb400700 1 --2- 172.21.15.99:0/4251865996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ec9780 0x55e7f2ec9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cdaa00700 1 --2- 172.21.15.99:0/4251865996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2eca9a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1cc809b040 tx=0x7f1cc800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eeea89691446f38 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.876+0000 7f1cd0c00700 1 -- 172.21.15.99:0/4251865996 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc809ba40 con 0x55e7f2eca0e0 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.877+0000 7f1cd0c00700 1 -- 172.21.15.99:0/4251865996 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1cc809b340 con 0x55e7f2eca0e0 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.877+0000 7f1cd0c00700 1 -- 172.21.15.99:0/4251865996 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc809aa90 con 0x55e7f2eca0e0 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.877+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/4251865996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 msgr2=0x55e7f2eca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.877+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/4251865996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2eca9a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1cc809b040 tx=0x7f1cc800bf90 comp rx=0 tx=0).stop 2024-03-30T00:34:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.877+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/4251865996 shutdown_connections 2024-03-30T00:34:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.877+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/4251865996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 0x55e7f2ecb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.877+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/4251865996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ec9780 0x55e7f2ec9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.877+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/4251865996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2eca9a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.877+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/4251865996 >> 172.21.15.99:0/4251865996 conn(0x55e7f2e28650 msgr2=0x55e7f2ec4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.877+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/4251865996 shutdown_connections 2024-03-30T00:34:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.877+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/4251865996 wait complete. 2024-03-30T00:34:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.878+0000 7f1cf60e8b80 1 -- start start 2024-03-30T00:34:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.878+0000 7f1cf60e8b80 1 Processor -- start 2024-03-30T00:34:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ec9780 0x55e7f2ed3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ed3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ecaf80 0x55e7f2eda7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2db6d30 con 0x55e7f2eca0e0 2024-03-30T00:34:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2db69d0 con 0x55e7f2ec9780 2024-03-30T00:34:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2c9e650 con 0x55e7f2ecaf80 2024-03-30T00:34:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cf60e8b80 10 client.0 _ll_register_callbacks cb 0x55e7f2dc0420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cdaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ed3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cdaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ed3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38774/0 (socket says 172.21.15.99:38774) 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cdaa00700 1 -- 172.21.15.99:0/3277118705 learned_addr learned my addr 172.21.15.99:0/3277118705 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cdb400700 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ec9780 0x55e7f2ed3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.879+0000 7f1cdbe00700 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ecaf80 0x55e7f2eda7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cdaa00700 1 -- 172.21.15.99:0/3277118705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ecaf80 msgr2=0x55e7f2eda7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cdaa00700 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ecaf80 0x55e7f2eda7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cdaa00700 1 -- 172.21.15.99:0/3277118705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ec9780 msgr2=0x55e7f2ed3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cdaa00700 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ec9780 0x55e7f2ed3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cdaa00700 1 -- 172.21.15.99:0/3277118705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc1a1270 con 0x55e7f2eca0e0 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cdb400700 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ec9780 0x55e7f2ed3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cdaa00700 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ed3790 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009d10 tx=0x7f1cc809add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d1f46da6c8c430 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cc5600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cc5600700 1 -- 172.21.15.99:0/3277118705 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc809ba40 con 0x55e7f2eca0e0 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cc5600700 1 -- 172.21.15.99:0/3277118705 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1cc809f170 con 0x55e7f2eca0e0 2024-03-30T00:34:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/3277118705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7f2ddb330 con 0x55e7f2eca0e0 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cc5600700 1 -- 172.21.15.99:0/3277118705 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cc8022680 con 0x55e7f2eca0e0 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cf60e8b80 10 client.15321 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.880+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/3277118705 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e7f2c48b10 con 0x55e7f2eca0e0 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.881+0000 7f1cc5600700 1 -- 172.21.15.99:0/3277118705 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e7f2ddb330 con 0x55e7f2eca0e0 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.881+0000 7f1cc5600700 1 -- 172.21.15.99:0/3277118705 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1cc809a2a0 con 0x55e7f2eca0e0 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.881+0000 7f1cf60e8b80 10 client.15321 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.881+0000 7f1cf60e8b80 10 client.15321 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.881+0000 7f1cf60e8b80 20 client.15321 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.881+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7f2eddee0 0x55e7f2ee31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.881+0000 7f1cf60e8b80 4 client.15321 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.881+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/3277118705 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7f2e136b0 con 0x55e7f2eddee0 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.881+0000 7f1cdb400700 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7f2eddee0 0x55e7f2ee31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.882+0000 7f1cdb400700 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7f2eddee0 0x55e7f2ee31a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daace4012eaa1dbb server_cookie=12c3384c523dc083 in_seq=0 out_seq=0 2024-03-30T00:34:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.882+0000 7f1cc5600700 10 client.15321 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cc5600700 1 -- 172.21.15.99:0/3277118705 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e7f2e136b0 con 0x55e7f2eddee0 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cc5600700 10 client.15321 handle_command_reply: tid=0 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cf60e8b80 1 client.15321 shutdown 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/3277118705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 msgr2=0x55e7f2ed3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ed3790 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009d10 tx=0x7f1cc809add0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/3277118705 shutdown_connections 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7f2eddee0 0x55e7f2ee31a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ecaf80 0x55e7f2eda7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ec9780 0x55e7f2ed3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/3277118705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ed3790 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cc5600700 0 client.15321 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/3277118705 >> 172.21.15.99:0/3277118705 conn(0x55e7f2e28650 msgr2=0x55e7f2ec8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/3277118705 shutdown_connections 2024-03-30T00:34:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.883+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/3277118705 wait complete. 2024-03-30T00:34:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.884+0000 7f1cf60e8b80 20 client.15321 trim_cache size 0 max 16384 2024-03-30T00:34:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.901+0000 7f1cf60e8b80 1 Processor -- start 2024-03-30T00:34:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.901+0000 7f1cf60e8b80 1 -- start start 2024-03-30T00:34:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ee78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ec9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ecaf80 0x55e7f2ed2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2ec5610 con 0x55e7f2ec9780 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2ec4ed0 con 0x55e7f2eca0e0 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2e136e0 con 0x55e7f2ecaf80 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cdb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ee78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cdb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ee78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38782/0 (socket says 172.21.15.99:38782) 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cdb400700 1 -- 172.21.15.99:0/2295791174 learned_addr learned my addr 172.21.15.99:0/2295791174 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cdbe00700 1 --2- 172.21.15.99:0/2295791174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ecaf80 0x55e7f2ed2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cdb400700 1 -- 172.21.15.99:0/2295791174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ecaf80 msgr2=0x55e7f2ed2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cdb400700 1 --2- 172.21.15.99:0/2295791174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ecaf80 0x55e7f2ed2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cdb400700 1 -- 172.21.15.99:0/2295791174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2eca0e0 msgr2=0x55e7f2ec9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cdb400700 1 --2- 172.21.15.99:0/2295791174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ec9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cdb400700 1 -- 172.21.15.99:0/2295791174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7f2de0e50 con 0x55e7f2ec9780 2024-03-30T00:34:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.902+0000 7f1cdbe00700 1 --2- 172.21.15.99:0/2295791174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ecaf80 0x55e7f2ed2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cdb400700 1 --2- 172.21.15.99:0/2295791174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ee78d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55e7f2db6a90 tx=0x7f1ccc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d26c883a5e489b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cd1600700 1 -- 172.21.15.99:0/2295791174 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc011ea0 con 0x55e7f2ec9780 2024-03-30T00:34:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cd1600700 1 -- 172.21.15.99:0/2295791174 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ccc025040 con 0x55e7f2ec9780 2024-03-30T00:34:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cd1600700 1 -- 172.21.15.99:0/2295791174 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc01e680 con 0x55e7f2ec9780 2024-03-30T00:34:30.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/2295791174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 msgr2=0x55e7f2ee78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/2295791174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ee78d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55e7f2db6a90 tx=0x7f1ccc013000 comp rx=0 tx=0).stop 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/2295791174 shutdown_connections 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/2295791174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2ecaf80 0x55e7f2ed2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/2295791174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ee78d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/2295791174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ec9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/2295791174 >> 172.21.15.99:0/2295791174 conn(0x55e7f2e28650 msgr2=0x55e7f2ee71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.903+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/2295791174 shutdown_connections 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.904+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/2295791174 wait complete. 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.904+0000 7f1cf60e8b80 1 -- start start 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.904+0000 7f1cf60e8b80 1 Processor -- start 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.904+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ed74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.904+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ed7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.904+0000 7f1cf60e8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 0x55e7f2f32790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.904+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2eda980 con 0x55e7f2ec9780 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.904+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2edaaf0 con 0x55e7f2ecaf80 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.904+0000 7f1cf60e8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f2f32cd0 con 0x55e7f2eca0e0 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cf60e8b80 10 client.0 _ll_register_callbacks cb 0x55e7f2dc0420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cdb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ed74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cdb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ed74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38798/0 (socket says 172.21.15.99:38798) 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cdb400700 1 -- 172.21.15.99:0/979183027 learned_addr learned my addr 172.21.15.99:0/979183027 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cdbe00700 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 0x55e7f2f32790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cdb400700 1 -- 172.21.15.99:0/979183027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2eca0e0 msgr2=0x55e7f2ed7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cdb400700 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ed7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cdb400700 1 -- 172.21.15.99:0/979183027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 msgr2=0x55e7f2f32790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cdb400700 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 0x55e7f2f32790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cdb400700 1 -- 172.21.15.99:0/979183027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ccc010c40 con 0x55e7f2ec9780 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cdbe00700 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 0x55e7f2f32790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cdb400700 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ed74c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc026c50 tx=0x7f1ccc0108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a39fddd020ad9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cc5600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cc5600700 1 -- 172.21.15.99:0/979183027 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc004220 con 0x55e7f2ec9780 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cc5600700 1 -- 172.21.15.99:0/979183027 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1ccc017070 con 0x55e7f2ec9780 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.905+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/979183027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cc809b340 con 0x55e7f2ec9780 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.906+0000 7f1cc5600700 1 -- 172.21.15.99:0/979183027 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ccc004380 con 0x55e7f2ec9780 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.906+0000 7f1cf60e8b80 10 client.15327 fetch_fsmap learned FSMap version 23 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.906+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/979183027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e7f2ca0c80 con 0x55e7f2ec9780 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.906+0000 7f1cc5600700 1 -- 172.21.15.99:0/979183027 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1cc809b340 con 0x55e7f2ec9780 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.906+0000 7f1cc5600700 1 -- 172.21.15.99:0/979183027 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1ccc03f070 con 0x55e7f2ec9780 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.906+0000 7f1cf60e8b80 10 client.15327 fetch_fsmap finished waiting for FSMap version 23 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.906+0000 7f1cf60e8b80 10 client.15327 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.906+0000 7f1cf60e8b80 20 client.15327 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.906+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7f2f37910 0x55e7f2f34240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.906+0000 7f1cf60e8b80 4 client.15327 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T00:34:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.906+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/979183027 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e7f2c69ca0 con 0x55e7f2f37910 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.907+0000 7f1cdaa00700 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7f2f37910 0x55e7f2f34240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.907+0000 7f1cc5600700 10 client.15327 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.907+0000 7f1cdaa00700 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7f2f37910 0x55e7f2f34240 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8a42bd9421f9af server_cookie=3e86cbed8d52803c in_seq=0 out_seq=0 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.907+0000 7f1cc5600700 1 -- 172.21.15.99:0/979183027 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e7f2c69ca0 con 0x55e7f2f37910 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.907+0000 7f1cc5600700 10 client.15327 handle_command_reply: tid=0 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.907+0000 7f1cf60e8b80 1 client.15327 shutdown 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/979183027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 msgr2=0x55e7f2ed74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ed74c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc026c50 tx=0x7f1ccc0108a0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/979183027 shutdown_connections 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7f2f37910 0x55e7f2f34240 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7f2ecaf80 0x55e7f2f32790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7f2ec9780 0x55e7f2ed74c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cf60e8b80 1 --2- 172.21.15.99:0/979183027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7f2eca0e0 0x55e7f2ed7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cc5600700 0 client.15327 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/979183027 >> 172.21.15.99:0/979183027 conn(0x55e7f2e28650 msgr2=0x55e7f2ec8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/979183027 shutdown_connections 2024-03-30T00:34:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cf60e8b80 1 -- 172.21.15.99:0/979183027 wait complete. 2024-03-30T00:34:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1cf60e8b80 20 client.15327 trim_cache size 0 max 16384 2024-03-30T00:34:30.919 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:34:30.919 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "03801b1a-ee2c-11ee-b647-cb9ed24678a4", 2024-03-30T00:34:30.919 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-03-30T00:34:30.919 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 24572, 2024-03-30T00:34:30.919 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-30T00:34:30.919 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-30T00:34:30.920 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-30T00:34:30.920 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 75.255203503000004, 2024-03-30T00:34:30.920 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 23, 2024-03-30T00:34:30.920 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 85, 2024-03-30T00:34:30.920 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T00:34:30.920 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 82.313767695999999 2024-03-30T00:34:30.920 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1ce2600700 1 -- 172.21.15.99:0/1183794856 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1cc00a11b0 msgr2=0x7f1cc00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1ce2600700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1cc00a11b0 0x7f1cc00a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc101dd0 tx=0x7f1ccc007400 comp rx=0 tx=0).stop 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1ce2600700 1 -- 172.21.15.99:0/1183794856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cdc101f70 msgr2=0x7f1cdc199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.908+0000 7f1ce2600700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cdc101f70 0x7f1cdc199370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8000c00 tx=0x7f1cc8003a50 comp rx=0 tx=0).stop 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.909+0000 7f1ce2600700 1 -- 172.21.15.99:0/1183794856 shutdown_connections 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.909+0000 7f1ce2600700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1cc00a11b0 0x7f1cc00a3670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.909+0000 7f1ce2600700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cdc102910 0x7f1cdc1a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.909+0000 7f1ce2600700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1cdc100d80 0x7f1cdc198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.909+0000 7f1ce2600700 1 --2- 172.21.15.99:0/1183794856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cdc101f70 0x7f1cdc199370 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.909+0000 7f1ce2600700 1 -- 172.21.15.99:0/1183794856 >> 172.21.15.99:0/1183794856 conn(0x7f1cdc0fc2f0 msgr2=0x7f1cdc107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.909+0000 7f1ce2600700 1 -- 172.21.15.99:0/1183794856 shutdown_connections 2024-03-30T00:34:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:30.909+0000 7f1ce2600700 1 -- 172.21.15.99:0/1183794856 wait complete. 2024-03-30T00:34:30.934 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-30T00:34:30.957 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-30T00:34:30.957 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-30T00:34:30.958 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-30T00:34:30.958 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T00:34:30.958 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:34:30.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:30.958 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-30T00:34:31.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:31.001 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-03-30T00:34:31.110 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2024-03-30T00:34:31.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:34:31.113 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-30T00:34:31.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-03-30T00:34:31.470 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:34:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.596+0000 7fac7f600700 1 -- 172.21.15.99:0/3752983051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac78106ef0 msgr2=0x7fac781072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.596+0000 7fac7f600700 1 --2- 172.21.15.99:0/3752983051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac78106ef0 0x7fac781072d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0099c0 tx=0x7fac6c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.597+0000 7fac7f600700 1 -- 172.21.15.99:0/3752983051 shutdown_connections 2024-03-30T00:34:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.597+0000 7fac7f600700 1 --2- 172.21.15.99:0/3752983051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78101820 0x7fac7810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.597+0000 7fac7f600700 1 --2- 172.21.15.99:0/3752983051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac78106ef0 0x7fac781072d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.597+0000 7fac7f600700 1 --2- 172.21.15.99:0/3752983051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac78100e60 0x7fac781012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.597+0000 7fac7f600700 1 -- 172.21.15.99:0/3752983051 >> 172.21.15.99:0/3752983051 conn(0x7fac78077fa0 msgr2=0x7fac780783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.597+0000 7fac7f600700 1 -- 172.21.15.99:0/3752983051 shutdown_connections 2024-03-30T00:34:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.597+0000 7fac7f600700 1 -- 172.21.15.99:0/3752983051 wait complete. 2024-03-30T00:34:32.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac7f600700 1 Processor -- start 2024-03-30T00:34:32.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac7f600700 1 -- start start 2024-03-30T00:34:32.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac7f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78100e60 0x7fac78199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac7f600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac78101820 0x7fac78199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac7f600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac78106ef0 0x7fac7819dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac7f600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7819e250 con 0x7fac78101820 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac7f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7819e3c0 con 0x7fac78100e60 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac7f600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7819e580 con 0x7fac78106ef0 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac77e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac78101820 0x7fac78199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac77e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac78101820 0x7fac78199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38802/0 (socket says 172.21.15.99:38802) 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac77e00700 1 -- 172.21.15.99:0/1313590818 learned_addr learned my addr 172.21.15.99:0/1313590818 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.598+0000 7fac7cc00700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78100e60 0x7fac78199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac7d600700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac78106ef0 0x7fac7819dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac77e00700 1 -- 172.21.15.99:0/1313590818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac78106ef0 msgr2=0x7fac7819dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac77e00700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac78106ef0 0x7fac7819dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac77e00700 1 -- 172.21.15.99:0/1313590818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78100e60 msgr2=0x7fac78199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:32.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac77e00700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78100e60 0x7fac78199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:32.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac77e00700 1 -- 172.21.15.99:0/1313590818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c0095f0 con 0x7fac78101820 2024-03-30T00:34:32.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac7d600700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac78106ef0 0x7fac7819dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:32.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac77e00700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac78101820 0x7fac78199890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac6400d950 tx=0x7fac6400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:32.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac75600700 1 -- 172.21.15.99:0/1313590818 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64009940 con 0x7fac78101820 2024-03-30T00:34:32.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac75600700 1 -- 172.21.15.99:0/1313590818 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac64010460 con 0x7fac78101820 2024-03-30T00:34:32.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac75600700 1 -- 172.21.15.99:0/1313590818 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6400f620 con 0x7fac78101820 2024-03-30T00:34:32.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.599+0000 7fac7f600700 1 -- 172.21.15.99:0/1313590818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7819e860 con 0x7fac78101820 2024-03-30T00:34:32.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.600+0000 7fac7f600700 1 -- 172.21.15.99:0/1313590818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac781a60c0 con 0x7fac78101820 2024-03-30T00:34:32.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.601+0000 7fac7f600700 1 -- 172.21.15.99:0/1313590818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac78104180 con 0x7fac78101820 2024-03-30T00:34:32.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.602+0000 7fac75600700 1 -- 172.21.15.99:0/1313590818 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fac640105d0 con 0x7fac78101820 2024-03-30T00:34:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.604+0000 7fac75600700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fac600a11b0 0x7fac600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.604+0000 7fac75600700 1 -- 172.21.15.99:0/1313590818 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fac640cc940 con 0x7fac78101820 2024-03-30T00:34:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.604+0000 7fac7cc00700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fac600a11b0 0x7fac600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.604+0000 7fac7cc00700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fac600a11b0 0x7fac600a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:32.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.608+0000 7fac75600700 1 -- 172.21.15.99:0/1313590818 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fac64095f00 con 0x7fac78101820 2024-03-30T00:34:32.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:32 smithi161 ceph-mon[42221]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:34:32.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:32 smithi099 ceph-mon[38079]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:34:32.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:32 smithi142 ceph-mon[41779]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:34:32.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.854+0000 7fac7f600700 1 -- 172.21.15.99:0/1313590818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fac781024f0 con 0x7fac78101820 2024-03-30T00:34:32.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.854+0000 7fac75600700 1 -- 172.21.15.99:0/1313590818 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fac64016070 con 0x7fac78101820 2024-03-30T00:34:32.861 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 -- 172.21.15.99:0/1313590818 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fac600a11b0 msgr2=0x7fac600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fac600a11b0 0x7fac600a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fac6c000c00 tx=0x7fac6c009f10 comp rx=0 tx=0).stop 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 -- 172.21.15.99:0/1313590818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac78101820 msgr2=0x7fac78199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac78101820 0x7fac78199890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac6400d950 tx=0x7fac6400dc60 comp rx=0 tx=0).stop 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 -- 172.21.15.99:0/1313590818 shutdown_connections 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac78106ef0 0x7fac7819dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fac600a11b0 0x7fac600a3670 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78100e60 0x7fac78199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 --2- 172.21.15.99:0/1313590818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac78101820 0x7fac78199890 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 -- 172.21.15.99:0/1313590818 >> 172.21.15.99:0/1313590818 conn(0x7fac78077fa0 msgr2=0x7fac7810c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 -- 172.21.15.99:0/1313590818 shutdown_connections 2024-03-30T00:34:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:32.857+0000 7fac7f600700 1 -- 172.21.15.99:0/1313590818 wait complete. 2024-03-30T00:34:33.250 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T00:34:33.250 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T00:34:33.250 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-03-30T00:34:33.250 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T00:34:33.250 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T00:34:33.250 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T00:34:33.250 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T00:34:33.250 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T00:34:33.250 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:34:33.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:33.251 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-03-30T00:34:33.294 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T00:34:33.294 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T00:34:33.294 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 41770sec preferred_lft 41770sec 2024-03-30T00:34:33.295 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-30T00:34:33.296 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-30T00:34:33.296 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T00:34:33.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:33.296 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-30T00:34:33.296 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-30T00:34:33.296 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add name ceph-brx type bridge 2024-03-30T00:34:33.296 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr flush dev ceph-brx 2024-03-30T00:34:33.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set ceph-brx up 2024-03-30T00:34:33.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T00:34:33.297 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-30T00:34:33.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:33 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1313590818' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-30T00:34:33.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:33 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1313590818' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-30T00:34:33.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:33.773 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T00:34:33.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:33 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1313590818' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-30T00:34:33.839 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-03-30T00:34:33.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:33.844 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-03-30T00:34:33.903 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-03-30T00:34:33.904 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T00:34:33.904 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T00:34:33.904 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T00:34:33.904 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T00:34:33.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:33.904 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-30T00:34:33.904 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-30T00:34:33.904 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T00:34:33.905 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T00:34:33.905 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T00:34:33.905 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-30T00:34:34.576 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:34 smithi099 ceph-mon[38079]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-30T00:34:34.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:34.579 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-30T00:34:34.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:34.622 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2024-03-30T00:34:34.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:34.704 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-30T00:34:34.705 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-30T00:34:34.705 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:34:34.705 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-30T00:34:34.705 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-30T00:34:34.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:34 smithi161 ceph-mon[42221]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-30T00:34:34.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:34 smithi142 ceph-mon[41779]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-30T00:34:34.881 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-30T00:34:34.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:34.881 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-30T00:34:34.881 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-30T00:34:34.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-30T00:34:34.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T00:34:34.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-30T00:34:34.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-30T00:34:34.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-30T00:34:34.881 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-30T00:34:35.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:35.156 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-30T00:34:35.156 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-30T00:34:35.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-03-30T00:34:35.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T00:34:35.156 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-30T00:34:35.324 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-30T00:34:35.324 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:34:35.324 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T00:34:35.366 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T00:34:35.367 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T00:34:35.367 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:34:35.448 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-30T00:34:35.555 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:34:35.556 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:34:35.623 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:34:35.626 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:34:35.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:35.626 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-30T00:34:35.668 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T00:34:35.668 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af --client_fs=cephfs) 2024-03-30T00:34:35.713 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-30T00:34:35.777 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:34:35.777 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:34:35.842 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:34:35.845 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:34:35.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:35.846 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-30T00:34:35.891 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-03-30T00:34:35.885+0000 7f643fdfd5c0 -1 init, newargv = 0x55724b9c8050 newargc=15 2024-03-30T00:34:35.891 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[100990]: starting ceph client 2024-03-30T00:34:35.915 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[100990]: starting fuse 2024-03-30T00:34:36.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:36 smithi161 ceph-mon[42221]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:34:36.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:36 smithi099 ceph-mon[38079]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:34:36.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:36 smithi142 ceph-mon[41779]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:34:36.988 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-30T00:34:37.059 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T00:34:37.059 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T00:34:37.123 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T00:34:37.126 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T00:34:37.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:37.127 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-30T00:34:37.168 INFO:teuthology.orchestra.run.smithi099.stdout:122 2024-03-30T00:34:37.168 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [122] 2024-03-30T00:34:37.168 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T00:34:37.168 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-03-30T00:34:37.168 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-03-30T00:34:37.168 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-30T00:34:37.168 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-03-30T00:34:37.168 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-30T00:34:37.168 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-03-30T00:34:37.168 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-03-30T00:34:37.169 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-03-30T00:34:37.170 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-03-30T00:34:37.170 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T00:34:37.170 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-30T00:34:37.170 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-03-30T00:34:37.170 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-30T00:34:37.348 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.100990.asok 2024-03-30T00:34:37.527 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.100990.asok 2024-03-30T00:34:37.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:37.527 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.100990.asok status 2024-03-30T00:34:37.749 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:34:37.749 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-03-30T00:34:37.749 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T00:34:37.749 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T00:34:37.749 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-03-30T00:34:37.749 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-03-30T00:34:37.749 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T00:34:37.749 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "100990", 2024-03-30T00:34:37.749 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af" 2024-03-30T00:34:37.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 15339, 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 15339 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2719172276 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:34:37.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2719172276 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.15339 192.168.0.1:0/2719172276", 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/2719172276", 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 5, 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 23, 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 85, 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 83, 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-03-30T00:34:37.751 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:34:37.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:37.763 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-03-30T00:34:37.764 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:34:37.805 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-03-30T00:34:37.806 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T00:34:37.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:34:37.806 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T00:34:37.914 INFO:teuthology.run_tasks:Running task exec... 2024-03-30T00:34:37.926 INFO:teuthology.task.exec:Executing custom commands... 2024-03-30T00:34:37.926 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-03-30T00:34:37.926 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-30T00:34:38.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.107+0000 7f485ee00700 1 -- 172.21.15.99:0/3410241352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4858101f10 msgr2=0x7f4858102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:38.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.107+0000 7f485ee00700 1 --2- 172.21.15.99:0/3410241352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4858101f10 0x7f4858102370 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f48440099c0 tx=0x7f4844009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:38.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.108+0000 7f485ee00700 1 -- 172.21.15.99:0/3410241352 shutdown_connections 2024-03-30T00:34:38.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.108+0000 7f485ee00700 1 --2- 172.21.15.99:0/3410241352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48581028b0 0x7f4858109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:38.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.108+0000 7f485ee00700 1 --2- 172.21.15.99:0/3410241352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4858100d20 0x7f4858101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:38.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.108+0000 7f485ee00700 1 --2- 172.21.15.99:0/3410241352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4858101f10 0x7f4858102370 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:38.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.108+0000 7f485ee00700 1 -- 172.21.15.99:0/3410241352 >> 172.21.15.99:0/3410241352 conn(0x7f48580fc2b0 msgr2=0x7f48580fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:38.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.108+0000 7f485ee00700 1 -- 172.21.15.99:0/3410241352 shutdown_connections 2024-03-30T00:34:38.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.108+0000 7f485ee00700 1 -- 172.21.15.99:0/3410241352 wait complete. 2024-03-30T00:34:38.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f485ee00700 1 Processor -- start 2024-03-30T00:34:38.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f485ee00700 1 -- start start 2024-03-30T00:34:38.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f485ee00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4858100d20 0x7f4858198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:38.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f485ee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4858101f10 0x7f48581992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:38.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f485ee00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48581028b0 0x7f48581a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:38.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f485ee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48581a0850 con 0x7f4858101f10 2024-03-30T00:34:38.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f485ee00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48581a09c0 con 0x7f4858100d20 2024-03-30T00:34:38.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f485ee00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48581a0b80 con 0x7f48581028b0 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f4857400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4858101f10 0x7f48581992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f4857400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4858101f10 0x7f48581992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52774/0 (socket says 172.21.15.99:52774) 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f4857400700 1 -- 172.21.15.99:0/492046870 learned_addr learned my addr 172.21.15.99:0/492046870 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.109+0000 7f4857e00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4858100d20 0x7f4858198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f485ce00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48581028b0 0x7f48581a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f4857400700 1 -- 172.21.15.99:0/492046870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48581028b0 msgr2=0x7f48581a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f4857400700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48581028b0 0x7f48581a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f4857400700 1 -- 172.21.15.99:0/492046870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4858100d20 msgr2=0x7f4858198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f4857400700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4858100d20 0x7f4858198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f4857400700 1 -- 172.21.15.99:0/492046870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48440095f0 con 0x7f4858101f10 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f4857e00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4858100d20 0x7f4858198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f4857400700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4858101f10 0x7f48581992a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4844000c00 tx=0x7f4844003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f4854c00700 1 -- 172.21.15.99:0/492046870 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484401d070 con 0x7f4858101f10 2024-03-30T00:34:38.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f4854c00700 1 -- 172.21.15.99:0/492046870 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f484400bd30 con 0x7f4858101f10 2024-03-30T00:34:38.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f485ee00700 1 -- 172.21.15.99:0/492046870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48581a0e00 con 0x7f4858101f10 2024-03-30T00:34:38.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f4854c00700 1 -- 172.21.15.99:0/492046870 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4844022dc0 con 0x7f4858101f10 2024-03-30T00:34:38.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.110+0000 7f485ee00700 1 -- 172.21.15.99:0/492046870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48581a11d0 con 0x7f4858101f10 2024-03-30T00:34:38.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.111+0000 7f485ee00700 1 -- 172.21.15.99:0/492046870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4858067c30 con 0x7f4858101f10 2024-03-30T00:34:38.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.113+0000 7f4854c00700 1 -- 172.21.15.99:0/492046870 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4844003ff0 con 0x7f4858101f10 2024-03-30T00:34:38.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.114+0000 7f4854c00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f483c0a11b0 0x7f483c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:38.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.114+0000 7f4854c00700 1 -- 172.21.15.99:0/492046870 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f48440cdfe0 con 0x7f4858101f10 2024-03-30T00:34:38.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.115+0000 7f4857e00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f483c0a11b0 0x7f483c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:38.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.115+0000 7f4857e00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f483c0a11b0 0x7f483c0a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f485819a1e0 tx=0x7f484800a2b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:38.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.118+0000 7f4854c00700 1 -- 172.21.15.99:0/492046870 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4844097650 con 0x7f4858101f10 2024-03-30T00:34:38.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:38.372+0000 7f485ee00700 1 -- 172.21.15.99:0/492046870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f485810c680 con 0x7f4858101f10 2024-03-30T00:34:38.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:38 smithi161 ceph-mon[42221]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-30T00:34:38.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:38 smithi099 ceph-mon[38079]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-30T00:34:38.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:38 smithi142 ceph-mon[41779]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-30T00:34:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.385+0000 7f4854c00700 1 -- 172.21.15.99:0/492046870 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f4844097a20 con 0x7f4858101f10 2024-03-30T00:34:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.386+0000 7f4854c00700 1 -- 172.21.15.99:0/492046870 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v24) v1 ==== 117+0+0 (secure 0 0 0) 0x7f4844077030 con 0x7f4858101f10 2024-03-30T00:34:39.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 -- 172.21.15.99:0/492046870 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f483c0a11b0 msgr2=0x7f483c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f483c0a11b0 0x7f483c0a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f485819a1e0 tx=0x7f484800a2b0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 -- 172.21.15.99:0/492046870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4858101f10 msgr2=0x7f48581992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4858101f10 0x7f48581992a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4844000c00 tx=0x7f4844003b00 comp rx=0 tx=0).stop 2024-03-30T00:34:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 -- 172.21.15.99:0/492046870 shutdown_connections 2024-03-30T00:34:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48581028b0 0x7f48581a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f483c0a11b0 0x7f483c0a3670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4858100d20 0x7f4858198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 --2- 172.21.15.99:0/492046870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4858101f10 0x7f48581992a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 -- 172.21.15.99:0/492046870 >> 172.21.15.99:0/492046870 conn(0x7f48580fc2b0 msgr2=0x7f4858072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 -- 172.21.15.99:0/492046870 shutdown_connections 2024-03-30T00:34:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.389+0000 7f485ee00700 1 -- 172.21.15.99:0/492046870 wait complete. 2024-03-30T00:34:39.414 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-30T00:34:39.452 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-30T00:34:39.473 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:34:39.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.659+0000 7f1b4c600700 1 -- 172.21.15.99:0/3537674295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b44101e70 msgr2=0x7f1b441022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:39.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.659+0000 7f1b4c600700 1 --2- 172.21.15.99:0/3537674295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b44101e70 0x7f1b441022d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1b300099c0 tx=0x7f1b30009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.659+0000 7f1b4c600700 1 -- 172.21.15.99:0/3537674295 shutdown_connections 2024-03-30T00:34:39.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.659+0000 7f1b4c600700 1 --2- 172.21.15.99:0/3537674295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44102810 0x7f1b441090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.659+0000 7f1b4c600700 1 --2- 172.21.15.99:0/3537674295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b44100c80 0x7f1b441010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.659+0000 7f1b4c600700 1 --2- 172.21.15.99:0/3537674295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b44101e70 0x7f1b441022d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.659+0000 7f1b4c600700 1 -- 172.21.15.99:0/3537674295 >> 172.21.15.99:0/3537674295 conn(0x7f1b440fc1f0 msgr2=0x7f1b440fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:39.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.659+0000 7f1b4c600700 1 -- 172.21.15.99:0/3537674295 shutdown_connections 2024-03-30T00:34:39.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.659+0000 7f1b4c600700 1 -- 172.21.15.99:0/3537674295 wait complete. 2024-03-30T00:34:39.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.660+0000 7f1b4c600700 1 Processor -- start 2024-03-30T00:34:39.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.660+0000 7f1b4c600700 1 -- start start 2024-03-30T00:34:39.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4c600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44100c80 0x7f1b44198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:39.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4c600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b44101e70 0x7f1b441992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4c600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b44102810 0x7f1b441a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4c600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b441a0830 con 0x7f1b44102810 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4c600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b441a09a0 con 0x7f1b44101e70 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4c600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b441a0b60 con 0x7f1b44100c80 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4a600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b44102810 0x7f1b441a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4a600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b44102810 0x7f1b441a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52794/0 (socket says 172.21.15.99:52794) 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4a600700 1 -- 172.21.15.99:0/489080948 learned_addr learned my addr 172.21.15.99:0/489080948 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b49c00700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44100c80 0x7f1b44198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b49200700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b44101e70 0x7f1b441992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4a600700 1 -- 172.21.15.99:0/489080948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44100c80 msgr2=0x7f1b44198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4a600700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44100c80 0x7f1b44198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4a600700 1 -- 172.21.15.99:0/489080948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b44101e70 msgr2=0x7f1b441992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:39.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4a600700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b44101e70 0x7f1b441992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.661+0000 7f1b4a600700 1 -- 172.21.15.99:0/489080948 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b300095f0 con 0x7f1b44102810 2024-03-30T00:34:39.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.662+0000 7f1b4a600700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b44102810 0x7f1b441a0260 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c007ee0 tx=0x7f1b3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:39.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.662+0000 7f1b42000700 1 -- 172.21.15.99:0/489080948 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c017070 con 0x7f1b44102810 2024-03-30T00:34:39.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.662+0000 7f1b42000700 1 -- 172.21.15.99:0/489080948 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b3c00f040 con 0x7f1b44102810 2024-03-30T00:34:39.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.662+0000 7f1b42000700 1 -- 172.21.15.99:0/489080948 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c005570 con 0x7f1b44102810 2024-03-30T00:34:39.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.662+0000 7f1b4c600700 1 -- 172.21.15.99:0/489080948 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b441a0e40 con 0x7f1b44102810 2024-03-30T00:34:39.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.662+0000 7f1b4c600700 1 -- 172.21.15.99:0/489080948 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b441a12c0 con 0x7f1b44102810 2024-03-30T00:34:39.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.663+0000 7f1b4c600700 1 -- 172.21.15.99:0/489080948 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b44067a10 con 0x7f1b44102810 2024-03-30T00:34:39.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.665+0000 7f1b42000700 1 -- 172.21.15.99:0/489080948 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1b3c00db20 con 0x7f1b44102810 2024-03-30T00:34:39.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.666+0000 7f1b42000700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1b2c0a11a0 0x7f1b2c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:39.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.666+0000 7f1b42000700 1 -- 172.21.15.99:0/489080948 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f1b3c0cbff0 con 0x7f1b44102810 2024-03-30T00:34:39.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.666+0000 7f1b49c00700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1b2c0a11a0 0x7f1b2c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:39.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.667+0000 7f1b49c00700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1b2c0a11a0 0x7f1b2c0a3660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1b440fe520 tx=0x7f1b3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:39.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.670+0000 7f1b42000700 1 -- 172.21.15.99:0/489080948 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1b3c095710 con 0x7f1b44102810 2024-03-30T00:34:39.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:39 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/492046870' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-30T00:34:39.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:39 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/492046870' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-30T00:34:39.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:39 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/492046870' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-30T00:34:39.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.912+0000 7f1b4c600700 1 -- 172.21.15.99:0/489080948 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b4410c390 con 0x7f1b44102810 2024-03-30T00:34:39.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.914+0000 7f1b42000700 1 -- 172.21.15.99:0/489080948 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22615 (secure 0 0 0) 0x7f1b3c098d30 con 0x7f1b44102810 2024-03-30T00:34:39.921 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:39.921 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:34:10.248126+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8},{"from":5,"to":7}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:34:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 -- 172.21.15.99:0/489080948 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1b2c0a11a0 msgr2=0x7f1b2c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1b2c0a11a0 0x7f1b2c0a3660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1b440fe520 tx=0x7f1b3800b410 comp rx=0 tx=0).stop 2024-03-30T00:34:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 -- 172.21.15.99:0/489080948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b44102810 msgr2=0x7f1b441a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b44102810 0x7f1b441a0260 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c007ee0 tx=0x7f1b3c00d3b0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 -- 172.21.15.99:0/489080948 shutdown_connections 2024-03-30T00:34:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b44102810 0x7f1b441a0260 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b44100c80 0x7f1b44198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f1b2c0a11a0 0x7f1b2c0a3660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 --2- 172.21.15.99:0/489080948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b44101e70 0x7f1b441992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:39.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 -- 172.21.15.99:0/489080948 >> 172.21.15.99:0/489080948 conn(0x7f1b440fc1f0 msgr2=0x7f1b44107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:39.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 -- 172.21.15.99:0/489080948 shutdown_connections 2024-03-30T00:34:39.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:39.916+0000 7f1b4c600700 1 -- 172.21.15.99:0/489080948 wait complete. 2024-03-30T00:34:39.942 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:33:14.085184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:33:15.090699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:34:39.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:34:40.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.129+0000 7fcccb800700 1 -- 172.21.15.99:0/36807699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc4104170 msgr2=0x7fccc4106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:40.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.129+0000 7fcccb800700 1 --2- 172.21.15.99:0/36807699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc4104170 0x7fccc4106560 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009990 tx=0x7fccb0009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.129+0000 7fcccb800700 1 -- 172.21.15.99:0/36807699 shutdown_connections 2024-03-30T00:34:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.129+0000 7fcccb800700 1 --2- 172.21.15.99:0/36807699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4106aa0 0x7fccc4108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.129+0000 7fcccb800700 1 --2- 172.21.15.99:0/36807699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccc4101840 0x7fccc4103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.129+0000 7fcccb800700 1 --2- 172.21.15.99:0/36807699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc4104170 0x7fccc4106560 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.129+0000 7fcccb800700 1 -- 172.21.15.99:0/36807699 >> 172.21.15.99:0/36807699 conn(0x7fccc40fb1d0 msgr2=0x7fccc40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.129+0000 7fcccb800700 1 -- 172.21.15.99:0/36807699 shutdown_connections 2024-03-30T00:34:40.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.130+0000 7fcccb800700 1 -- 172.21.15.99:0/36807699 wait complete. 2024-03-30T00:34:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.130+0000 7fcccb800700 1 Processor -- start 2024-03-30T00:34:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fcccb800700 1 -- start start 2024-03-30T00:34:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fcccb800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccc4101840 0x7fccc4198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fcccb800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4104170 0x7fccc41993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:40.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fcccb800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc4106aa0 0x7fccc41a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fcccb800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc41a09a0 con 0x7fccc4106aa0 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fcccb800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc41a0ae0 con 0x7fccc4101840 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fcccb800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc41a0c70 con 0x7fccc4104170 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fccc9800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc4106aa0 0x7fccc41a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fccc9800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc4106aa0 0x7fccc41a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52804/0 (socket says 172.21.15.99:52804) 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fccc9800700 1 -- 172.21.15.99:0/3109325432 learned_addr learned my addr 172.21.15.99:0/3109325432 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fccc8e00700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccc4101840 0x7fccc4198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.131+0000 7fccc3e00700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4104170 0x7fccc41993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.132+0000 7fccc9800700 1 -- 172.21.15.99:0/3109325432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4104170 msgr2=0x7fccc41993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.132+0000 7fccc9800700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4104170 0x7fccc41993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.132+0000 7fccc9800700 1 -- 172.21.15.99:0/3109325432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccc4101840 msgr2=0x7fccc4198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.132+0000 7fccc9800700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccc4101840 0x7fccc4198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.132+0000 7fccc9800700 1 -- 172.21.15.99:0/3109325432 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb00095f0 con 0x7fccc4106aa0 2024-03-30T00:34:40.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.132+0000 7fccc9800700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc4106aa0 0x7fccc41a03d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00ef30 tx=0x7fccbc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:40.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.132+0000 7fccc1600700 1 -- 172.21.15.99:0/3109325432 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc018070 con 0x7fccc4106aa0 2024-03-30T00:34:40.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.132+0000 7fccc1600700 1 -- 172.21.15.99:0/3109325432 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fccbc00f040 con 0x7fccc4106aa0 2024-03-30T00:34:40.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.132+0000 7fccc1600700 1 -- 172.21.15.99:0/3109325432 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00cc70 con 0x7fccc4106aa0 2024-03-30T00:34:40.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.132+0000 7fcccb800700 1 -- 172.21.15.99:0/3109325432 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc41a0ef0 con 0x7fccc4106aa0 2024-03-30T00:34:40.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.132+0000 7fcccb800700 1 -- 172.21.15.99:0/3109325432 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc41a1370 con 0x7fccc4106aa0 2024-03-30T00:34:40.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.133+0000 7fcccb800700 1 -- 172.21.15.99:0/3109325432 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc4067af0 con 0x7fccc4106aa0 2024-03-30T00:34:40.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.135+0000 7fccc1600700 1 -- 172.21.15.99:0/3109325432 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fccbc007500 con 0x7fccc4106aa0 2024-03-30T00:34:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.136+0000 7fccc1600700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fccac0a11a0 0x7fccac0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.137+0000 7fccc1600700 1 -- 172.21.15.99:0/3109325432 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fccbc0cc3f0 con 0x7fccc4106aa0 2024-03-30T00:34:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.137+0000 7fccc8e00700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fccac0a11a0 0x7fccac0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:40.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.137+0000 7fccc8e00700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fccac0a11a0 0x7fccac0a3660 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fccb8004630 tx=0x7fccb800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:40.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.141+0000 7fccc1600700 1 -- 172.21.15.99:0/3109325432 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fccbc095ba0 con 0x7fccc4106aa0 2024-03-30T00:34:40.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.346+0000 7fcccb800700 1 -- 172.21.15.99:0/3109325432 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fccc41a16e0 con 0x7fccc4106aa0 2024-03-30T00:34:40.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.347+0000 7fccc1600700 1 -- 172.21.15.99:0/3109325432 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fccbc0991c0 con 0x7fccc4106aa0 2024-03-30T00:34:40.353 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:34:40.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 -- 172.21.15.99:0/3109325432 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fccac0a11a0 msgr2=0x7fccac0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:40.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fccac0a11a0 0x7fccac0a3660 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fccb8004630 tx=0x7fccb800b3c0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 -- 172.21.15.99:0/3109325432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc4106aa0 msgr2=0x7fccc41a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:40.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc4106aa0 0x7fccc41a03d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00ef30 tx=0x7fccbc00c3a0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 -- 172.21.15.99:0/3109325432 shutdown_connections 2024-03-30T00:34:40.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccc4106aa0 0x7fccc41a03d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fccac0a11a0 0x7fccac0a3660 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccc4101840 0x7fccc4198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 --2- 172.21.15.99:0/3109325432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccc4104170 0x7fccc41993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 -- 172.21.15.99:0/3109325432 >> 172.21.15.99:0/3109325432 conn(0x7fccc40fb1d0 msgr2=0x7fccc40ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:40.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 -- 172.21.15.99:0/3109325432 shutdown_connections 2024-03-30T00:34:40.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.349+0000 7fcccb800700 1 -- 172.21.15.99:0/3109325432 wait complete. 2024-03-30T00:34:40.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:34:40.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.563+0000 7f6347a00700 1 -- 172.21.15.99:0/712772058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6340101f70 msgr2=0x7f63401023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:40.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.563+0000 7f6347a00700 1 --2- 172.21.15.99:0/712772058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6340101f70 0x7f63401023d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f632c0099c0 tx=0x7f632c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.563+0000 7f6347a00700 1 -- 172.21.15.99:0/712772058 shutdown_connections 2024-03-30T00:34:40.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.563+0000 7f6347a00700 1 --2- 172.21.15.99:0/712772058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6340102910 0x7f63401091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.563+0000 7f6347a00700 1 --2- 172.21.15.99:0/712772058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6340100d80 0x7f63401011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.563+0000 7f6347a00700 1 --2- 172.21.15.99:0/712772058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6340101f70 0x7f63401023d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.563+0000 7f6347a00700 1 -- 172.21.15.99:0/712772058 >> 172.21.15.99:0/712772058 conn(0x7f63400fc2f0 msgr2=0x7f63400fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:40.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.563+0000 7f6347a00700 1 -- 172.21.15.99:0/712772058 shutdown_connections 2024-03-30T00:34:40.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.563+0000 7f6347a00700 1 -- 172.21.15.99:0/712772058 wait complete. 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.564+0000 7f6347a00700 1 Processor -- start 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.564+0000 7f6347a00700 1 -- start start 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f6347a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6340100d80 0x7f6340198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f6347a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6340101f70 0x7f6340199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f6347a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6340102910 0x7f63401a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f6347a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63401a09a0 con 0x7f6340101f70 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f6347a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63401a0ae0 con 0x7f6340102910 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f6347a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63401a0c70 con 0x7f6340100d80 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f633fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6340101f70 0x7f6340199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f633fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6340101f70 0x7f6340199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52834/0 (socket says 172.21.15.99:52834) 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f633fe00700 1 -- 172.21.15.99:0/2154087118 learned_addr learned my addr 172.21.15.99:0/2154087118 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:40.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f6345000700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6340100d80 0x7f6340198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f6345a00700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6340102910 0x7f63401a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f633fe00700 1 -- 172.21.15.99:0/2154087118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6340100d80 msgr2=0x7f6340198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f633fe00700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6340100d80 0x7f6340198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f633fe00700 1 -- 172.21.15.99:0/2154087118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6340102910 msgr2=0x7f63401a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f633fe00700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6340102910 0x7f63401a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f633fe00700 1 -- 172.21.15.99:0/2154087118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c0095f0 con 0x7f6340101f70 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.565+0000 7f633fe00700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6340101f70 0x7f6340199400 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f632c000c00 tx=0x7f632c003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.566+0000 7f633d600700 1 -- 172.21.15.99:0/2154087118 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c01d070 con 0x7f6340101f70 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.566+0000 7f633d600700 1 -- 172.21.15.99:0/2154087118 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f632c00bd30 con 0x7f6340101f70 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.566+0000 7f6347a00700 1 -- 172.21.15.99:0/2154087118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63401a0ef0 con 0x7f6340101f70 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.566+0000 7f633d600700 1 -- 172.21.15.99:0/2154087118 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c022bd0 con 0x7f6340101f70 2024-03-30T00:34:40.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.566+0000 7f6347a00700 1 -- 172.21.15.99:0/2154087118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63401a12b0 con 0x7f6340101f70 2024-03-30T00:34:40.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.567+0000 7f6347a00700 1 -- 172.21.15.99:0/2154087118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6340067af0 con 0x7f6340101f70 2024-03-30T00:34:40.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.569+0000 7f633d600700 1 -- 172.21.15.99:0/2154087118 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f632c022d50 con 0x7f6340101f70 2024-03-30T00:34:40.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.570+0000 7f633d600700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f63280a1120 0x7f63280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:40.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.570+0000 7f633d600700 1 -- 172.21.15.99:0/2154087118 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f632c0cdc60 con 0x7f6340101f70 2024-03-30T00:34:40.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.570+0000 7f6345000700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f63280a1120 0x7f63280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:40.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.571+0000 7f6345000700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f63280a1120 0x7f63280a35e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6340101dd0 tx=0x7f633400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:40.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.574+0000 7f633d600700 1 -- 172.21.15.99:0/2154087118 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f632c097410 con 0x7f6340101f70 2024-03-30T00:34:40.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:40 smithi161 ceph-mon[42221]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:34:40.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:40 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/492046870' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-30T00:34:40.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:40 smithi161 ceph-mon[42221]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:34:40.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:40 smithi161 ceph-mon[42221]: mgrmap e18: x(active, since 7m), standbys: y, z 2024-03-30T00:34:40.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:40 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/489080948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:40.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:40 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3109325432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:40.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:40 smithi099 ceph-mon[38079]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:34:40.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:40 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/492046870' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-30T00:34:40.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:40 smithi099 ceph-mon[38079]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:34:40.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:40 smithi099 ceph-mon[38079]: mgrmap e18: x(active, since 7m), standbys: y, z 2024-03-30T00:34:40.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:40 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/489080948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:40.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:40 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3109325432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:40.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.780+0000 7f6347a00700 1 -- 172.21.15.99:0/2154087118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63401a15e0 con 0x7f6340101f70 2024-03-30T00:34:40.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.781+0000 7f633d600700 1 -- 172.21.15.99:0/2154087118 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f632c027020 con 0x7f6340101f70 2024-03-30T00:34:40.788 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-30T00:34:40.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 -- 172.21.15.99:0/2154087118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f63280a1120 msgr2=0x7f63280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:40.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f63280a1120 0x7f63280a35e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6340101dd0 tx=0x7f633400b410 comp rx=0 tx=0).stop 2024-03-30T00:34:40.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 -- 172.21.15.99:0/2154087118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6340101f70 msgr2=0x7f6340199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:40.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6340101f70 0x7f6340199400 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f632c000c00 tx=0x7f632c003a50 comp rx=0 tx=0).stop 2024-03-30T00:34:40.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 -- 172.21.15.99:0/2154087118 shutdown_connections 2024-03-30T00:34:40.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f63280a1120 0x7f63280a35e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6340102910 0x7f63401a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6340100d80 0x7f6340198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 --2- 172.21.15.99:0/2154087118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6340101f70 0x7f6340199400 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:40.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 -- 172.21.15.99:0/2154087118 >> 172.21.15.99:0/2154087118 conn(0x7f63400fc2f0 msgr2=0x7f6340072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:40.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 -- 172.21.15.99:0/2154087118 shutdown_connections 2024-03-30T00:34:40.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.783+0000 7f6347a00700 1 -- 172.21.15.99:0/2154087118 wait complete. 2024-03-30T00:34:40.808 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:34:40.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:40 smithi142 ceph-mon[41779]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:34:40.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:40 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/492046870' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-30T00:34:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:40 smithi142 ceph-mon[41779]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:34:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:40 smithi142 ceph-mon[41779]: mgrmap e18: x(active, since 7m), standbys: y, z 2024-03-30T00:34:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:40 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/489080948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:40.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:40 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3109325432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:41.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.996+0000 7fb741800700 1 -- 172.21.15.99:0/3238128699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb73c102910 msgr2=0x7fb73c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.996+0000 7fb741800700 1 --2- 172.21.15.99:0/3238128699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb73c102910 0x7fb73c1091c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb73400b0d0 tx=0x7fb73400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.996+0000 7fb741800700 1 -- 172.21.15.99:0/3238128699 shutdown_connections 2024-03-30T00:34:41.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.996+0000 7fb741800700 1 --2- 172.21.15.99:0/3238128699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb73c102910 0x7fb73c1091c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.996+0000 7fb741800700 1 --2- 172.21.15.99:0/3238128699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c100d80 0x7fb73c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.996+0000 7fb741800700 1 --2- 172.21.15.99:0/3238128699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb73c101f70 0x7fb73c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.996+0000 7fb741800700 1 -- 172.21.15.99:0/3238128699 >> 172.21.15.99:0/3238128699 conn(0x7fb73c0fc2f0 msgr2=0x7fb73c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:41.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.996+0000 7fb741800700 1 -- 172.21.15.99:0/3238128699 shutdown_connections 2024-03-30T00:34:41.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.996+0000 7fb741800700 1 -- 172.21.15.99:0/3238128699 wait complete. 2024-03-30T00:34:41.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.997+0000 7fb741800700 1 Processor -- start 2024-03-30T00:34:41.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.997+0000 7fb741800700 1 -- start start 2024-03-30T00:34:41.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb741800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb73c100d80 0x7fb73c074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb741800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb73c101f70 0x7fb73c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb741800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c102910 0x7fb73c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb741800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c073de0 con 0x7fb73c101f70 2024-03-30T00:34:41.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb741800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c073f50 con 0x7fb73c100d80 2024-03-30T00:34:41.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb741800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c1a0e20 con 0x7fb73c102910 2024-03-30T00:34:41.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb73a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb73c101f70 0x7fb73c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb73a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb73c101f70 0x7fb73c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52862/0 (socket says 172.21.15.99:52862) 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb73a000700 1 -- 172.21.15.99:0/3495333547 learned_addr learned my addr 172.21.15.99:0/3495333547 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb73aa00700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb73c100d80 0x7fb73c074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb73b400700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c102910 0x7fb73c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb73a000700 1 -- 172.21.15.99:0/3495333547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c102910 msgr2=0x7fb73c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb73a000700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c102910 0x7fb73c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb73a000700 1 -- 172.21.15.99:0/3495333547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb73c100d80 msgr2=0x7fb73c074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb73a000700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb73c100d80 0x7fb73c074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb73a000700 1 -- 172.21.15.99:0/3495333547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb734009d00 con 0x7fb73c101f70 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.998+0000 7fb73aa00700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb73c100d80 0x7fb73c074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.999+0000 7fb73a000700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb73c101f70 0x7fb73c0731a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb72800d980 tx=0x7fb72800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.999+0000 7fb733400700 1 -- 172.21.15.99:0/3495333547 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72800bb60 con 0x7fb73c101f70 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.999+0000 7fb733400700 1 -- 172.21.15.99:0/3495333547 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb728007640 con 0x7fb73c101f70 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.999+0000 7fb741800700 1 -- 172.21.15.99:0/3495333547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73c1a1080 con 0x7fb73c101f70 2024-03-30T00:34:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.999+0000 7fb733400700 1 -- 172.21.15.99:0/3495333547 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728004d60 con 0x7fb73c101f70 2024-03-30T00:34:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:40.999+0000 7fb741800700 1 -- 172.21.15.99:0/3495333547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73c1a1590 con 0x7fb73c101f70 2024-03-30T00:34:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.000+0000 7fb741800700 1 -- 172.21.15.99:0/3495333547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73c108250 con 0x7fb73c101f70 2024-03-30T00:34:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.002+0000 7fb733400700 1 -- 172.21.15.99:0/3495333547 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fb728004020 con 0x7fb73c101f70 2024-03-30T00:34:41.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.003+0000 7fb733400700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7200a1120 0x7fb7200a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.003+0000 7fb733400700 1 -- 172.21.15.99:0/3495333547 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb7280cc800 con 0x7fb73c101f70 2024-03-30T00:34:41.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.003+0000 7fb73aa00700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7200a1120 0x7fb7200a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.004+0000 7fb73aa00700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7200a1120 0x7fb7200a35e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb72c005950 tx=0x7fb72c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:41.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.007+0000 7fb733400700 1 -- 172.21.15.99:0/3495333547 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb728095f20 con 0x7fb73c101f70 2024-03-30T00:34:41.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.216+0000 7fb741800700 1 -- 172.21.15.99:0/3495333547 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb73c10c6d0 con 0x7fb73c101f70 2024-03-30T00:34:41.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.217+0000 7fb733400700 1 -- 172.21.15.99:0/3495333547 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7280160a0 con 0x7fb73c101f70 2024-03-30T00:34:41.224 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-30T00:34:41.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.219+0000 7fb741800700 1 -- 172.21.15.99:0/3495333547 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7200a1120 msgr2=0x7fb7200a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.219+0000 7fb741800700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7200a1120 0x7fb7200a35e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb72c005950 tx=0x7fb72c0058e0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.219+0000 7fb741800700 1 -- 172.21.15.99:0/3495333547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb73c101f70 msgr2=0x7fb73c0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.219+0000 7fb741800700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb73c101f70 0x7fb73c0731a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb72800d980 tx=0x7fb72800dd40 comp rx=0 tx=0).stop 2024-03-30T00:34:41.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.219+0000 7fb741800700 1 -- 172.21.15.99:0/3495333547 shutdown_connections 2024-03-30T00:34:41.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.219+0000 7fb741800700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb73c102910 0x7fb73c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.219+0000 7fb741800700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7200a1120 0x7fb7200a35e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.219+0000 7fb741800700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb73c100d80 0x7fb73c074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.219+0000 7fb741800700 1 --2- 172.21.15.99:0/3495333547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb73c101f70 0x7fb73c0731a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.219+0000 7fb741800700 1 -- 172.21.15.99:0/3495333547 >> 172.21.15.99:0/3495333547 conn(0x7fb73c0fc2f0 msgr2=0x7fb73c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:41.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.220+0000 7fb741800700 1 -- 172.21.15.99:0/3495333547 shutdown_connections 2024-03-30T00:34:41.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.220+0000 7fb741800700 1 -- 172.21.15.99:0/3495333547 wait complete. 2024-03-30T00:34:41.244 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:34:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.432+0000 7f72c7000700 1 -- 172.21.15.99:0/1678980187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c0102800 msgr2=0x7f72c01090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.432+0000 7f72c7000700 1 --2- 172.21.15.99:0/1678980187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c0102800 0x7f72c01090b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f72b800b0d0 tx=0x7f72b800b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.432+0000 7f72c7000700 1 -- 172.21.15.99:0/1678980187 shutdown_connections 2024-03-30T00:34:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.432+0000 7f72c7000700 1 --2- 172.21.15.99:0/1678980187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c0102800 0x7f72c01090b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.432+0000 7f72c7000700 1 --2- 172.21.15.99:0/1678980187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c0100c70 0x7f72c0101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.432+0000 7f72c7000700 1 --2- 172.21.15.99:0/1678980187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c0101e60 0x7f72c01022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.432+0000 7f72c7000700 1 -- 172.21.15.99:0/1678980187 >> 172.21.15.99:0/1678980187 conn(0x7f72c00fc1e0 msgr2=0x7f72c00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:41.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.432+0000 7f72c7000700 1 -- 172.21.15.99:0/1678980187 shutdown_connections 2024-03-30T00:34:41.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.432+0000 7f72c7000700 1 -- 172.21.15.99:0/1678980187 wait complete. 2024-03-30T00:34:41.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.433+0000 7f72c7000700 1 Processor -- start 2024-03-30T00:34:41.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.433+0000 7f72c7000700 1 -- start start 2024-03-30T00:34:41.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72c7000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c0100c70 0x7f72c0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72c7000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c0101e60 0x7f72c00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72c7000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c0102800 0x7f72c00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72c7000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0073e80 con 0x7f72c0101e60 2024-03-30T00:34:41.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72c7000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0073ff0 con 0x7f72c0102800 2024-03-30T00:34:41.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72c7000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c01a0c30 con 0x7f72c0100c70 2024-03-30T00:34:41.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72bf400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c0101e60 0x7f72c00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72bf400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c0101e60 0x7f72c00731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52884/0 (socket says 172.21.15.99:52884) 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72bf400700 1 -- 172.21.15.99:0/3009927719 learned_addr learned my addr 172.21.15.99:0/3009927719 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72bfe00700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c0100c70 0x7f72c0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72c5000700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c0102800 0x7f72c00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72bf400700 1 -- 172.21.15.99:0/3009927719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c0100c70 msgr2=0x7f72c0074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72bf400700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c0100c70 0x7f72c0074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72bf400700 1 -- 172.21.15.99:0/3009927719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c0102800 msgr2=0x7f72c00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72bf400700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c0102800 0x7f72c00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72bf400700 1 -- 172.21.15.99:0/3009927719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8009d00 con 0x7f72c0101e60 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.434+0000 7f72c5000700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c0102800 0x7f72c00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.435+0000 7f72bf400700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c0101e60 0x7f72c00731a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00e9a0 tx=0x7f72ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.435+0000 7f72bcc00700 1 -- 172.21.15.99:0/3009927719 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac009790 con 0x7f72c0101e60 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.435+0000 7f72bcc00700 1 -- 172.21.15.99:0/3009927719 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72ac010430 con 0x7f72c0101e60 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.435+0000 7f72c7000700 1 -- 172.21.15.99:0/3009927719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c01a0ec0 con 0x7f72c0101e60 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.435+0000 7f72bcc00700 1 -- 172.21.15.99:0/3009927719 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac018680 con 0x7f72c0101e60 2024-03-30T00:34:41.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.435+0000 7f72c7000700 1 -- 172.21.15.99:0/3009927719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c01a1340 con 0x7f72c0101e60 2024-03-30T00:34:41.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.436+0000 7f72c7000700 1 -- 172.21.15.99:0/3009927719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c01080c0 con 0x7f72c0101e60 2024-03-30T00:34:41.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.438+0000 7f72bcc00700 1 -- 172.21.15.99:0/3009927719 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f72ac0105a0 con 0x7f72c0101e60 2024-03-30T00:34:41.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.439+0000 7f72bcc00700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72a40a1120 0x7f72a40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.439+0000 7f72bcc00700 1 -- 172.21.15.99:0/3009927719 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f72ac014070 con 0x7f72c0101e60 2024-03-30T00:34:41.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.439+0000 7f72bfe00700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72a40a1120 0x7f72a40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.440+0000 7f72bfe00700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72a40a1120 0x7f72a40a35e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72c0101cc0 tx=0x7f72b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:41.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.443+0000 7f72bcc00700 1 -- 172.21.15.99:0/3009927719 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f72ac0529f0 con 0x7f72c0101e60 2024-03-30T00:34:41.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.680+0000 7f72c7000700 1 -- 172.21.15.99:0/3009927719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72c0067e60 con 0x7f72c0101e60 2024-03-30T00:34:41.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.681+0000 7f72bcc00700 1 -- 172.21.15.99:0/3009927719 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22615 (secure 0 0 0) 0x7f72ac096290 con 0x7f72c0101e60 2024-03-30T00:34:41.688 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:41.689 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:34:10.248126+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8},{"from":5,"to":7}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:34:41.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 -- 172.21.15.99:0/3009927719 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72a40a1120 msgr2=0x7f72a40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72a40a1120 0x7f72a40a35e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72c0101cc0 tx=0x7f72b0009260 comp rx=0 tx=0).stop 2024-03-30T00:34:41.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 -- 172.21.15.99:0/3009927719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c0101e60 msgr2=0x7f72c00731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c0101e60 0x7f72c00731a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00e9a0 tx=0x7f72ac00ecb0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 -- 172.21.15.99:0/3009927719 shutdown_connections 2024-03-30T00:34:41.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72a40a1120 0x7f72a40a35e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c0102800 0x7f72c00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c0100c70 0x7f72c0074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 --2- 172.21.15.99:0/3009927719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c0101e60 0x7f72c00731a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 -- 172.21.15.99:0/3009927719 >> 172.21.15.99:0/3009927719 conn(0x7f72c00fc1e0 msgr2=0x7f72c0105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:41.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 -- 172.21.15.99:0/3009927719 shutdown_connections 2024-03-30T00:34:41.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.684+0000 7f72c7000700 1 -- 172.21.15.99:0/3009927719 wait complete. 2024-03-30T00:34:41.707 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:41 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2154087118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:41.707 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:41 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3495333547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:41.710 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:33:14.085184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:33:15.090699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:34:41.710 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:34:41.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:41 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2154087118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:41.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:41 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3495333547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:41.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:41 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2154087118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:41.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:41 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3495333547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:41.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.899+0000 7fb8e0a00700 1 -- 172.21.15.99:0/3186703013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8dc100d80 msgr2=0x7fb8dc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.899+0000 7fb8e0a00700 1 --2- 172.21.15.99:0/3186703013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8dc100d80 0x7fb8dc1011a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0099c0 tx=0x7fb8cc009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.899+0000 7fb8e0a00700 1 -- 172.21.15.99:0/3186703013 shutdown_connections 2024-03-30T00:34:41.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.899+0000 7fb8e0a00700 1 --2- 172.21.15.99:0/3186703013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8dc102910 0x7fb8dc1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.899+0000 7fb8e0a00700 1 --2- 172.21.15.99:0/3186703013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8dc100d80 0x7fb8dc1011a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.899+0000 7fb8e0a00700 1 --2- 172.21.15.99:0/3186703013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc101f70 0x7fb8dc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.899+0000 7fb8e0a00700 1 -- 172.21.15.99:0/3186703013 >> 172.21.15.99:0/3186703013 conn(0x7fb8dc0fc2f0 msgr2=0x7fb8dc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:41.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.899+0000 7fb8e0a00700 1 -- 172.21.15.99:0/3186703013 shutdown_connections 2024-03-30T00:34:41.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.899+0000 7fb8e0a00700 1 -- 172.21.15.99:0/3186703013 wait complete. 2024-03-30T00:34:41.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.900+0000 7fb8e0a00700 1 Processor -- start 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.900+0000 7fb8e0a00700 1 -- start start 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.900+0000 7fb8e0a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8dc100d80 0x7fb8dc196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.900+0000 7fb8e0a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8dc101f70 0x7fb8dc197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.900+0000 7fb8e0a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc102910 0x7fb8dc19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.900+0000 7fb8e0a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc19e780 con 0x7fb8dc100d80 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.900+0000 7fb8e0a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc19e8f0 con 0x7fb8dc101f70 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.900+0000 7fb8e0a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc19eab0 con 0x7fb8dc102910 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8d9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8dc100d80 0x7fb8dc196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8d9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8dc100d80 0x7fb8dc196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52906/0 (socket says 172.21.15.99:52906) 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8d9e00700 1 -- 172.21.15.99:0/1836800468 learned_addr learned my addr 172.21.15.99:0/1836800468 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8da800700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc102910 0x7fb8dc19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8d9e00700 1 -- 172.21.15.99:0/1836800468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc102910 msgr2=0x7fb8dc19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8d9e00700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc102910 0x7fb8dc19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8d9400700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8dc101f70 0x7fb8dc197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8d9e00700 1 -- 172.21.15.99:0/1836800468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8dc101f70 msgr2=0x7fb8dc197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:41.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8d9e00700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8dc101f70 0x7fb8dc197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:41.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8d9e00700 1 -- 172.21.15.99:0/1836800468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc0095f0 con 0x7fb8dc100d80 2024-03-30T00:34:41.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8d9e00700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8dc100d80 0x7fb8dc196ce0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009f50 tx=0x7fb8cc003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:41.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.901+0000 7fb8d2a00700 1 -- 172.21.15.99:0/1836800468 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc01d070 con 0x7fb8dc100d80 2024-03-30T00:34:41.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.902+0000 7fb8d2a00700 1 -- 172.21.15.99:0/1836800468 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8cc00bda0 con 0x7fb8dc100d80 2024-03-30T00:34:41.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.902+0000 7fb8e0a00700 1 -- 172.21.15.99:0/1836800468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc19ed30 con 0x7fb8dc100d80 2024-03-30T00:34:41.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.902+0000 7fb8d2a00700 1 -- 172.21.15.99:0/1836800468 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc022dc0 con 0x7fb8dc100d80 2024-03-30T00:34:41.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.902+0000 7fb8e0a00700 1 -- 172.21.15.99:0/1836800468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc19f150 con 0x7fb8dc100d80 2024-03-30T00:34:41.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.903+0000 7fb8e0a00700 1 -- 172.21.15.99:0/1836800468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8dc1082a0 con 0x7fb8dc100d80 2024-03-30T00:34:41.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.905+0000 7fb8d2a00700 1 -- 172.21.15.99:0/1836800468 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fb8cc004000 con 0x7fb8dc100d80 2024-03-30T00:34:41.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.906+0000 7fb8d2a00700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb8c00a1120 0x7fb8c00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:41.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.906+0000 7fb8d2a00700 1 -- 172.21.15.99:0/1836800468 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fb8cc0cde00 con 0x7fb8dc100d80 2024-03-30T00:34:41.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.906+0000 7fb8d9400700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb8c00a1120 0x7fb8c00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:41.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.907+0000 7fb8d9400700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb8c00a1120 0x7fb8c00a35e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8005950 tx=0x7fb8c80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:41.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:41.909+0000 7fb8d2a00700 1 -- 172.21.15.99:0/1836800468 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb8cc0975b0 con 0x7fb8dc100d80 2024-03-30T00:34:42.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.116+0000 7fb8e0a00700 1 -- 172.21.15.99:0/1836800468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8dc10c640 con 0x7fb8dc100d80 2024-03-30T00:34:42.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.116+0000 7fb8d2a00700 1 -- 172.21.15.99:0/1836800468 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8cc0270a0 con 0x7fb8dc100d80 2024-03-30T00:34:42.123 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:34:42.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.118+0000 7fb8e0a00700 1 -- 172.21.15.99:0/1836800468 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb8c00a1120 msgr2=0x7fb8c00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.118+0000 7fb8e0a00700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb8c00a1120 0x7fb8c00a35e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8005950 tx=0x7fb8c80058e0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.119+0000 7fb8e0a00700 1 -- 172.21.15.99:0/1836800468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8dc100d80 msgr2=0x7fb8dc196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.119+0000 7fb8e0a00700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8dc100d80 0x7fb8dc196ce0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009f50 tx=0x7fb8cc003b00 comp rx=0 tx=0).stop 2024-03-30T00:34:42.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.119+0000 7fb8e0a00700 1 -- 172.21.15.99:0/1836800468 shutdown_connections 2024-03-30T00:34:42.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.119+0000 7fb8e0a00700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb8dc102910 0x7fb8dc19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.119+0000 7fb8e0a00700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8dc100d80 0x7fb8dc196ce0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.119+0000 7fb8e0a00700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb8c00a1120 0x7fb8c00a35e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.119+0000 7fb8e0a00700 1 --2- 172.21.15.99:0/1836800468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8dc101f70 0x7fb8dc197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.119+0000 7fb8e0a00700 1 -- 172.21.15.99:0/1836800468 >> 172.21.15.99:0/1836800468 conn(0x7fb8dc0fc2f0 msgr2=0x7fb8dc1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:42.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.119+0000 7fb8e0a00700 1 -- 172.21.15.99:0/1836800468 shutdown_connections 2024-03-30T00:34:42.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.119+0000 7fb8e0a00700 1 -- 172.21.15.99:0/1836800468 wait complete. 2024-03-30T00:34:42.144 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:34:42.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.333+0000 7f7fafe00700 1 -- 172.21.15.99:0/3235348634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb0074df0 msgr2=0x7f7fb0073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.333+0000 7f7fafe00700 1 --2- 172.21.15.99:0/3235348634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb0074df0 0x7f7fb0073240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009990 tx=0x7f7fa0009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.333+0000 7f7fafe00700 1 -- 172.21.15.99:0/3235348634 shutdown_connections 2024-03-30T00:34:42.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.333+0000 7f7fafe00700 1 --2- 172.21.15.99:0/3235348634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb0108a80 0x7f7fb010af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.333+0000 7f7fafe00700 1 --2- 172.21.15.99:0/3235348634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb0074df0 0x7f7fb0073240 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.333+0000 7f7fafe00700 1 --2- 172.21.15.99:0/3235348634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb0073810 0x7f7fb0073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.333+0000 7f7fafe00700 1 -- 172.21.15.99:0/3235348634 >> 172.21.15.99:0/3235348634 conn(0x7f7fb00fc110 msgr2=0x7f7fb00fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:42.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.333+0000 7f7fafe00700 1 -- 172.21.15.99:0/3235348634 shutdown_connections 2024-03-30T00:34:42.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.333+0000 7f7fafe00700 1 -- 172.21.15.99:0/3235348634 wait complete. 2024-03-30T00:34:42.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.335+0000 7f7fafe00700 1 Processor -- start 2024-03-30T00:34:42.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.335+0000 7f7fafe00700 1 -- start start 2024-03-30T00:34:42.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.335+0000 7f7fafe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb0073810 0x7f7fb0198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:42.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.335+0000 7f7fafe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb0074df0 0x7f7fb0199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:42.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.335+0000 7f7fafe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb0108a80 0x7f7fb01a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:42.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.335+0000 7f7fafe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01a07f0 con 0x7f7fb0074df0 2024-03-30T00:34:42.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.335+0000 7f7fafe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01a0960 con 0x7f7fb0073810 2024-03-30T00:34:42.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.335+0000 7f7fafe00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb01a0b20 con 0x7f7fb0108a80 2024-03-30T00:34:42.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.335+0000 7f7fae000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb0074df0 0x7f7fb0199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fae000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb0074df0 0x7f7fb0199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52930/0 (socket says 172.21.15.99:52930) 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fae000700 1 -- 172.21.15.99:0/1589169405 learned_addr learned my addr 172.21.15.99:0/1589169405 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7faf400700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb0108a80 0x7f7fb01a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7faea00700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb0073810 0x7f7fb0198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fae000700 1 -- 172.21.15.99:0/1589169405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb0108a80 msgr2=0x7f7fb01a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fae000700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb0108a80 0x7f7fb01a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fae000700 1 -- 172.21.15.99:0/1589169405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb0073810 msgr2=0x7f7fb0198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fae000700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb0073810 0x7f7fb0198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fae000700 1 -- 172.21.15.99:0/1589169405 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa00095f0 con 0x7f7fb0074df0 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7faea00700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb0073810 0x7f7fb0198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fae000700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb0074df0 0x7f7fb0199210 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c00e970 tx=0x7f7f9c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fa7400700 1 -- 172.21.15.99:0/1589169405 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c009790 con 0x7f7fb0074df0 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fa7400700 1 -- 172.21.15.99:0/1589169405 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7f9c00eec0 con 0x7f7fb0074df0 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fafe00700 1 -- 172.21.15.99:0/1589169405 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb01a0e00 con 0x7f7fb0074df0 2024-03-30T00:34:42.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.336+0000 7f7fa7400700 1 -- 172.21.15.99:0/1589169405 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c003c10 con 0x7f7fb0074df0 2024-03-30T00:34:42.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.337+0000 7f7fafe00700 1 -- 172.21.15.99:0/1589169405 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb01a1230 con 0x7f7fb0074df0 2024-03-30T00:34:42.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.338+0000 7f7fafe00700 1 -- 172.21.15.99:0/1589169405 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb0107840 con 0x7f7fb0074df0 2024-03-30T00:34:42.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.339+0000 7f7fa7400700 1 -- 172.21.15.99:0/1589169405 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7f9c003db0 con 0x7f7fb0074df0 2024-03-30T00:34:42.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.340+0000 7f7fa7400700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7f940a11a0 0x7f7f940a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:42.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.341+0000 7f7fa7400700 1 -- 172.21.15.99:0/1589169405 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f7f9c014070 con 0x7f7fb0074df0 2024-03-30T00:34:42.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.341+0000 7f7faea00700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7f940a11a0 0x7f7f940a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:42.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.341+0000 7f7faea00700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7f940a11a0 0x7f7f940a3660 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00ff970 tx=0x7f7fa0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:42.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.344+0000 7f7fa7400700 1 -- 172.21.15.99:0/1589169405 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7f9c096310 con 0x7f7fb0074df0 2024-03-30T00:34:42.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.550+0000 7f7fafe00700 1 -- 172.21.15.99:0/1589169405 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7fb010e4b0 con 0x7f7fb0074df0 2024-03-30T00:34:42.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.551+0000 7f7fa7400700 1 -- 172.21.15.99:0/1589169405 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f9c099930 con 0x7f7fb0074df0 2024-03-30T00:34:42.557 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-30T00:34:42.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.553+0000 7f7fafe00700 1 -- 172.21.15.99:0/1589169405 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7f940a11a0 msgr2=0x7f7f940a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.553+0000 7f7fafe00700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7f940a11a0 0x7f7f940a3660 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00ff970 tx=0x7f7fa0005fb0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.553+0000 7f7fafe00700 1 -- 172.21.15.99:0/1589169405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb0074df0 msgr2=0x7f7fb0199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.553+0000 7f7fafe00700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb0074df0 0x7f7fb0199210 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c00e970 tx=0x7f7f9c00ec80 comp rx=0 tx=0).stop 2024-03-30T00:34:42.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.553+0000 7f7fafe00700 1 -- 172.21.15.99:0/1589169405 shutdown_connections 2024-03-30T00:34:42.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.553+0000 7f7fafe00700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fb0108a80 0x7f7fb01a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.553+0000 7f7fafe00700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7f940a11a0 0x7f7f940a3660 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.553+0000 7f7fafe00700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fb0073810 0x7f7fb0198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.553+0000 7f7fafe00700 1 --2- 172.21.15.99:0/1589169405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fb0074df0 0x7f7fb0199210 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.553+0000 7f7fafe00700 1 -- 172.21.15.99:0/1589169405 >> 172.21.15.99:0/1589169405 conn(0x7f7fb00fc110 msgr2=0x7f7fb0104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:42.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.553+0000 7f7fafe00700 1 -- 172.21.15.99:0/1589169405 shutdown_connections 2024-03-30T00:34:42.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.554+0000 7f7fafe00700 1 -- 172.21.15.99:0/1589169405 wait complete. 2024-03-30T00:34:42.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:34:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:42 smithi161 ceph-mon[42221]: pgmap v301: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:42 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3009927719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:42 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1836800468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:42.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:42 smithi099 ceph-mon[38079]: pgmap v301: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:42 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3009927719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:42.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:42 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1836800468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:42.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.768+0000 7fda69e00700 1 -- 172.21.15.99:0/213622636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda64074df0 msgr2=0x7fda64073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.768+0000 7fda69e00700 1 --2- 172.21.15.99:0/213622636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda64074df0 0x7fda64073240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fda54009990 tx=0x7fda54009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.768+0000 7fda69e00700 1 -- 172.21.15.99:0/213622636 shutdown_connections 2024-03-30T00:34:42.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.768+0000 7fda69e00700 1 --2- 172.21.15.99:0/213622636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda64108c40 0x7fda6410b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.768+0000 7fda69e00700 1 --2- 172.21.15.99:0/213622636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda64074df0 0x7fda64073240 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.768+0000 7fda69e00700 1 --2- 172.21.15.99:0/213622636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda64073810 0x7fda64073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.768+0000 7fda69e00700 1 -- 172.21.15.99:0/213622636 >> 172.21.15.99:0/213622636 conn(0x7fda640fc2f0 msgr2=0x7fda640fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:42.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.768+0000 7fda69e00700 1 -- 172.21.15.99:0/213622636 shutdown_connections 2024-03-30T00:34:42.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.768+0000 7fda69e00700 1 -- 172.21.15.99:0/213622636 wait complete. 2024-03-30T00:34:42.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.769+0000 7fda69e00700 1 Processor -- start 2024-03-30T00:34:42.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.769+0000 7fda69e00700 1 -- start start 2024-03-30T00:34:42.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.769+0000 7fda69e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda64073810 0x7fda64198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:42.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.769+0000 7fda69e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda64074df0 0x7fda641993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:42.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.769+0000 7fda69e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda64108c40 0x7fda641a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:42.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.769+0000 7fda69e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda641a0940 con 0x7fda64108c40 2024-03-30T00:34:42.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.769+0000 7fda69e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda641a0a80 con 0x7fda64074df0 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.769+0000 7fda69e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda641a0c10 con 0x7fda64073810 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda63400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda64108c40 0x7fda641a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda63400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda64108c40 0x7fda641a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52942/0 (socket says 172.21.15.99:52942) 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda63400700 1 -- 172.21.15.99:0/1938070511 learned_addr learned my addr 172.21.15.99:0/1938070511 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda62000700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda64074df0 0x7fda641993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda62a00700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda64073810 0x7fda64198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda63400700 1 -- 172.21.15.99:0/1938070511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda64073810 msgr2=0x7fda64198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda63400700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda64073810 0x7fda64198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda63400700 1 -- 172.21.15.99:0/1938070511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda64074df0 msgr2=0x7fda641993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda63400700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda64074df0 0x7fda641993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda63400700 1 -- 172.21.15.99:0/1938070511 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda540095f0 con 0x7fda64108c40 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda62a00700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda64073810 0x7fda64198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda62000700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda64074df0 0x7fda641993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:42.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda63400700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda64108c40 0x7fda641a0370 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007ee0 tx=0x7fda5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:42.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda68a00700 1 -- 172.21.15.99:0/1938070511 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c017070 con 0x7fda64108c40 2024-03-30T00:34:42.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda68a00700 1 -- 172.21.15.99:0/1938070511 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda5c00f040 con 0x7fda64108c40 2024-03-30T00:34:42.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda69e00700 1 -- 172.21.15.99:0/1938070511 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda641a0ef0 con 0x7fda64108c40 2024-03-30T00:34:42.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.770+0000 7fda68a00700 1 -- 172.21.15.99:0/1938070511 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c005710 con 0x7fda64108c40 2024-03-30T00:34:42.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.771+0000 7fda69e00700 1 -- 172.21.15.99:0/1938070511 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda641a1370 con 0x7fda64108c40 2024-03-30T00:34:42.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.772+0000 7fda69e00700 1 -- 172.21.15.99:0/1938070511 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda64107a00 con 0x7fda64108c40 2024-03-30T00:34:42.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.773+0000 7fda68a00700 1 -- 172.21.15.99:0/1938070511 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fda5c005870 con 0x7fda64108c40 2024-03-30T00:34:42.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.775+0000 7fda68a00700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fda480a1120 0x7fda480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:42.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.775+0000 7fda68a00700 1 -- 172.21.15.99:0/1938070511 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fda5c0cc160 con 0x7fda64108c40 2024-03-30T00:34:42.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.775+0000 7fda62a00700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fda480a1120 0x7fda480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:42.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.776+0000 7fda62a00700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fda480a1120 0x7fda480a35e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fda54006010 tx=0x7fda5400b810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:42.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.779+0000 7fda68a00700 1 -- 172.21.15.99:0/1938070511 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fda5c095880 con 0x7fda64108c40 2024-03-30T00:34:42.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:42 smithi142 ceph-mon[41779]: pgmap v301: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:42.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:42 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3009927719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:42.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:42 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1836800468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:42.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.984+0000 7fda69e00700 1 -- 172.21.15.99:0/1938070511 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda6402ccb0 con 0x7fda64108c40 2024-03-30T00:34:42.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.984+0000 7fda68a00700 1 -- 172.21.15.99:0/1938070511 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda5c098ea0 con 0x7fda64108c40 2024-03-30T00:34:42.991 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-30T00:34:42.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 -- 172.21.15.99:0/1938070511 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fda480a1120 msgr2=0x7fda480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fda480a1120 0x7fda480a35e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fda54006010 tx=0x7fda5400b810 comp rx=0 tx=0).stop 2024-03-30T00:34:42.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 -- 172.21.15.99:0/1938070511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda64108c40 msgr2=0x7fda641a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:42.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda64108c40 0x7fda641a0370 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007ee0 tx=0x7fda5c00d3b0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 -- 172.21.15.99:0/1938070511 shutdown_connections 2024-03-30T00:34:42.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda64108c40 0x7fda641a0370 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda64073810 0x7fda64198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fda480a1120 0x7fda480a35e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 --2- 172.21.15.99:0/1938070511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda64074df0 0x7fda641993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:42.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 -- 172.21.15.99:0/1938070511 >> 172.21.15.99:0/1938070511 conn(0x7fda640fc2f0 msgr2=0x7fda64104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:42.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 -- 172.21.15.99:0/1938070511 shutdown_connections 2024-03-30T00:34:42.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:42.987+0000 7fda69e00700 1 -- 172.21.15.99:0/1938070511 wait complete. 2024-03-30T00:34:43.013 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-30T00:34:43.013 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:34:43.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.201+0000 7fcabde00700 1 -- 172.21.15.99:0/646485452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcab8100c10 msgr2=0x7fcab8101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:43.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.201+0000 7fcabde00700 1 --2- 172.21.15.99:0/646485452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcab8100c10 0x7fcab8101030 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80099c0 tx=0x7fcaa8009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.201+0000 7fcabde00700 1 -- 172.21.15.99:0/646485452 shutdown_connections 2024-03-30T00:34:43.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.201+0000 7fcabde00700 1 --2- 172.21.15.99:0/646485452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab81027a0 0x7fcab8109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.201+0000 7fcabde00700 1 --2- 172.21.15.99:0/646485452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcab8100c10 0x7fcab8101030 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.201+0000 7fcabde00700 1 --2- 172.21.15.99:0/646485452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab8101e00 0x7fcab8102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.201+0000 7fcabde00700 1 -- 172.21.15.99:0/646485452 >> 172.21.15.99:0/646485452 conn(0x7fcab80fc160 msgr2=0x7fcab80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:43.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.201+0000 7fcabde00700 1 -- 172.21.15.99:0/646485452 shutdown_connections 2024-03-30T00:34:43.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.201+0000 7fcabde00700 1 -- 172.21.15.99:0/646485452 wait complete. 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.202+0000 7fcabde00700 1 Processor -- start 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcabde00700 1 -- start start 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcabde00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab8100c10 0x7fcab8198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcabde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcab8101e00 0x7fcab8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcabde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab81027a0 0x7fcab81a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcabde00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab81a0820 con 0x7fcab8101e00 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcabde00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab81a0990 con 0x7fcab81027a0 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcabde00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab81a0b50 con 0x7fcab8100c10 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcab6000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcab8101e00 0x7fcab8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcab6000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcab8101e00 0x7fcab8199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52968/0 (socket says 172.21.15.99:52968) 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcab6000700 1 -- 172.21.15.99:0/3595719086 learned_addr learned my addr 172.21.15.99:0/3595719086 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcab7400700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab81027a0 0x7fcab81a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:43.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.203+0000 7fcab6a00700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab8100c10 0x7fcab8198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.204+0000 7fcab6000700 1 -- 172.21.15.99:0/3595719086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab8100c10 msgr2=0x7fcab8198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.204+0000 7fcab6000700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab8100c10 0x7fcab8198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.204+0000 7fcab6000700 1 -- 172.21.15.99:0/3595719086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab81027a0 msgr2=0x7fcab81a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.204+0000 7fcab6000700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab81027a0 0x7fcab81a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.204+0000 7fcab6000700 1 -- 172.21.15.99:0/3595719086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa80095f0 con 0x7fcab8101e00 2024-03-30T00:34:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.204+0000 7fcab6000700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcab8101e00 0x7fcab8199240 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400d950 tx=0x7fcaa400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.204+0000 7fcabca00700 1 -- 172.21.15.99:0/3595719086 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa400f890 con 0x7fcab8101e00 2024-03-30T00:34:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.204+0000 7fcabca00700 1 -- 172.21.15.99:0/3595719086 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcaa400fe80 con 0x7fcab8101e00 2024-03-30T00:34:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.204+0000 7fcabde00700 1 -- 172.21.15.99:0/3595719086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab81a0e30 con 0x7fcab8101e00 2024-03-30T00:34:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.204+0000 7fcabca00700 1 -- 172.21.15.99:0/3595719086 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa400e3b0 con 0x7fcab8101e00 2024-03-30T00:34:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.204+0000 7fcabde00700 1 -- 172.21.15.99:0/3595719086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab81a1260 con 0x7fcab8101e00 2024-03-30T00:34:43.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.206+0000 7fcabde00700 1 -- 172.21.15.99:0/3595719086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab80679a0 con 0x7fcab8101e00 2024-03-30T00:34:43.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.207+0000 7fcabca00700 1 -- 172.21.15.99:0/3595719086 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fcaa4010460 con 0x7fcab8101e00 2024-03-30T00:34:43.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.208+0000 7fcabca00700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fca9c0a1120 0x7fca9c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:43.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.209+0000 7fcabca00700 1 -- 172.21.15.99:0/3595719086 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fcaa40cca40 con 0x7fcab8101e00 2024-03-30T00:34:43.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.209+0000 7fcab6a00700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fca9c0a1120 0x7fca9c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:43.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.209+0000 7fcab6a00700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fca9c0a1120 0x7fca9c0a35e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8006010 tx=0x7fcaa8003740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:43.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.212+0000 7fcabca00700 1 -- 172.21.15.99:0/3595719086 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcaa4096160 con 0x7fcab8101e00 2024-03-30T00:34:43.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.468+0000 7fcabde00700 1 -- 172.21.15.99:0/3595719086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcab810c320 con 0x7fcab8101e00 2024-03-30T00:34:43.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.469+0000 7fcabca00700 1 -- 172.21.15.99:0/3595719086 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10860 (secure 0 0 0) 0x7fcaa4020090 con 0x7fcab8101e00 2024-03-30T00:34:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:43.476 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:34:39.366251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:34:43.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.471+0000 7fcabde00700 1 -- 172.21.15.99:0/3595719086 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fca9c0a1120 msgr2=0x7fca9c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:43.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.471+0000 7fcabde00700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fca9c0a1120 0x7fca9c0a35e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8006010 tx=0x7fcaa8003740 comp rx=0 tx=0).stop 2024-03-30T00:34:43.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.471+0000 7fcabde00700 1 -- 172.21.15.99:0/3595719086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcab8101e00 msgr2=0x7fcab8199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:43.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.471+0000 7fcabde00700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcab8101e00 0x7fcab8199240 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400d950 tx=0x7fcaa400dc60 comp rx=0 tx=0).stop 2024-03-30T00:34:43.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.471+0000 7fcabde00700 1 -- 172.21.15.99:0/3595719086 shutdown_connections 2024-03-30T00:34:43.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.471+0000 7fcabde00700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fca9c0a1120 0x7fca9c0a35e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.471+0000 7fcabde00700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcab81027a0 0x7fcab81a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.471+0000 7fcabde00700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab8100c10 0x7fcab8198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.471+0000 7fcabde00700 1 --2- 172.21.15.99:0/3595719086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcab8101e00 0x7fcab8199240 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.471+0000 7fcabde00700 1 -- 172.21.15.99:0/3595719086 >> 172.21.15.99:0/3595719086 conn(0x7fcab80fc160 msgr2=0x7fcab81075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:43.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.472+0000 7fcabde00700 1 -- 172.21.15.99:0/3595719086 shutdown_connections 2024-03-30T00:34:43.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.472+0000 7fcabde00700 1 -- 172.21.15.99:0/3595719086 wait complete. 2024-03-30T00:34:43.479 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2024-03-30T00:34:43.496 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-30T00:34:43.496 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-30T00:34:43.497 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:34:43.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.683+0000 7fce03e00700 1 -- 172.21.15.99:0/1208564225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce04072bb0 msgr2=0x7fce04107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:43.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.683+0000 7fce03e00700 1 --2- 172.21.15.99:0/1208564225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce04072bb0 0x7fce04107db0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009990 tx=0x7fcdf4009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.683+0000 7fce03e00700 1 -- 172.21.15.99:0/1208564225 shutdown_connections 2024-03-30T00:34:43.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.683+0000 7fce03e00700 1 --2- 172.21.15.99:0/1208564225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce0410f180 0x7fce04111640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.683+0000 7fce03e00700 1 --2- 172.21.15.99:0/1208564225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce04072bb0 0x7fce04107db0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.683+0000 7fce03e00700 1 --2- 172.21.15.99:0/1208564225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce04108380 0x7fce04108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.683+0000 7fce03e00700 1 -- 172.21.15.99:0/1208564225 >> 172.21.15.99:0/1208564225 conn(0x7fce0406de10 msgr2=0x7fce04070250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:43.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.683+0000 7fce03e00700 1 -- 172.21.15.99:0/1208564225 shutdown_connections 2024-03-30T00:34:43.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.683+0000 7fce03e00700 1 -- 172.21.15.99:0/1208564225 wait complete. 2024-03-30T00:34:43.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.684+0000 7fce03e00700 1 Processor -- start 2024-03-30T00:34:43.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce03e00700 1 -- start start 2024-03-30T00:34:43.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce03e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce04072bb0 0x7fce0419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:43.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce03e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce04108380 0x7fce0419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:43.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce03e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce0410f180 0x7fce041a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:43.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce03e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce041a6e90 con 0x7fce04072bb0 2024-03-30T00:34:43.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce03e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce041a6fd0 con 0x7fce0410f180 2024-03-30T00:34:43.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce03e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce041a7160 con 0x7fce04108380 2024-03-30T00:34:43.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce02a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce04072bb0 0x7fce0419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:43.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce02a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce04072bb0 0x7fce0419f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52994/0 (socket says 172.21.15.99:52994) 2024-03-30T00:34:43.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce02a00700 1 -- 172.21.15.99:0/98872559 learned_addr learned my addr 172.21.15.99:0/98872559 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:43.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce03400700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce0410f180 0x7fce041a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:43.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.685+0000 7fce02000700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce04108380 0x7fce0419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fce02a00700 1 -- 172.21.15.99:0/98872559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce04108380 msgr2=0x7fce0419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fce02a00700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce04108380 0x7fce0419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fce02a00700 1 -- 172.21.15.99:0/98872559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce0410f180 msgr2=0x7fce041a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fce02a00700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce0410f180 0x7fce041a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fce02a00700 1 -- 172.21.15.99:0/98872559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf40095f0 con 0x7fce04072bb0 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fce02000700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce04108380 0x7fce0419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fce02a00700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce04072bb0 0x7fce0419f360 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4005910 tx=0x7fcdf4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fcdfb400700 1 -- 172.21.15.99:0/98872559 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf401d070 con 0x7fce04072bb0 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fce03e00700 1 -- 172.21.15.99:0/98872559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce041a73e0 con 0x7fce04072bb0 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fcdfb400700 1 -- 172.21.15.99:0/98872559 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdf400bdb0 con 0x7fce04072bb0 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fcdfb400700 1 -- 172.21.15.99:0/98872559 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4022d40 con 0x7fce04072bb0 2024-03-30T00:34:43.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.686+0000 7fce03e00700 1 -- 172.21.15.99:0/98872559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce041a7800 con 0x7fce04072bb0 2024-03-30T00:34:43.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.688+0000 7fce03e00700 1 -- 172.21.15.99:0/98872559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0410df90 con 0x7fce04072bb0 2024-03-30T00:34:43.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.690+0000 7fcdfb400700 1 -- 172.21.15.99:0/98872559 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fcdf4022470 con 0x7fce04072bb0 2024-03-30T00:34:43.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.691+0000 7fcdfb400700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcde80a1120 0x7fcde80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:43.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.691+0000 7fcdfb400700 1 -- 172.21.15.99:0/98872559 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fcdf40cde30 con 0x7fce04072bb0 2024-03-30T00:34:43.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.691+0000 7fce02000700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcde80a1120 0x7fcde80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:43.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.692+0000 7fce02000700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcde80a1120 0x7fcde80a35e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce04072840 tx=0x7fcdf0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:43.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.695+0000 7fcdfb400700 1 -- 172.21.15.99:0/98872559 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcdf40975e0 con 0x7fce04072bb0 2024-03-30T00:34:43.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:43 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1589169405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:43.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:43 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1938070511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:43.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:43 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1589169405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:43.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:43 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1938070511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:43.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:43 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1589169405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:43.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:43 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1938070511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:43.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.901+0000 7fce03e00700 1 -- 172.21.15.99:0/98872559 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce040611f0 con 0x7fcde80a1120 2024-03-30T00:34:43.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.912+0000 7fcdfb400700 1 -- 172.21.15.99:0/98872559 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343408 (secure 0 0 0) 0x7fce040611f0 con 0x7fcde80a1120 2024-03-30T00:34:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 -- 172.21.15.99:0/98872559 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcde80a1120 msgr2=0x7fcde80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcde80a1120 0x7fcde80a35e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fce04072840 tx=0x7fcdf0009260 comp rx=0 tx=0).stop 2024-03-30T00:34:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 -- 172.21.15.99:0/98872559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce04072bb0 msgr2=0x7fce0419f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce04072bb0 0x7fce0419f360 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4005910 tx=0x7fcdf4004730 comp rx=0 tx=0).stop 2024-03-30T00:34:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 -- 172.21.15.99:0/98872559 shutdown_connections 2024-03-30T00:34:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fcde80a1120 0x7fcde80a35e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce0410f180 0x7fce041a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce04072bb0 0x7fce0419f360 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 --2- 172.21.15.99:0/98872559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce04108380 0x7fce0419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 -- 172.21.15.99:0/98872559 >> 172.21.15.99:0/98872559 conn(0x7fce0406de10 msgr2=0x7fce0410b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 -- 172.21.15.99:0/98872559 shutdown_connections 2024-03-30T00:34:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:43.917+0000 7fce03e00700 1 -- 172.21.15.99:0/98872559 wait complete. 2024-03-30T00:34:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-30T00:34:43.943 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-03-30T00:34:43.029365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664438,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":639,"num_read_kb":441,"num_write":351,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":631,"ondisk_log_size":631,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378260,"kb_used_data":13084,"kb_used_omap":18,"kb_used_meta":365037,"kb_avail":1124563564,"statfs":{"total":1151940427776,"available":1151553089536,"internally_reserved":0,"allocated":13398016,"data_stored":7346982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":373798214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6051,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":38,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003339"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064138+0000","last_change":"2024-03-30T00:33:15.686814+0000","last_active":"2024-03-30T00:34:39.064138+0000","last_peered":"2024-03-30T00:34:39.064138+0000","last_clean":"2024-03-30T00:34:39.064138+0000","last_became_active":"2024-03-30T00:33:15.686432+0000","last_became_peered":"2024-03-30T00:33:15.686432+0000","last_unstale":"2024-03-30T00:34:39.064138+0000","last_undegraded":"2024-03-30T00:34:39.064138+0000","last_fullsized":"2024-03-30T00:34:39.064138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:35:05.898973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260589+0000","last_change":"2024-03-30T00:33:15.678089+0000","last_active":"2024-03-30T00:34:09.260589+0000","last_peered":"2024-03-30T00:34:09.260589+0000","last_clean":"2024-03-30T00:34:09.260589+0000","last_became_active":"2024-03-30T00:33:15.677254+0000","last_became_peered":"2024-03-30T00:33:15.677254+0000","last_unstale":"2024-03-30T00:34:09.260589+0000","last_undegraded":"2024-03-30T00:34:09.260589+0000","last_fullsized":"2024-03-30T00:34:09.260589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:45.922652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.305494+0000","last_change":"2024-03-30T00:34:10.305494+0000","last_active":"2024-03-30T00:34:10.305494+0000","last_peered":"2024-03-30T00:34:10.305494+0000","last_clean":"2024-03-30T00:34:10.305494+0000","last_became_active":"2024-03-30T00:34:10.305141+0000","last_became_peered":"2024-03-30T00:34:10.305141+0000","last_unstale":"2024-03-30T00:34:10.305494+0000","last_undegraded":"2024-03-30T00:34:10.305494+0000","last_fullsized":"2024-03-30T00:34:10.305494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:25:18.167062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,8],"acting":[0,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.058589+0000","last_change":"2024-03-30T00:33:15.681237+0000","last_active":"2024-03-30T00:34:39.058589+0000","last_peered":"2024-03-30T00:34:39.058589+0000","last_clean":"2024-03-30T00:34:39.058589+0000","last_became_active":"2024-03-30T00:33:15.680669+0000","last_became_peered":"2024-03-30T00:33:15.680669+0000","last_unstale":"2024-03-30T00:34:39.058589+0000","last_undegraded":"2024-03-30T00:34:39.058589+0000","last_fullsized":"2024-03-30T00:34:39.058589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:59.300262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.037005+0000","last_change":"2024-03-30T00:33:15.678825+0000","last_active":"2024-03-30T00:34:39.037005+0000","last_peered":"2024-03-30T00:34:39.037005+0000","last_clean":"2024-03-30T00:34:39.037005+0000","last_became_active":"2024-03-30T00:33:15.676864+0000","last_became_peered":"2024-03-30T00:33:15.676864+0000","last_unstale":"2024-03-30T00:34:39.037005+0000","last_undegraded":"2024-03-30T00:34:39.037005+0000","last_fullsized":"2024-03-30T00:34:39.037005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:12:09.724441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.060702+0000","last_change":"2024-03-30T00:33:15.681866+0000","last_active":"2024-03-30T00:34:39.060702+0000","last_peered":"2024-03-30T00:34:39.060702+0000","last_clean":"2024-03-30T00:34:39.060702+0000","last_became_active":"2024-03-30T00:33:15.678118+0000","last_became_peered":"2024-03-30T00:33:15.678118+0000","last_unstale":"2024-03-30T00:34:39.060702+0000","last_undegraded":"2024-03-30T00:34:39.060702+0000","last_fullsized":"2024-03-30T00:34:39.060702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:24.433027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051906+0000","last_change":"2024-03-30T00:33:15.675999+0000","last_active":"2024-03-30T00:34:39.051906+0000","last_peered":"2024-03-30T00:34:39.051906+0000","last_clean":"2024-03-30T00:34:39.051906+0000","last_became_active":"2024-03-30T00:33:15.675319+0000","last_became_peered":"2024-03-30T00:33:15.675319+0000","last_unstale":"2024-03-30T00:34:39.051906+0000","last_undegraded":"2024-03-30T00:34:39.051906+0000","last_fullsized":"2024-03-30T00:34:39.051906+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:42.668747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257311+0000","last_change":"2024-03-30T00:33:15.679211+0000","last_active":"2024-03-30T00:34:09.257311+0000","last_peered":"2024-03-30T00:34:09.257311+0000","last_clean":"2024-03-30T00:34:09.257311+0000","last_became_active":"2024-03-30T00:33:15.675061+0000","last_became_peered":"2024-03-30T00:33:15.675061+0000","last_unstale":"2024-03-30T00:34:09.257311+0000","last_undegraded":"2024-03-30T00:34:09.257311+0000","last_fullsized":"2024-03-30T00:34:09.257311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:43.973542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064325+0000","last_change":"2024-03-30T00:33:15.680842+0000","last_active":"2024-03-30T00:34:39.064325+0000","last_peered":"2024-03-30T00:34:39.064325+0000","last_clean":"2024-03-30T00:34:39.064325+0000","last_became_active":"2024-03-30T00:33:15.680307+0000","last_became_peered":"2024-03-30T00:33:15.680307+0000","last_unstale":"2024-03-30T00:34:39.064325+0000","last_undegraded":"2024-03-30T00:34:39.064325+0000","last_fullsized":"2024-03-30T00:34:39.064325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:59:40.331779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.060850+0000","last_change":"2024-03-30T00:33:15.692397+0000","last_active":"2024-03-30T00:34:39.060850+0000","last_peered":"2024-03-30T00:34:39.060850+0000","last_clean":"2024-03-30T00:34:39.060850+0000","last_became_active":"2024-03-30T00:33:15.691226+0000","last_became_peered":"2024-03-30T00:33:15.691226+0000","last_unstale":"2024-03-30T00:34:39.060850+0000","last_undegraded":"2024-03-30T00:34:39.060850+0000","last_fullsized":"2024-03-30T00:34:39.060850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:48.816644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.265637+0000","last_change":"2024-03-30T00:33:15.688417+0000","last_active":"2024-03-30T00:34:10.265637+0000","last_peered":"2024-03-30T00:34:10.265637+0000","last_clean":"2024-03-30T00:34:10.265637+0000","last_became_active":"2024-03-30T00:33:15.687755+0000","last_became_peered":"2024-03-30T00:33:15.687755+0000","last_unstale":"2024-03-30T00:34:10.265637+0000","last_undegraded":"2024-03-30T00:34:10.265637+0000","last_fullsized":"2024-03-30T00:34:10.265637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:30:19.782280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.061117+0000","last_change":"2024-03-30T00:33:15.681942+0000","last_active":"2024-03-30T00:34:39.061117+0000","last_peered":"2024-03-30T00:34:39.061117+0000","last_clean":"2024-03-30T00:34:39.061117+0000","last_became_active":"2024-03-30T00:33:15.679854+0000","last_became_peered":"2024-03-30T00:33:15.679854+0000","last_unstale":"2024-03-30T00:34:39.061117+0000","last_undegraded":"2024-03-30T00:34:39.061117+0000","last_fullsized":"2024-03-30T00:34:39.061117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:19.771959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264276+0000","last_change":"2024-03-30T00:33:15.682968+0000","last_active":"2024-03-30T00:34:09.264276+0000","last_peered":"2024-03-30T00:34:09.264276+0000","last_clean":"2024-03-30T00:34:09.264276+0000","last_became_active":"2024-03-30T00:33:15.681216+0000","last_became_peered":"2024-03-30T00:33:15.681216+0000","last_unstale":"2024-03-30T00:34:09.264276+0000","last_undegraded":"2024-03-30T00:34:09.264276+0000","last_fullsized":"2024-03-30T00:34:09.264276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:50.038843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260360+0000","last_change":"2024-03-30T00:33:15.677727+0000","last_active":"2024-03-30T00:34:09.260360+0000","last_peered":"2024-03-30T00:34:09.260360+0000","last_clean":"2024-03-30T00:34:09.260360+0000","last_became_active":"2024-03-30T00:33:15.676758+0000","last_became_peered":"2024-03-30T00:33:15.676758+0000","last_unstale":"2024-03-30T00:34:09.260360+0000","last_undegraded":"2024-03-30T00:34:09.260360+0000","last_fullsized":"2024-03-30T00:34:09.260360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:34.745920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260097+0000","last_change":"2024-03-30T00:33:15.684545+0000","last_active":"2024-03-30T00:34:09.260097+0000","last_peered":"2024-03-30T00:34:09.260097+0000","last_clean":"2024-03-30T00:34:09.260097+0000","last_became_active":"2024-03-30T00:33:15.682989+0000","last_became_peered":"2024-03-30T00:33:15.682989+0000","last_unstale":"2024-03-30T00:34:09.260097+0000","last_undegraded":"2024-03-30T00:34:09.260097+0000","last_fullsized":"2024-03-30T00:34:09.260097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:11.120981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.067066+0000","last_change":"2024-03-30T00:33:15.678122+0000","last_active":"2024-03-30T00:34:39.067066+0000","last_peered":"2024-03-30T00:34:39.067066+0000","last_clean":"2024-03-30T00:34:39.067066+0000","last_became_active":"2024-03-30T00:33:15.677681+0000","last_became_peered":"2024-03-30T00:33:15.677681+0000","last_unstale":"2024-03-30T00:34:39.067066+0000","last_undegraded":"2024-03-30T00:34:39.067066+0000","last_fullsized":"2024-03-30T00:34:39.067066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:50.454125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064339+0000","last_change":"2024-03-30T00:33:15.687689+0000","last_active":"2024-03-30T00:34:39.064339+0000","last_peered":"2024-03-30T00:34:39.064339+0000","last_clean":"2024-03-30T00:34:39.064339+0000","last_became_active":"2024-03-30T00:33:15.686009+0000","last_became_peered":"2024-03-30T00:33:15.686009+0000","last_unstale":"2024-03-30T00:34:39.064339+0000","last_undegraded":"2024-03-30T00:34:39.064339+0000","last_fullsized":"2024-03-30T00:34:39.064339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:17.234749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062258+0000","last_change":"2024-03-30T00:33:15.686053+0000","last_active":"2024-03-30T00:34:39.062258+0000","last_peered":"2024-03-30T00:34:39.062258+0000","last_clean":"2024-03-30T00:34:39.062258+0000","last_became_active":"2024-03-30T00:33:15.685723+0000","last_became_peered":"2024-03-30T00:33:15.685723+0000","last_unstale":"2024-03-30T00:34:39.062258+0000","last_undegraded":"2024-03-30T00:34:39.062258+0000","last_fullsized":"2024-03-30T00:34:39.062258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:27.322834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.050110+0000","last_change":"2024-03-30T00:33:15.674783+0000","last_active":"2024-03-30T00:34:39.050110+0000","last_peered":"2024-03-30T00:34:39.050110+0000","last_clean":"2024-03-30T00:34:39.050110+0000","last_became_active":"2024-03-30T00:33:15.674152+0000","last_became_peered":"2024-03-30T00:33:15.674152+0000","last_unstale":"2024-03-30T00:34:39.050110+0000","last_undegraded":"2024-03-30T00:34:39.050110+0000","last_fullsized":"2024-03-30T00:34:39.050110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:39.517240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.251874+0000","last_change":"2024-03-30T00:33:15.172217+0000","last_active":"2024-03-30T00:34:09.251874+0000","last_peered":"2024-03-30T00:34:09.251874+0000","last_clean":"2024-03-30T00:34:09.251874+0000","last_became_active":"2024-03-30T00:33:15.171718+0000","last_became_peered":"2024-03-30T00:33:15.171718+0000","last_unstale":"2024-03-30T00:34:09.251874+0000","last_undegraded":"2024-03-30T00:34:09.251874+0000","last_fullsized":"2024-03-30T00:34:09.251874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:11.691572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258046+0000","last_change":"2024-03-30T00:33:15.669937+0000","last_active":"2024-03-30T00:34:09.258046+0000","last_peered":"2024-03-30T00:34:09.258046+0000","last_clean":"2024-03-30T00:34:09.258046+0000","last_became_active":"2024-03-30T00:33:15.668704+0000","last_became_peered":"2024-03-30T00:33:15.668704+0000","last_unstale":"2024-03-30T00:34:09.258046+0000","last_undegraded":"2024-03-30T00:34:09.258046+0000","last_fullsized":"2024-03-30T00:34:09.258046+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:23.914755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258142+0000","last_change":"2024-03-30T00:33:15.173663+0000","last_active":"2024-03-30T00:34:09.258142+0000","last_peered":"2024-03-30T00:34:09.258142+0000","last_clean":"2024-03-30T00:34:09.258142+0000","last_became_active":"2024-03-30T00:33:15.173288+0000","last_became_peered":"2024-03-30T00:33:15.173288+0000","last_unstale":"2024-03-30T00:34:09.258142+0000","last_undegraded":"2024-03-30T00:34:09.258142+0000","last_fullsized":"2024-03-30T00:34:09.258142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.435142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260485+0000","last_change":"2024-03-30T00:33:15.686757+0000","last_active":"2024-03-30T00:34:09.260485+0000","last_peered":"2024-03-30T00:34:09.260485+0000","last_clean":"2024-03-30T00:34:09.260485+0000","last_became_active":"2024-03-30T00:33:15.686306+0000","last_became_peered":"2024-03-30T00:33:15.686306+0000","last_unstale":"2024-03-30T00:34:09.260485+0000","last_undegraded":"2024-03-30T00:34:09.260485+0000","last_fullsized":"2024-03-30T00:34:09.260485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:50:21.048212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254756+0000","last_change":"2024-03-30T00:33:15.173402+0000","last_active":"2024-03-30T00:34:09.254756+0000","last_peered":"2024-03-30T00:34:09.254756+0000","last_clean":"2024-03-30T00:34:09.254756+0000","last_became_active":"2024-03-30T00:33:15.172995+0000","last_became_peered":"2024-03-30T00:33:15.172995+0000","last_unstale":"2024-03-30T00:34:09.254756+0000","last_undegraded":"2024-03-30T00:34:09.254756+0000","last_fullsized":"2024-03-30T00:34:09.254756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:25.694792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065176+0000","last_change":"2024-03-30T00:33:15.688292+0000","last_active":"2024-03-30T00:34:39.065176+0000","last_peered":"2024-03-30T00:34:39.065176+0000","last_clean":"2024-03-30T00:34:39.065176+0000","last_became_active":"2024-03-30T00:33:15.682722+0000","last_became_peered":"2024-03-30T00:33:15.682722+0000","last_unstale":"2024-03-30T00:34:39.065176+0000","last_undegraded":"2024-03-30T00:34:39.065176+0000","last_fullsized":"2024-03-30T00:34:39.065176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:52:10.074656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254648+0000","last_change":"2024-03-30T00:33:15.162316+0000","last_active":"2024-03-30T00:34:09.254648+0000","last_peered":"2024-03-30T00:34:09.254648+0000","last_clean":"2024-03-30T00:34:09.254648+0000","last_became_active":"2024-03-30T00:33:15.161805+0000","last_became_peered":"2024-03-30T00:33:15.161805+0000","last_unstale":"2024-03-30T00:34:09.254648+0000","last_undegraded":"2024-03-30T00:34:09.254648+0000","last_fullsized":"2024-03-30T00:34:09.254648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:26:16.470245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062423+0000","last_change":"2024-03-30T00:33:15.675442+0000","last_active":"2024-03-30T00:34:39.062423+0000","last_peered":"2024-03-30T00:34:39.062423+0000","last_clean":"2024-03-30T00:34:39.062423+0000","last_became_active":"2024-03-30T00:33:15.675068+0000","last_became_peered":"2024-03-30T00:33:15.675068+0000","last_unstale":"2024-03-30T00:34:39.062423+0000","last_undegraded":"2024-03-30T00:34:39.062423+0000","last_fullsized":"2024-03-30T00:34:39.062423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:36:31.353262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":105,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:40.249081+0000","last_change":"2024-03-30T00:33:15.172087+0000","last_active":"2024-03-30T00:34:40.249081+0000","last_peered":"2024-03-30T00:34:40.249081+0000","last_clean":"2024-03-30T00:34:40.249081+0000","last_became_active":"2024-03-30T00:33:15.171768+0000","last_became_peered":"2024-03-30T00:33:15.171768+0000","last_unstale":"2024-03-30T00:34:40.249081+0000","last_undegraded":"2024-03-30T00:34:40.249081+0000","last_fullsized":"2024-03-30T00:34:40.249081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:17.198995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1070,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.042702+0000","last_change":"2024-03-30T00:33:15.665219+0000","last_active":"2024-03-30T00:34:39.042702+0000","last_peered":"2024-03-30T00:34:39.042702+0000","last_clean":"2024-03-30T00:34:39.042702+0000","last_became_active":"2024-03-30T00:33:15.664935+0000","last_became_peered":"2024-03-30T00:33:15.664935+0000","last_unstale":"2024-03-30T00:34:39.042702+0000","last_undegraded":"2024-03-30T00:34:39.042702+0000","last_fullsized":"2024-03-30T00:34:39.042702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:37.123937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254914+0000","last_change":"2024-03-30T00:33:15.165668+0000","last_active":"2024-03-30T00:34:09.254914+0000","last_peered":"2024-03-30T00:34:09.254914+0000","last_clean":"2024-03-30T00:34:09.254914+0000","last_became_active":"2024-03-30T00:33:15.164637+0000","last_became_peered":"2024-03-30T00:33:15.164637+0000","last_unstale":"2024-03-30T00:34:09.254914+0000","last_undegraded":"2024-03-30T00:34:09.254914+0000","last_fullsized":"2024-03-30T00:34:09.254914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:47:59.389414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.048514+0000","last_change":"2024-03-30T00:34:10.309779+0000","last_active":"2024-03-30T00:34:39.048514+0000","last_peered":"2024-03-30T00:34:39.048514+0000","last_clean":"2024-03-30T00:34:39.048514+0000","last_became_active":"2024-03-30T00:34:10.309272+0000","last_became_peered":"2024-03-30T00:34:10.309272+0000","last_unstale":"2024-03-30T00:34:39.048514+0000","last_undegraded":"2024-03-30T00:34:39.048514+0000","last_fullsized":"2024-03-30T00:34:39.048514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:22:54.666608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264857+0000","last_change":"2024-03-30T00:33:15.177269+0000","last_active":"2024-03-30T00:34:09.264857+0000","last_peered":"2024-03-30T00:34:09.264857+0000","last_clean":"2024-03-30T00:34:09.264857+0000","last_became_active":"2024-03-30T00:33:15.176937+0000","last_became_peered":"2024-03-30T00:33:15.176937+0000","last_unstale":"2024-03-30T00:34:09.264857+0000","last_undegraded":"2024-03-30T00:34:09.264857+0000","last_fullsized":"2024-03-30T00:34:09.264857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:44.658775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257873+0000","last_change":"2024-03-30T00:33:15.675832+0000","last_active":"2024-03-30T00:34:09.257873+0000","last_peered":"2024-03-30T00:34:09.257873+0000","last_clean":"2024-03-30T00:34:09.257873+0000","last_became_active":"2024-03-30T00:33:15.675481+0000","last_became_peered":"2024-03-30T00:33:15.675481+0000","last_unstale":"2024-03-30T00:34:09.257873+0000","last_undegraded":"2024-03-30T00:34:09.257873+0000","last_fullsized":"2024-03-30T00:34:09.257873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:11.638056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258682+0000","last_change":"2024-03-30T00:33:15.171238+0000","last_active":"2024-03-30T00:34:09.258682+0000","last_peered":"2024-03-30T00:34:09.258682+0000","last_clean":"2024-03-30T00:34:09.258682+0000","last_became_active":"2024-03-30T00:33:15.170931+0000","last_became_peered":"2024-03-30T00:33:15.170931+0000","last_unstale":"2024-03-30T00:34:09.258682+0000","last_undegraded":"2024-03-30T00:34:09.258682+0000","last_fullsized":"2024-03-30T00:34:09.258682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:43:50.178363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.060489+0000","last_change":"2024-03-30T00:33:15.676014+0000","last_active":"2024-03-30T00:34:39.060489+0000","last_peered":"2024-03-30T00:34:39.060489+0000","last_clean":"2024-03-30T00:34:39.060489+0000","last_became_active":"2024-03-30T00:33:15.675553+0000","last_became_peered":"2024-03-30T00:33:15.675553+0000","last_unstale":"2024-03-30T00:34:39.060489+0000","last_undegraded":"2024-03-30T00:34:39.060489+0000","last_fullsized":"2024-03-30T00:34:39.060489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:17.248242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":99,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.246658+0000","last_change":"2024-03-30T00:33:15.177033+0000","last_active":"2024-03-30T00:34:39.246658+0000","last_peered":"2024-03-30T00:34:39.246658+0000","last_clean":"2024-03-30T00:34:39.246658+0000","last_became_active":"2024-03-30T00:33:15.176602+0000","last_became_peered":"2024-03-30T00:33:15.176602+0000","last_unstale":"2024-03-30T00:34:39.246658+0000","last_undegraded":"2024-03-30T00:34:39.246658+0000","last_fullsized":"2024-03-30T00:34:39.246658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:10.804704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065450+0000","last_change":"2024-03-30T00:33:15.683388+0000","last_active":"2024-03-30T00:34:39.065450+0000","last_peered":"2024-03-30T00:34:39.065450+0000","last_clean":"2024-03-30T00:34:39.065450+0000","last_became_active":"2024-03-30T00:33:15.682976+0000","last_became_peered":"2024-03-30T00:33:15.682976+0000","last_unstale":"2024-03-30T00:34:39.065450+0000","last_undegraded":"2024-03-30T00:34:39.065450+0000","last_fullsized":"2024-03-30T00:34:39.065450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:50:34.907265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.256323+0000","last_change":"2024-03-30T00:33:15.167819+0000","last_active":"2024-03-30T00:34:09.256323+0000","last_peered":"2024-03-30T00:34:09.256323+0000","last_clean":"2024-03-30T00:34:09.256323+0000","last_became_active":"2024-03-30T00:33:15.167405+0000","last_became_peered":"2024-03-30T00:33:15.167405+0000","last_unstale":"2024-03-30T00:34:09.256323+0000","last_undegraded":"2024-03-30T00:34:09.256323+0000","last_fullsized":"2024-03-30T00:34:09.256323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:37.793792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260394+0000","last_change":"2024-03-30T00:33:15.690907+0000","last_active":"2024-03-30T00:34:09.260394+0000","last_peered":"2024-03-30T00:34:09.260394+0000","last_clean":"2024-03-30T00:34:09.260394+0000","last_became_active":"2024-03-30T00:33:15.690046+0000","last_became_peered":"2024-03-30T00:33:15.690046+0000","last_unstale":"2024-03-30T00:34:09.260394+0000","last_undegraded":"2024-03-30T00:34:09.260394+0000","last_fullsized":"2024-03-30T00:34:09.260394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:24.319685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258463+0000","last_change":"2024-03-30T00:33:15.166633+0000","last_active":"2024-03-30T00:34:09.258463+0000","last_peered":"2024-03-30T00:34:09.258463+0000","last_clean":"2024-03-30T00:34:09.258463+0000","last_became_active":"2024-03-30T00:33:15.166308+0000","last_became_peered":"2024-03-30T00:33:15.166308+0000","last_unstale":"2024-03-30T00:34:09.258463+0000","last_undegraded":"2024-03-30T00:34:09.258463+0000","last_fullsized":"2024-03-30T00:34:09.258463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:53.102872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065783+0000","last_change":"2024-03-30T00:33:15.691880+0000","last_active":"2024-03-30T00:34:39.065783+0000","last_peered":"2024-03-30T00:34:39.065783+0000","last_clean":"2024-03-30T00:34:39.065783+0000","last_became_active":"2024-03-30T00:33:15.690780+0000","last_became_peered":"2024-03-30T00:33:15.690780+0000","last_unstale":"2024-03-30T00:34:39.065783+0000","last_undegraded":"2024-03-30T00:34:39.065783+0000","last_fullsized":"2024-03-30T00:34:39.065783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:06.532709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":226,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:40.260983+0000","last_change":"2024-03-30T00:33:15.164287+0000","last_active":"2024-03-30T00:34:40.260983+0000","last_peered":"2024-03-30T00:34:40.260983+0000","last_clean":"2024-03-30T00:34:40.260983+0000","last_became_active":"2024-03-30T00:33:15.163557+0000","last_became_peered":"2024-03-30T00:33:15.163557+0000","last_unstale":"2024-03-30T00:34:40.260983+0000","last_undegraded":"2024-03-30T00:34:40.260983+0000","last_fullsized":"2024-03-30T00:34:40.260983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:41.117936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":174,"num_read_kb":154,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062751+0000","last_change":"2024-03-30T00:33:15.682841+0000","last_active":"2024-03-30T00:34:39.062751+0000","last_peered":"2024-03-30T00:34:39.062751+0000","last_clean":"2024-03-30T00:34:39.062751+0000","last_became_active":"2024-03-30T00:33:15.682237+0000","last_became_peered":"2024-03-30T00:33:15.682237+0000","last_unstale":"2024-03-30T00:34:39.062751+0000","last_undegraded":"2024-03-30T00:34:39.062751+0000","last_fullsized":"2024-03-30T00:34:39.062751+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:59:05.719216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.262163+0000","last_change":"2024-03-30T00:33:15.165193+0000","last_active":"2024-03-30T00:34:09.262163+0000","last_peered":"2024-03-30T00:34:09.262163+0000","last_clean":"2024-03-30T00:34:09.262163+0000","last_became_active":"2024-03-30T00:33:15.161741+0000","last_became_peered":"2024-03-30T00:33:15.161741+0000","last_unstale":"2024-03-30T00:34:09.262163+0000","last_undegraded":"2024-03-30T00:34:09.262163+0000","last_fullsized":"2024-03-30T00:34:09.262163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:22:11.891064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264117+0000","last_change":"2024-03-30T00:33:15.680929+0000","last_active":"2024-03-30T00:34:09.264117+0000","last_peered":"2024-03-30T00:34:09.264117+0000","last_clean":"2024-03-30T00:34:09.264117+0000","last_became_active":"2024-03-30T00:33:15.676671+0000","last_became_peered":"2024-03-30T00:33:15.676671+0000","last_unstale":"2024-03-30T00:34:09.264117+0000","last_undegraded":"2024-03-30T00:34:09.264117+0000","last_fullsized":"2024-03-30T00:34:09.264117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:28:33.819135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257972+0000","last_change":"2024-03-30T00:33:15.168029+0000","last_active":"2024-03-30T00:34:09.257972+0000","last_peered":"2024-03-30T00:34:09.257972+0000","last_clean":"2024-03-30T00:34:09.257972+0000","last_became_active":"2024-03-30T00:33:15.167636+0000","last_became_peered":"2024-03-30T00:33:15.167636+0000","last_unstale":"2024-03-30T00:34:09.257972+0000","last_undegraded":"2024-03-30T00:34:09.257972+0000","last_fullsized":"2024-03-30T00:34:09.257972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:25:47.834696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.058578+0000","last_change":"2024-03-30T00:33:15.687977+0000","last_active":"2024-03-30T00:34:39.058578+0000","last_peered":"2024-03-30T00:34:39.058578+0000","last_clean":"2024-03-30T00:34:39.058578+0000","last_became_active":"2024-03-30T00:33:15.687491+0000","last_became_peered":"2024-03-30T00:33:15.687491+0000","last_unstale":"2024-03-30T00:34:39.058578+0000","last_undegraded":"2024-03-30T00:34:39.058578+0000","last_fullsized":"2024-03-30T00:34:39.058578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:08:55.568039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":100,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:38.257303+0000","last_change":"2024-03-30T00:33:15.171826+0000","last_active":"2024-03-30T00:34:38.257303+0000","last_peered":"2024-03-30T00:34:38.257303+0000","last_clean":"2024-03-30T00:34:38.257303+0000","last_became_active":"2024-03-30T00:33:15.170749+0000","last_became_peered":"2024-03-30T00:33:15.170749+0000","last_unstale":"2024-03-30T00:34:38.257303+0000","last_undegraded":"2024-03-30T00:34:38.257303+0000","last_fullsized":"2024-03-30T00:34:38.257303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:57:52.034530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062584+0000","last_change":"2024-03-30T00:34:10.304456+0000","last_active":"2024-03-30T00:34:39.062584+0000","last_peered":"2024-03-30T00:34:39.062584+0000","last_clean":"2024-03-30T00:34:39.062584+0000","last_became_active":"2024-03-30T00:34:10.303934+0000","last_became_peered":"2024-03-30T00:34:10.303934+0000","last_unstale":"2024-03-30T00:34:39.062584+0000","last_undegraded":"2024-03-30T00:34:39.062584+0000","last_fullsized":"2024-03-30T00:34:39.062584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:48:07.792123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254268+0000","last_change":"2024-03-30T00:33:15.171016+0000","last_active":"2024-03-30T00:34:09.254268+0000","last_peered":"2024-03-30T00:34:09.254268+0000","last_clean":"2024-03-30T00:34:09.254268+0000","last_became_active":"2024-03-30T00:33:15.170058+0000","last_became_peered":"2024-03-30T00:33:15.170058+0000","last_unstale":"2024-03-30T00:34:09.254268+0000","last_undegraded":"2024-03-30T00:34:09.254268+0000","last_fullsized":"2024-03-30T00:34:09.254268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:07:39.910286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051568+0000","last_change":"2024-03-30T00:34:10.305753+0000","last_active":"2024-03-30T00:34:39.051568+0000","last_peered":"2024-03-30T00:34:39.051568+0000","last_clean":"2024-03-30T00:34:39.051568+0000","last_became_active":"2024-03-30T00:34:10.305339+0000","last_became_peered":"2024-03-30T00:34:10.305339+0000","last_unstale":"2024-03-30T00:34:39.051568+0000","last_undegraded":"2024-03-30T00:34:39.051568+0000","last_fullsized":"2024-03-30T00:34:39.051568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:41.611151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254799+0000","last_change":"2024-03-30T00:33:15.163633+0000","last_active":"2024-03-30T00:34:09.254799+0000","last_peered":"2024-03-30T00:34:09.254799+0000","last_clean":"2024-03-30T00:34:09.254799+0000","last_became_active":"2024-03-30T00:33:15.161339+0000","last_became_peered":"2024-03-30T00:33:15.161339+0000","last_unstale":"2024-03-30T00:34:09.254799+0000","last_undegraded":"2024-03-30T00:34:09.254799+0000","last_fullsized":"2024-03-30T00:34:09.254799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:15:25.194650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.050235+0000","last_change":"2024-03-30T00:34:10.269652+0000","last_active":"2024-03-30T00:34:39.050235+0000","last_peered":"2024-03-30T00:34:39.050235+0000","last_clean":"2024-03-30T00:34:39.050235+0000","last_became_active":"2024-03-30T00:34:10.269267+0000","last_became_peered":"2024-03-30T00:34:10.269267+0000","last_unstale":"2024-03-30T00:34:39.050235+0000","last_undegraded":"2024-03-30T00:34:39.050235+0000","last_fullsized":"2024-03-30T00:34:39.050235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:13.724322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.251412+0000","last_change":"2024-03-30T00:33:15.177273+0000","last_active":"2024-03-30T00:34:09.251412+0000","last_peered":"2024-03-30T00:34:09.251412+0000","last_clean":"2024-03-30T00:34:09.251412+0000","last_became_active":"2024-03-30T00:33:15.175333+0000","last_became_peered":"2024-03-30T00:33:15.175333+0000","last_unstale":"2024-03-30T00:34:09.251412+0000","last_undegraded":"2024-03-30T00:34:09.251412+0000","last_fullsized":"2024-03-30T00:34:09.251412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:27:03.305671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.046660+0000","last_change":"2024-03-30T00:33:15.674746+0000","last_active":"2024-03-30T00:34:39.046660+0000","last_peered":"2024-03-30T00:34:39.046660+0000","last_clean":"2024-03-30T00:34:39.046660+0000","last_became_active":"2024-03-30T00:33:15.674388+0000","last_became_peered":"2024-03-30T00:33:15.674388+0000","last_unstale":"2024-03-30T00:34:39.046660+0000","last_undegraded":"2024-03-30T00:34:39.046660+0000","last_fullsized":"2024-03-30T00:34:39.046660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:21:15.024181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.256443+0000","last_change":"2024-03-30T00:33:15.173640+0000","last_active":"2024-03-30T00:34:09.256443+0000","last_peered":"2024-03-30T00:34:09.256443+0000","last_clean":"2024-03-30T00:34:09.256443+0000","last_became_active":"2024-03-30T00:33:15.173104+0000","last_became_peered":"2024-03-30T00:33:15.173104+0000","last_unstale":"2024-03-30T00:34:09.256443+0000","last_undegraded":"2024-03-30T00:34:09.256443+0000","last_fullsized":"2024-03-30T00:34:09.256443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:20:14.880730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.056823+0000","last_change":"2024-03-30T00:33:15.677824+0000","last_active":"2024-03-30T00:34:39.056823+0000","last_peered":"2024-03-30T00:34:39.056823+0000","last_clean":"2024-03-30T00:34:39.056823+0000","last_became_active":"2024-03-30T00:33:15.677435+0000","last_became_peered":"2024-03-30T00:33:15.677435+0000","last_unstale":"2024-03-30T00:34:39.056823+0000","last_undegraded":"2024-03-30T00:34:39.056823+0000","last_fullsized":"2024-03-30T00:34:39.056823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:22.992989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.044985+0000","last_change":"2024-03-30T00:34:10.304054+0000","last_active":"2024-03-30T00:34:39.044985+0000","last_peered":"2024-03-30T00:34:39.044985+0000","last_clean":"2024-03-30T00:34:39.044985+0000","last_became_active":"2024-03-30T00:34:10.303653+0000","last_became_peered":"2024-03-30T00:34:10.303653+0000","last_unstale":"2024-03-30T00:34:39.044985+0000","last_undegraded":"2024-03-30T00:34:39.044985+0000","last_fullsized":"2024-03-30T00:34:39.044985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:12:32.378165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":96,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:36.251806+0000","last_change":"2024-03-30T00:33:15.173264+0000","last_active":"2024-03-30T00:34:36.251806+0000","last_peered":"2024-03-30T00:34:36.251806+0000","last_clean":"2024-03-30T00:34:36.251806+0000","last_became_active":"2024-03-30T00:33:15.172932+0000","last_became_peered":"2024-03-30T00:33:15.172932+0000","last_unstale":"2024-03-30T00:34:36.251806+0000","last_undegraded":"2024-03-30T00:34:36.251806+0000","last_fullsized":"2024-03-30T00:34:36.251806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:18:25.079284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257668+0000","last_change":"2024-03-30T00:33:15.686606+0000","last_active":"2024-03-30T00:34:09.257668+0000","last_peered":"2024-03-30T00:34:09.257668+0000","last_clean":"2024-03-30T00:34:09.257668+0000","last_became_active":"2024-03-30T00:33:15.686076+0000","last_became_peered":"2024-03-30T00:33:15.686076+0000","last_unstale":"2024-03-30T00:34:09.257668+0000","last_undegraded":"2024-03-30T00:34:09.257668+0000","last_fullsized":"2024-03-30T00:34:09.257668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:46.962659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.261642+0000","last_change":"2024-03-30T00:33:15.177263+0000","last_active":"2024-03-30T00:34:09.261642+0000","last_peered":"2024-03-30T00:34:09.261642+0000","last_clean":"2024-03-30T00:34:09.261642+0000","last_became_active":"2024-03-30T00:33:15.176743+0000","last_became_peered":"2024-03-30T00:33:15.176743+0000","last_unstale":"2024-03-30T00:34:09.261642+0000","last_undegraded":"2024-03-30T00:34:09.261642+0000","last_fullsized":"2024-03-30T00:34:09.261642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:56:47.491371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.256985+0000","last_change":"2024-03-30T00:32:12.678730+0000","last_active":"2024-03-30T00:34:09.256985+0000","last_peered":"2024-03-30T00:34:09.256985+0000","last_clean":"2024-03-30T00:34:09.256985+0000","last_became_active":"2024-03-30T00:32:12.678256+0000","last_became_peered":"2024-03-30T00:32:12.678256+0000","last_unstale":"2024-03-30T00:34:09.256985+0000","last_undegraded":"2024-03-30T00:34:09.256985+0000","last_fullsized":"2024-03-30T00:34:09.256985+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_clean_scrub_stamp":"2024-03-30T00:29:17.768535+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:16.229072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.304196+0000","last_change":"2024-03-30T00:34:10.304196+0000","last_active":"2024-03-30T00:34:10.304196+0000","last_peered":"2024-03-30T00:34:10.304196+0000","last_clean":"2024-03-30T00:34:10.304196+0000","last_became_active":"2024-03-30T00:34:10.303740+0000","last_became_peered":"2024-03-30T00:34:10.303740+0000","last_unstale":"2024-03-30T00:34:10.304196+0000","last_undegraded":"2024-03-30T00:34:10.304196+0000","last_fullsized":"2024-03-30T00:34:10.304196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:25:10.417989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257070+0000","last_change":"2024-03-30T00:33:15.167490+0000","last_active":"2024-03-30T00:34:09.257070+0000","last_peered":"2024-03-30T00:34:09.257070+0000","last_clean":"2024-03-30T00:34:09.257070+0000","last_became_active":"2024-03-30T00:33:15.166888+0000","last_became_peered":"2024-03-30T00:33:15.166888+0000","last_unstale":"2024-03-30T00:34:09.257070+0000","last_undegraded":"2024-03-30T00:34:09.257070+0000","last_fullsized":"2024-03-30T00:34:09.257070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:00:32.571281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254643+0000","last_change":"2024-03-30T00:33:15.178388+0000","last_active":"2024-03-30T00:34:09.254643+0000","last_peered":"2024-03-30T00:34:09.254643+0000","last_clean":"2024-03-30T00:34:09.254643+0000","last_became_active":"2024-03-30T00:33:15.178160+0000","last_became_peered":"2024-03-30T00:33:15.178160+0000","last_unstale":"2024-03-30T00:34:09.254643+0000","last_undegraded":"2024-03-30T00:34:09.254643+0000","last_fullsized":"2024-03-30T00:34:09.254643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:28:51.567598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264257+0000","last_change":"2024-03-30T00:33:15.675426+0000","last_active":"2024-03-30T00:34:09.264257+0000","last_peered":"2024-03-30T00:34:09.264257+0000","last_clean":"2024-03-30T00:34:09.264257+0000","last_became_active":"2024-03-30T00:33:15.675083+0000","last_became_peered":"2024-03-30T00:33:15.675083+0000","last_unstale":"2024-03-30T00:34:09.264257+0000","last_undegraded":"2024-03-30T00:34:09.264257+0000","last_fullsized":"2024-03-30T00:34:09.264257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:37:03.086331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.251872+0000","last_change":"2024-03-30T00:33:15.171537+0000","last_active":"2024-03-30T00:34:09.251872+0000","last_peered":"2024-03-30T00:34:09.251872+0000","last_clean":"2024-03-30T00:34:09.251872+0000","last_became_active":"2024-03-30T00:33:15.171128+0000","last_became_peered":"2024-03-30T00:33:15.171128+0000","last_unstale":"2024-03-30T00:34:09.251872+0000","last_undegraded":"2024-03-30T00:34:09.251872+0000","last_fullsized":"2024-03-30T00:34:09.251872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:52.062144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257830+0000","last_change":"2024-03-30T00:33:15.674075+0000","last_active":"2024-03-30T00:34:09.257830+0000","last_peered":"2024-03-30T00:34:09.257830+0000","last_clean":"2024-03-30T00:34:09.257830+0000","last_became_active":"2024-03-30T00:33:15.672400+0000","last_became_peered":"2024-03-30T00:33:15.672400+0000","last_unstale":"2024-03-30T00:34:09.257830+0000","last_undegraded":"2024-03-30T00:34:09.257830+0000","last_fullsized":"2024-03-30T00:34:09.257830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:59.309835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.261775+0000","last_change":"2024-03-30T00:33:15.168245+0000","last_active":"2024-03-30T00:34:09.261775+0000","last_peered":"2024-03-30T00:34:09.261775+0000","last_clean":"2024-03-30T00:34:09.261775+0000","last_became_active":"2024-03-30T00:33:15.167852+0000","last_became_peered":"2024-03-30T00:33:15.167852+0000","last_unstale":"2024-03-30T00:34:09.261775+0000","last_undegraded":"2024-03-30T00:34:09.261775+0000","last_fullsized":"2024-03-30T00:34:09.261775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:33:28.780614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.305258+0000","last_change":"2024-03-30T00:34:10.305258+0000","last_active":"2024-03-30T00:34:10.305258+0000","last_peered":"2024-03-30T00:34:10.305258+0000","last_clean":"2024-03-30T00:34:10.305258+0000","last_became_active":"2024-03-30T00:34:10.304913+0000","last_became_peered":"2024-03-30T00:34:10.304913+0000","last_unstale":"2024-03-30T00:34:10.305258+0000","last_undegraded":"2024-03-30T00:34:10.305258+0000","last_fullsized":"2024-03-30T00:34:10.305258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:29:08.202307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.263955+0000","last_change":"2024-03-30T00:33:15.170052+0000","last_active":"2024-03-30T00:34:09.263955+0000","last_peered":"2024-03-30T00:34:09.263955+0000","last_clean":"2024-03-30T00:34:09.263955+0000","last_became_active":"2024-03-30T00:33:15.169706+0000","last_became_peered":"2024-03-30T00:33:15.169706+0000","last_unstale":"2024-03-30T00:34:09.263955+0000","last_undegraded":"2024-03-30T00:34:09.263955+0000","last_fullsized":"2024-03-30T00:34:09.263955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:32.043071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.014521+0000","last_change":"2024-03-30T00:33:15.667061+0000","last_active":"2024-03-30T00:34:39.014521+0000","last_peered":"2024-03-30T00:34:39.014521+0000","last_clean":"2024-03-30T00:34:39.014521+0000","last_became_active":"2024-03-30T00:33:15.666763+0000","last_became_peered":"2024-03-30T00:33:15.666763+0000","last_unstale":"2024-03-30T00:34:39.014521+0000","last_undegraded":"2024-03-30T00:34:39.014521+0000","last_fullsized":"2024-03-30T00:34:39.014521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:46.558297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257420+0000","last_change":"2024-03-30T00:33:15.176152+0000","last_active":"2024-03-30T00:34:09.257420+0000","last_peered":"2024-03-30T00:34:09.257420+0000","last_clean":"2024-03-30T00:34:09.257420+0000","last_became_active":"2024-03-30T00:33:15.175640+0000","last_became_peered":"2024-03-30T00:33:15.175640+0000","last_unstale":"2024-03-30T00:34:09.257420+0000","last_undegraded":"2024-03-30T00:34:09.257420+0000","last_fullsized":"2024-03-30T00:34:09.257420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:58:11.575577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051405+0000","last_change":"2024-03-30T00:33:15.672957+0000","last_active":"2024-03-30T00:34:39.051405+0000","last_peered":"2024-03-30T00:34:39.051405+0000","last_clean":"2024-03-30T00:34:39.051405+0000","last_became_active":"2024-03-30T00:33:15.668868+0000","last_became_peered":"2024-03-30T00:33:15.668868+0000","last_unstale":"2024-03-30T00:34:39.051405+0000","last_undegraded":"2024-03-30T00:34:39.051405+0000","last_fullsized":"2024-03-30T00:34:39.051405+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:50.342693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.301965+0000","last_change":"2024-03-30T00:33:15.160345+0000","last_active":"2024-03-30T00:34:10.301965+0000","last_peered":"2024-03-30T00:34:10.301965+0000","last_clean":"2024-03-30T00:34:10.301965+0000","last_became_active":"2024-03-30T00:33:15.160026+0000","last_became_peered":"2024-03-30T00:33:15.160026+0000","last_unstale":"2024-03-30T00:34:10.301965+0000","last_undegraded":"2024-03-30T00:34:10.301965+0000","last_fullsized":"2024-03-30T00:34:10.301965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:06:51.177680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257738+0000","last_change":"2024-03-30T00:33:15.674058+0000","last_active":"2024-03-30T00:34:09.257738+0000","last_peered":"2024-03-30T00:34:09.257738+0000","last_clean":"2024-03-30T00:34:09.257738+0000","last_became_active":"2024-03-30T00:33:15.673165+0000","last_became_peered":"2024-03-30T00:33:15.673165+0000","last_unstale":"2024-03-30T00:34:09.257738+0000","last_undegraded":"2024-03-30T00:34:09.257738+0000","last_fullsized":"2024-03-30T00:34:09.257738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:14.869963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254400+0000","last_change":"2024-03-30T00:33:15.173178+0000","last_active":"2024-03-30T00:34:09.254400+0000","last_peered":"2024-03-30T00:34:09.254400+0000","last_clean":"2024-03-30T00:34:09.254400+0000","last_became_active":"2024-03-30T00:33:15.172830+0000","last_became_peered":"2024-03-30T00:33:15.172830+0000","last_unstale":"2024-03-30T00:34:09.254400+0000","last_undegraded":"2024-03-30T00:34:09.254400+0000","last_fullsized":"2024-03-30T00:34:09.254400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:49:09.823855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065387+0000","last_change":"2024-03-30T00:33:15.693485+0000","last_active":"2024-03-30T00:34:39.065387+0000","last_peered":"2024-03-30T00:34:39.065387+0000","last_clean":"2024-03-30T00:34:39.065387+0000","last_became_active":"2024-03-30T00:33:15.693066+0000","last_became_peered":"2024-03-30T00:33:15.693066+0000","last_unstale":"2024-03-30T00:34:39.065387+0000","last_undegraded":"2024-03-30T00:34:39.065387+0000","last_fullsized":"2024-03-30T00:34:39.065387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:38.161331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.262703+0000","last_change":"2024-03-30T00:33:15.170808+0000","last_active":"2024-03-30T00:34:09.262703+0000","last_peered":"2024-03-30T00:34:09.262703+0000","last_clean":"2024-03-30T00:34:09.262703+0000","last_became_active":"2024-03-30T00:33:15.170448+0000","last_became_peered":"2024-03-30T00:33:15.170448+0000","last_unstale":"2024-03-30T00:34:09.262703+0000","last_undegraded":"2024-03-30T00:34:09.262703+0000","last_fullsized":"2024-03-30T00:34:09.262703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:08:28.179041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062015+0000","last_change":"2024-03-30T00:33:15.687619+0000","last_active":"2024-03-30T00:34:39.062015+0000","last_peered":"2024-03-30T00:34:39.062015+0000","last_clean":"2024-03-30T00:34:39.062015+0000","last_became_active":"2024-03-30T00:33:15.687313+0000","last_became_peered":"2024-03-30T00:33:15.687313+0000","last_unstale":"2024-03-30T00:34:39.062015+0000","last_undegraded":"2024-03-30T00:34:39.062015+0000","last_fullsized":"2024-03-30T00:34:39.062015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:36:51.182558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258190+0000","last_change":"2024-03-30T00:33:15.168601+0000","last_active":"2024-03-30T00:34:09.258190+0000","last_peered":"2024-03-30T00:34:09.258190+0000","last_clean":"2024-03-30T00:34:09.258190+0000","last_became_active":"2024-03-30T00:33:15.168065+0000","last_became_peered":"2024-03-30T00:33:15.168065+0000","last_unstale":"2024-03-30T00:34:09.258190+0000","last_undegraded":"2024-03-30T00:34:09.258190+0000","last_fullsized":"2024-03-30T00:34:09.258190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:22:36.477280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.063142+0000","last_change":"2024-03-30T00:33:15.678299+0000","last_active":"2024-03-30T00:34:39.063142+0000","last_peered":"2024-03-30T00:34:39.063142+0000","last_clean":"2024-03-30T00:34:39.063142+0000","last_became_active":"2024-03-30T00:33:15.675499+0000","last_became_peered":"2024-03-30T00:33:15.675499+0000","last_unstale":"2024-03-30T00:34:39.063142+0000","last_undegraded":"2024-03-30T00:34:39.063142+0000","last_fullsized":"2024-03-30T00:34:39.063142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:15.317125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264445+0000","last_change":"2024-03-30T00:33:15.172058+0000","last_active":"2024-03-30T00:34:09.264445+0000","last_peered":"2024-03-30T00:34:09.264445+0000","last_clean":"2024-03-30T00:34:09.264445+0000","last_became_active":"2024-03-30T00:33:15.167857+0000","last_became_peered":"2024-03-30T00:33:15.167857+0000","last_unstale":"2024-03-30T00:34:09.264445+0000","last_undegraded":"2024-03-30T00:34:09.264445+0000","last_fullsized":"2024-03-30T00:34:09.264445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:03:35.951559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062087+0000","last_change":"2024-03-30T00:33:15.682519+0000","last_active":"2024-03-30T00:34:39.062087+0000","last_peered":"2024-03-30T00:34:39.062087+0000","last_clean":"2024-03-30T00:34:39.062087+0000","last_became_active":"2024-03-30T00:33:15.681055+0000","last_became_peered":"2024-03-30T00:33:15.681055+0000","last_unstale":"2024-03-30T00:34:39.062087+0000","last_undegraded":"2024-03-30T00:34:39.062087+0000","last_fullsized":"2024-03-30T00:34:39.062087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:38.160151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051654+0000","last_change":"2024-03-30T00:33:15.673411+0000","last_active":"2024-03-30T00:34:39.051654+0000","last_peered":"2024-03-30T00:34:39.051654+0000","last_clean":"2024-03-30T00:34:39.051654+0000","last_became_active":"2024-03-30T00:33:15.672852+0000","last_became_peered":"2024-03-30T00:33:15.672852+0000","last_unstale":"2024-03-30T00:34:39.051654+0000","last_undegraded":"2024-03-30T00:34:39.051654+0000","last_fullsized":"2024-03-30T00:34:39.051654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:55:46.705785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.055100+0000","last_change":"2024-03-30T00:33:15.680624+0000","last_active":"2024-03-30T00:34:39.055100+0000","last_peered":"2024-03-30T00:34:39.055100+0000","last_clean":"2024-03-30T00:34:39.055100+0000","last_became_active":"2024-03-30T00:33:15.679900+0000","last_became_peered":"2024-03-30T00:33:15.679900+0000","last_unstale":"2024-03-30T00:34:39.055100+0000","last_undegraded":"2024-03-30T00:34:39.055100+0000","last_fullsized":"2024-03-30T00:34:39.055100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:44.484608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051062+0000","last_change":"2024-03-30T00:33:15.672612+0000","last_active":"2024-03-30T00:34:39.051062+0000","last_peered":"2024-03-30T00:34:39.051062+0000","last_clean":"2024-03-30T00:34:39.051062+0000","last_became_active":"2024-03-30T00:33:15.671536+0000","last_became_peered":"2024-03-30T00:33:15.671536+0000","last_unstale":"2024-03-30T00:34:39.051062+0000","last_undegraded":"2024-03-30T00:34:39.051062+0000","last_fullsized":"2024-03-30T00:34:39.051062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:40.783988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260508+0000","last_change":"2024-03-30T00:33:15.681577+0000","last_active":"2024-03-30T00:34:09.260508+0000","last_peered":"2024-03-30T00:34:09.260508+0000","last_clean":"2024-03-30T00:34:09.260508+0000","last_became_active":"2024-03-30T00:33:15.681313+0000","last_became_peered":"2024-03-30T00:33:15.681313+0000","last_unstale":"2024-03-30T00:34:09.260508+0000","last_undegraded":"2024-03-30T00:34:09.260508+0000","last_fullsized":"2024-03-30T00:34:09.260508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:42:15.178957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264354+0000","last_change":"2024-03-30T00:33:15.688172+0000","last_active":"2024-03-30T00:34:09.264354+0000","last_peered":"2024-03-30T00:34:09.264354+0000","last_clean":"2024-03-30T00:34:09.264354+0000","last_became_active":"2024-03-30T00:33:15.687860+0000","last_became_peered":"2024-03-30T00:33:15.687860+0000","last_unstale":"2024-03-30T00:34:09.264354+0000","last_undegraded":"2024-03-30T00:34:09.264354+0000","last_fullsized":"2024-03-30T00:34:09.264354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:28.762798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.047772+0000","last_change":"2024-03-30T00:33:15.673868+0000","last_active":"2024-03-30T00:34:39.047772+0000","last_peered":"2024-03-30T00:34:39.047772+0000","last_clean":"2024-03-30T00:34:39.047772+0000","last_became_active":"2024-03-30T00:33:15.673467+0000","last_became_peered":"2024-03-30T00:33:15.673467+0000","last_unstale":"2024-03-30T00:34:39.047772+0000","last_undegraded":"2024-03-30T00:34:39.047772+0000","last_fullsized":"2024-03-30T00:34:39.047772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:07:39.448967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064293+0000","last_change":"2024-03-30T00:33:15.684639+0000","last_active":"2024-03-30T00:34:39.064293+0000","last_peered":"2024-03-30T00:34:39.064293+0000","last_clean":"2024-03-30T00:34:39.064293+0000","last_became_active":"2024-03-30T00:33:15.684357+0000","last_became_peered":"2024-03-30T00:33:15.684357+0000","last_unstale":"2024-03-30T00:34:39.064293+0000","last_undegraded":"2024-03-30T00:34:39.064293+0000","last_fullsized":"2024-03-30T00:34:39.064293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:02:50.995640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257545+0000","last_change":"2024-03-30T00:33:15.680810+0000","last_active":"2024-03-30T00:34:09.257545+0000","last_peered":"2024-03-30T00:34:09.257545+0000","last_clean":"2024-03-30T00:34:09.257545+0000","last_became_active":"2024-03-30T00:33:15.680457+0000","last_became_peered":"2024-03-30T00:33:15.680457+0000","last_unstale":"2024-03-30T00:34:09.257545+0000","last_undegraded":"2024-03-30T00:34:09.257545+0000","last_fullsized":"2024-03-30T00:34:09.257545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:02.575683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.061425+0000","last_change":"2024-03-30T00:34:10.305867+0000","last_active":"2024-03-30T00:34:39.061425+0000","last_peered":"2024-03-30T00:34:39.061425+0000","last_clean":"2024-03-30T00:34:39.061425+0000","last_became_active":"2024-03-30T00:34:10.304998+0000","last_became_peered":"2024-03-30T00:34:10.304998+0000","last_unstale":"2024-03-30T00:34:39.061425+0000","last_undegraded":"2024-03-30T00:34:39.061425+0000","last_fullsized":"2024-03-30T00:34:39.061425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:34.668457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257506+0000","last_change":"2024-03-30T00:33:15.671491+0000","last_active":"2024-03-30T00:34:09.257506+0000","last_peered":"2024-03-30T00:34:09.257506+0000","last_clean":"2024-03-30T00:34:09.257506+0000","last_became_active":"2024-03-30T00:33:15.671110+0000","last_became_peered":"2024-03-30T00:33:15.671110+0000","last_unstale":"2024-03-30T00:34:09.257506+0000","last_undegraded":"2024-03-30T00:34:09.257506+0000","last_fullsized":"2024-03-30T00:34:09.257506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:57.623730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.067543+0000","last_change":"2024-03-30T00:33:15.686795+0000","last_active":"2024-03-30T00:34:39.067543+0000","last_peered":"2024-03-30T00:34:39.067543+0000","last_clean":"2024-03-30T00:34:39.067543+0000","last_became_active":"2024-03-30T00:33:15.685715+0000","last_became_peered":"2024-03-30T00:33:15.685715+0000","last_unstale":"2024-03-30T00:34:39.067543+0000","last_undegraded":"2024-03-30T00:34:39.067543+0000","last_fullsized":"2024-03-30T00:34:39.067543+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:37:53.838823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264242+0000","last_change":"2024-03-30T00:33:15.687369+0000","last_active":"2024-03-30T00:34:09.264242+0000","last_peered":"2024-03-30T00:34:09.264242+0000","last_clean":"2024-03-30T00:34:09.264242+0000","last_became_active":"2024-03-30T00:33:15.686852+0000","last_became_peered":"2024-03-30T00:33:15.686852+0000","last_unstale":"2024-03-30T00:34:09.264242+0000","last_undegraded":"2024-03-30T00:34:09.264242+0000","last_fullsized":"2024-03-30T00:34:09.264242+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:09:21.003739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.058661+0000","last_change":"2024-03-30T00:33:15.676840+0000","last_active":"2024-03-30T00:34:39.058661+0000","last_peered":"2024-03-30T00:34:39.058661+0000","last_clean":"2024-03-30T00:34:39.058661+0000","last_became_active":"2024-03-30T00:33:15.675884+0000","last_became_peered":"2024-03-30T00:33:15.675884+0000","last_unstale":"2024-03-30T00:34:39.058661+0000","last_undegraded":"2024-03-30T00:34:39.058661+0000","last_fullsized":"2024-03-30T00:34:39.058661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:02.900843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73836,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":513,"num_read_kb":332,"num_write":168,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":221508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36120,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709032,"statfs":{"total":95995035648,"available":95958048768,"internally_reserved":0,"allocated":933888,"data_stored":446772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.622,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.872,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":1.167,"15min":1.167},"last":0.503}]},{"osd":1,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.60899999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1020000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.67900000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0229999999999999,"5min":1.623,"15min":1.623},"last":0.90800000000000003}]},{"osd":2,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88600000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.44900000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.028,"5min":1.304,"15min":1.304},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83799999999999997,"15min":0.83599999999999997},"min":{"1min":0.64200000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.0069999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.623},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.052,"15min":1.0880000000000001},"min":{"1min":0.64400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.98699999999999999,"5min":6.6029999999999998,"15min":6.6029999999999998},"last":0.80300000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.71399999999999997,"15min":0.72899999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84999999999999998,"5min":1.486,"15min":1.486},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.099,"5min":1.536,"15min":1.536},"last":0.749}]},{"osd":6,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.81999999999999995,"15min":0.83399999999999996},"min":{"1min":0.504,"5min":0.501,"15min":0.501},"max":{"1min":1.052,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.84099999999999997,"15min":0.86199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93799999999999994,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.96099999999999997},"min":{"1min":0.65100000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.145,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.995,"15min":1.0109999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1359999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.93899999999999995}]},{"osd":8,"last update":"Sat Mar 30 00:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.96499999999999997}]},{"osd":9,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":1.1200000000000001,"15min":1.1679999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0089999999999999,"5min":6.6399999999999997,"15min":6.6399999999999997},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":0.96199999999999997},"min":{"1min":0.71799999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3560000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.91700000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.75900000000000001,"5min":0.754,"15min":0.754},"max":{"1min":1.262,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.95599999999999996,"15min":0.96499999999999997},"min":{"1min":0.73599999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.234,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.85399999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678050,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1107750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.5219999999999998,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.48899999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.56200000000000006}]},{"osd":2,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.46600000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.74399999999999999,"15min":0.73899999999999999},"min":{"1min":0.63400000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.075,"5min":1.171,"15min":1.171},"last":0.60399999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.43099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.497,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1160000000000001,"5min":1.194,"15min":1.194},"last":0.66900000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.83699999999999997,"15min":0.83499999999999996},"min":{"1min":0.58599999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.626},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.80000000000000004,"15min":0.78600000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.79800000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.86699999999999999,"15min":0.83099999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":6.4870000000000001,"5min":6.4870000000000001,"15min":6.4870000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.81699999999999995,"15min":0.80400000000000005},"min":{"1min":0.65100000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.97999999999999998}]},{"osd":8,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.879,"15min":0.86099999999999999},"min":{"1min":0.72499999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88100000000000001,"15min":0.88300000000000001},"min":{"1min":0.66200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1679999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.58499999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.89500000000000002,"15min":0.88300000000000001},"min":{"1min":0.755,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.90600000000000003,"15min":0.89900000000000002},"min":{"1min":0.61199999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":3.508,"5min":3.508,"15min":3.508},"last":0.76100000000000001}]},{"osd":11,"last update":"Sat Mar 30 00:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0820000000000001,"5min":1.526,"15min":1.526},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.91500000000000004,"15min":0.92800000000000005},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.034,"5min":2.262,"15min":2.262},"last":0.93500000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":447008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71299999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.439}]},{"osd":1,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.77800000000000002,"15min":0.78900000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0329999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.82299999999999995,"15min":0.83799999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.018,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.69299999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82799999999999996,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.66000000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.95399999999999996,"5min":1.278,"15min":1.278},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.91800000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.40899999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.86499999999999999,"15min":0.88800000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98499999999999999,"5min":1.476,"15min":1.476},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.85699999999999998,"15min":0.878},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92900000000000005,"5min":1.254,"15min":1.254},"last":0.76400000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.87,"15min":0.88600000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.89600000000000002,"15min":0.92200000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94799999999999995,"5min":1.577,"15min":1.577},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.86899999999999999,"15min":0.89500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0600000000000001,"5min":1.486,"15min":1.486},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.90200000000000002,"15min":0.92900000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.073,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.93899999999999995,"15min":0.96299999999999997},"min":{"1min":0.53100000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0669999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.028,"15min":1.0620000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.083,"5min":2.234,"15min":2.234},"last":0.74399999999999999}]},{"osd":8,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.998,"15min":1.024},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.095,"5min":1.611,"15min":1.611},"last":0.63},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.0369999999999999,"15min":1.075},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.998,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.79100000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.0329999999999999,"5min":1.355,"15min":1.355},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":1.002},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1990000000000001,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.77300000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":446928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":3.8370000000000002,"5min":3.8370000000000002,"15min":3.8370000000000002},"last":0.72299999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.75900000000000001,"15min":0.747},"min":{"1min":0.56699999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81899999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83799999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65700000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.629},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.8570000000000002,"5min":3.8570000000000002,"15min":3.8570000000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.90600000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.78300000000000003,"15min":0.77200000000000002},"min":{"1min":0.628,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0369999999999999,"5min":1.246,"15min":1.246},"last":0.90200000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95199999999999996,"5min":1.026,"15min":1.026},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92200000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.47599999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79900000000000004,"15min":0.81699999999999995},"min":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.98599999999999999,"5min":1.054,"15min":1.054},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.86199999999999999,"15min":0.87},"min":{"1min":0.61399999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.7650000000000001,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.75}]},{"osd":7,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.71699999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.83799999999999997,"15min":0.84599999999999997},"min":{"1min":0.58199999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0469999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83199999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.86299999999999999,"15min":0.84199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.8969999999999998,"5min":3.8969999999999998,"15min":3.8969999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.8250000000000002,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83599999999999997,"15min":0.83399999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.88200000000000001,"15min":0.86599999999999999},"min":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.93999999999999995}]},{"osd":11,"last update":"Sat Mar 30 00:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.85799999999999998}]}]},{"osd":7,"up_from":50,"seq":214748364846,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1544192,"data_stored":1039363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93899999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.746}]},{"osd":2,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89500000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.0049999999999999,"5min":1.111,"15min":1.111},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93400000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68400000000000005,"15min":0.67200000000000004},"min":{"1min":0.47499999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97599999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.81100000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68500000000000005,"15min":0.66300000000000003},"min":{"1min":0.45300000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0209999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94099999999999995,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.84199999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.76900000000000002,"15min":0.78600000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94799999999999995,"5min":1.095,"15min":1.095},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78500000000000003,"15min":0.78700000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.96899999999999997,"5min":1.252,"15min":1.252},"last":0.68600000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78500000000000003,"15min":0.77800000000000002},"min":{"1min":0.58099999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98999999999999999,"5min":1.254,"15min":1.254},"last":0.879},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.78100000000000003,"15min":0.78800000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.88400000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.65400000000000003}]},{"osd":8,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84399999999999997,"15min":0.85199999999999998},"min":{"1min":0.63300000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.026,"5min":1.236,"15min":1.236},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.84199999999999997,"15min":0.83599999999999997},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0640000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.91100000000000003}]},{"osd":9,"last update":"Sat Mar 30 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86899999999999999,"15min":0.877},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.028,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.85499999999999998,"15min":0.86899999999999999},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98799999999999999,"5min":1.284,"15min":1.284},"last":0.89100000000000001}]},{"osd":10,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.88900000000000001,"15min":0.89900000000000002},"min":{"1min":0.68200000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0009999999999999,"5min":1.321,"15min":1.321},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.91100000000000003,"15min":0.91600000000000004},"min":{"1min":0.63300000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.048,"5min":1.3,"15min":1.3},"last":0.96699999999999997}]},{"osd":11,"last update":"Sat Mar 30 00:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.90200000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":937984,"data_stored":446397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64900000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77700000000000002,"5min":0.873,"15min":0.873},"last":0.436}]},{"osd":1,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.46000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.626},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71399999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91100000000000003,"5min":1.002,"15min":1.002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0069999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.54400000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.877,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.497}]},{"osd":4,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.048,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.749,"15min":0.749},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.96899999999999997,"5min":1.238,"15min":1.238},"last":0.749}]},{"osd":5,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0329999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.55900000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1080000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.53800000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80600000000000005,"15min":0.80800000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90500000000000003,"5min":1.105,"15min":1.105},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.99099999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77300000000000002}]},{"osd":8,"last update":"Sat Mar 30 00:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.85799999999999998,"15min":0.89500000000000002},"min":{"1min":0.60399999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.96999999999999997,"5min":1.252,"15min":1.252},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.82799999999999996,"15min":0.85799999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95099999999999996,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.56699999999999995}]},{"osd":9,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.84299999999999997,"15min":0.86499999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.89400000000000002,"15min":0.90900000000000003},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":0.97799999999999998,"5min":1.155,"15min":1.155},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.88600000000000001,"15min":0.89600000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0680000000000001,"5min":1.206,"15min":1.206},"last":0.84199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":1007616,"data_stored":515520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64000000000000001,"15min":0.59699999999999998},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80100000000000005,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.498},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64400000000000002,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:34:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.66200000000000003,"15min":0.58699999999999997},"min":{"1min":0.495,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94199999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.70599999999999996,"15min":0.61799999999999999},"min":{"1min":0.54500000000000004,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0640000000000001,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.85399999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65100000000000002,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.125,"15min":1.125},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64000000000000001,"15min":0.59699999999999998},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.98899999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.65600000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.52700000000000002,"15min":0.48999999999999999},"min":{"1min":0.40500000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89400000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57399999999999995,"15min":0.51800000000000002},"min":{"1min":0.45800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91600000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.80600000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.70099999999999996,"15min":0.65000000000000002},"min":{"1min":0.65400000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.99099999999999999,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.876},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71499999999999997,"15min":0.69599999999999995},"min":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.079,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.52400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93100000000000005,"5min":1.046,"15min":1.046},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68700000000000006,"15min":0.65700000000000003},"min":{"1min":0.47899999999999998,"5min":0.438,"15min":0.438},"max":{"1min":1.099,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.90800000000000003}]},{"osd":8,"last update":"Sat Mar 30 00:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78100000000000003,"15min":0.75900000000000001},"min":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88200000000000001,"15min":0.88300000000000001},"min":{"1min":0.67200000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.159,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.95099999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.873,"15min":0.871},"min":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.125,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.018},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1200000000000001,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.92600000000000005}]},{"osd":10,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1679999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.87,"15min":0.86199999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.98799999999999999}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0409999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":958464,"data_stored":447668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78400000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68899999999999995,"15min":0.60299999999999998},"min":{"1min":0.42399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.619,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.68600000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6819999999999999,"5min":5.8550000000000004,"15min":5.8550000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.80300000000000005},"min":{"1min":0.442,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90800000000000003,"5min":5.282,"15min":5.282},"last":0.75900000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.621}]},{"osd":4,"last update":"Sat Mar 30 00:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.65900000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.58499999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:34:27 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.58799999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.79000000000000004,"15min":0.91300000000000003},"min":{"1min":0.46800000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89300000000000002,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.69199999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.377,"15min":0.377},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.629,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.113,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.48199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.125,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2470000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Mar 30 00:34:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.754,"15min":0.79000000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.84799999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.83299999999999996,"15min":0.874},"min":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.70599999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.84599999999999997,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":1.161,"15min":1.161},"last":0.54000000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.84499999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1899999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.90500000000000003,"15min":0.91200000000000003},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.77200000000000002}]},{"osd":11,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77000000000000002,"15min":0.76300000000000001},"min":{"1min":0.63900000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.749}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":1032192,"data_stored":517318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.623,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82499999999999996,"5min":1.131,"15min":1.131},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.45800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.63800000000000001,"15min":0.59699999999999998},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.501},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.70399999999999996,"15min":0.63800000000000001},"min":{"1min":0.56200000000000006,"5min":0.375,"15min":0.375},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68200000000000005,"15min":0.66800000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93799999999999994,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.45000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.68999999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.91400000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.59899999999999998,"15min":0.53100000000000003},"min":{"1min":0.49299999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0289999999999999,"5min":1.147,"15min":1.147},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.45700000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67900000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77300000000000002,"15min":0.74299999999999999},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0309999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72099999999999997,"15min":0.69699999999999995},"min":{"1min":0.55900000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.85599999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.65700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.83299999999999996}]},{"osd":8,"last update":"Sat Mar 30 00:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.74399999999999999,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97199999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.85699999999999998,"15min":0.83099999999999996},"min":{"1min":0.67000000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.052,"5min":1.321,"15min":1.321},"last":0.71799999999999997}]},{"osd":9,"last update":"Sat Mar 30 00:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95799999999999996,"15min":0.95999999999999996},"min":{"1min":0.68500000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.137,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.90800000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.88200000000000001,"15min":0.872},"min":{"1min":0.77300000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.1439999999999999,"5min":1.204,"15min":1.204},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.004,"15min":0.996},"min":{"1min":0.85499999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.218,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.0429999999999999}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.0129999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":446708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42499999999999999,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46800000000000003,"15min":0.437},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.40200000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.622,"15min":0.58599999999999997},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.56499999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68300000000000005,"15min":0.63600000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:34:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.48999999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.748,"15min":0.69999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.79500000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.65500000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78100000000000003,"15min":0.77400000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.74299999999999999}]},{"osd":8,"last update":"Sat Mar 30 00:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.66700000000000004}]},{"osd":9,"last update":"Sat Mar 30 00:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.87,"15min":0.88400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0209999999999999,"5min":1.099,"15min":1.099},"last":0.502}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.85399999999999998,"15min":0.86599999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.71899999999999997}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.82499999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":942080,"data_stored":447282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.497}]},{"osd":1,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48599999999999999,"15min":0.45200000000000001},"min":{"1min":0.372,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.48699999999999999,"15min":0.45500000000000002},"min":{"1min":0.39200000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71399999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65600000000000003,"15min":0.64300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.55900000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.81200000000000006}]},{"osd":3,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69999999999999996,"15min":0.65800000000000003},"min":{"1min":0.54700000000000004,"5min":0.439,"15min":0.439},"max":{"1min":0.97699999999999998,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68899999999999995,"15min":0.66300000000000003},"min":{"1min":0.52800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83999999999999997,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.94699999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70499999999999996,"15min":0.68300000000000005},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0049999999999999,"5min":1.137,"15min":1.137},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71399999999999997,"15min":0.67600000000000005},"min":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0720000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.76100000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0369999999999999,"5min":1.071,"15min":1.071},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82399999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68999999999999995,"15min":0.66200000000000003},"min":{"1min":0.51500000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.69699999999999995,"15min":0.67400000000000004},"min":{"1min":0.54200000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.70899999999999996}]},{"osd":8,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.83199999999999996,"15min":0.82399999999999995},"min":{"1min":0.64000000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0589999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.79400000000000004,"15min":0.77700000000000002},"min":{"1min":0.64600000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.026,"5min":1.153,"15min":1.153},"last":0.879}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.86499999999999999,"15min":0.88500000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97499999999999998,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.91200000000000003,"15min":0.92900000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0549999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.91400000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.872,"15min":0.87},"min":{"1min":0.69199999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.167,"5min":1.179,"15min":1.179},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.871,"15min":0.879},"min":{"1min":0.57499999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.73299999999999998}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.95999999999999996}]}]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32628,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712524,"statfs":{"total":95995035648,"available":95961624576,"internally_reserved":0,"allocated":1552384,"data_stored":1038268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.71599999999999997,"15min":0.81299999999999994},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88800000000000001,"5min":8.077,"15min":8.077},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.64900000000000002,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.67600000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.64100000000000001,"15min":0.61899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95199999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.69299999999999995,"15min":0.64800000000000002},"min":{"1min":0.52800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.82299999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.45400000000000001},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.503},"min":{"1min":0.38600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85399999999999998,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.47299999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.504},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.753,"15min":0.86699999999999999},"min":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94199999999999995,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":0.54600000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.73199999999999998,"15min":0.67800000000000005},"min":{"1min":0.55300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1419999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69799999999999995,"15min":0.67400000000000004},"min":{"1min":0.54500000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94199999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.748}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64700000000000002,"15min":0.629},"min":{"1min":0.56000000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84999999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.72399999999999998,"15min":0.68600000000000005},"min":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.73099999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.57199999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.72999999999999998,"15min":0.71399999999999997},"min":{"1min":0.623,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.93899999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89000000000000001}]},{"osd":8,"last update":"Sat Mar 30 00:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.77400000000000002,"15min":0.72999999999999998},"min":{"1min":0.65400000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.73699999999999999,"15min":0.70099999999999996},"min":{"1min":0.621,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.70899999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.875,"15min":0.86899999999999999},"min":{"1min":0.70699999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.83499999999999996,"15min":0.83399999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.071,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.84699999999999998}]},{"osd":10,"last update":"Sat Mar 30 00:34:12 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.86499999999999999,"15min":0.85899999999999999},"min":{"1min":0.71199999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.0589999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.872},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.79700000000000004,"15min":0.78900000000000003},"min":{"1min":0.67200000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.83099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:34:43.950 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:34:44.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.169+0000 7f7804c00700 1 -- 172.21.15.99:0/2238219384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7800101e70 msgr2=0x7f78001022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:44.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.169+0000 7f7804c00700 1 --2- 172.21.15.99:0/2238219384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7800101e70 0x7f78001022d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0099c0 tx=0x7f77ec009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.169+0000 7f7804c00700 1 -- 172.21.15.99:0/2238219384 shutdown_connections 2024-03-30T00:34:44.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.169+0000 7f7804c00700 1 --2- 172.21.15.99:0/2238219384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7800102810 0x7f78001090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.169+0000 7f7804c00700 1 --2- 172.21.15.99:0/2238219384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7800100c80 0x7f78001010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.169+0000 7f7804c00700 1 --2- 172.21.15.99:0/2238219384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7800101e70 0x7f78001022d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.169+0000 7f7804c00700 1 -- 172.21.15.99:0/2238219384 >> 172.21.15.99:0/2238219384 conn(0x7f78000fc1f0 msgr2=0x7f78000fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:44.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.169+0000 7f7804c00700 1 -- 172.21.15.99:0/2238219384 shutdown_connections 2024-03-30T00:34:44.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.169+0000 7f7804c00700 1 -- 172.21.15.99:0/2238219384 wait complete. 2024-03-30T00:34:44.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.170+0000 7f7804c00700 1 Processor -- start 2024-03-30T00:34:44.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.170+0000 7f7804c00700 1 -- start start 2024-03-30T00:34:44.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.170+0000 7f7804c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7800100c80 0x7f7800198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:44.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.170+0000 7f7804c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7800101e70 0x7f7800199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.170+0000 7f7804c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7800102810 0x7f78001a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.170+0000 7f7804c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78001a0800 con 0x7f7800100c80 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f7804c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78001a0970 con 0x7f7800101e70 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f7804c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78001a0b30 con 0x7f7800102810 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7800100c80 0x7f7800198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7800100c80 0x7f7800198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52998/0 (socket says 172.21.15.99:52998) 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fde00700 1 -- 172.21.15.99:0/2602780868 learned_addr learned my addr 172.21.15.99:0/2602780868 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fe800700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7800102810 0x7f78001a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fd400700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7800101e70 0x7f7800199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fde00700 1 -- 172.21.15.99:0/2602780868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7800102810 msgr2=0x7f78001a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fde00700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7800102810 0x7f78001a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fde00700 1 -- 172.21.15.99:0/2602780868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7800101e70 msgr2=0x7f7800199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:44.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fde00700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7800101e70 0x7f7800199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fde00700 1 -- 172.21.15.99:0/2602780868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec0095f0 con 0x7f7800100c80 2024-03-30T00:34:44.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fd400700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7800101e70 0x7f7800199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:44.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.171+0000 7f77fde00700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7800100c80 0x7f7800198ce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f77f000e970 tx=0x7f77f000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:44.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.172+0000 7f77f6a00700 1 -- 172.21.15.99:0/2602780868 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0009790 con 0x7f7800100c80 2024-03-30T00:34:44.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.172+0000 7f7804c00700 1 -- 172.21.15.99:0/2602780868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78001a0e10 con 0x7f7800100c80 2024-03-30T00:34:44.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.172+0000 7f77f6a00700 1 -- 172.21.15.99:0/2602780868 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77f00044b0 con 0x7f7800100c80 2024-03-30T00:34:44.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.172+0000 7f77f6a00700 1 -- 172.21.15.99:0/2602780868 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0003d70 con 0x7f7800100c80 2024-03-30T00:34:44.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.172+0000 7f7804c00700 1 -- 172.21.15.99:0/2602780868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78001a1290 con 0x7f7800100c80 2024-03-30T00:34:44.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.173+0000 7f7804c00700 1 -- 172.21.15.99:0/2602780868 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7800067a10 con 0x7f7800100c80 2024-03-30T00:34:44.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.174+0000 7f77f6a00700 1 -- 172.21.15.99:0/2602780868 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f77f0003ed0 con 0x7f7800100c80 2024-03-30T00:34:44.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.176+0000 7f77f6a00700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77e40a1120 0x7f77e40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:44.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.176+0000 7f77f6a00700 1 -- 172.21.15.99:0/2602780868 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f77f0014070 con 0x7f7800100c80 2024-03-30T00:34:44.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.176+0000 7f77fd400700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77e40a1120 0x7f77e40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:44.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.177+0000 7f77fd400700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77e40a1120 0x7f77e40a35e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00b580 tx=0x7f77ec01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:44.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.179+0000 7f77f6a00700 1 -- 172.21.15.99:0/2602780868 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f77f0096360 con 0x7f7800100c80 2024-03-30T00:34:44.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.383+0000 7f7804c00700 1 -- 172.21.15.99:0/2602780868 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7800074700 con 0x7f77e40a1120 2024-03-30T00:34:44.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.394+0000 7f77f6a00700 1 -- 172.21.15.99:0/2602780868 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343408 (secure 0 0 0) 0x7f7800074700 con 0x7f77e40a1120 2024-03-30T00:34:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.398+0000 7f7804c00700 1 -- 172.21.15.99:0/2602780868 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77e40a1120 msgr2=0x7f77e40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.398+0000 7f7804c00700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77e40a1120 0x7f77e40a35e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00b580 tx=0x7f77ec01a040 comp rx=0 tx=0).stop 2024-03-30T00:34:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.398+0000 7f7804c00700 1 -- 172.21.15.99:0/2602780868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7800100c80 msgr2=0x7f7800198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.398+0000 7f7804c00700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7800100c80 0x7f7800198ce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f77f000e970 tx=0x7f77f000ed30 comp rx=0 tx=0).stop 2024-03-30T00:34:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.398+0000 7f7804c00700 1 -- 172.21.15.99:0/2602780868 shutdown_connections 2024-03-30T00:34:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.398+0000 7f7804c00700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7800102810 0x7f78001a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.398+0000 7f7804c00700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7800100c80 0x7f7800198ce0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.398+0000 7f7804c00700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77e40a1120 0x7f77e40a35e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.398+0000 7f7804c00700 1 --2- 172.21.15.99:0/2602780868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7800101e70 0x7f7800199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.398+0000 7f7804c00700 1 -- 172.21.15.99:0/2602780868 >> 172.21.15.99:0/2602780868 conn(0x7f78000fc1f0 msgr2=0x7f7800072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:44.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.399+0000 7f7804c00700 1 -- 172.21.15.99:0/2602780868 shutdown_connections 2024-03-30T00:34:44.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.399+0000 7f7804c00700 1 -- 172.21.15.99:0/2602780868 wait complete. 2024-03-30T00:34:44.410 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-30T00:34:44.424 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-03-30T00:34:43.029365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664438,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":639,"num_read_kb":441,"num_write":351,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":631,"ondisk_log_size":631,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378260,"kb_used_data":13084,"kb_used_omap":18,"kb_used_meta":365037,"kb_avail":1124563564,"statfs":{"total":1151940427776,"available":1151553089536,"internally_reserved":0,"allocated":13398016,"data_stored":7346982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":373798214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6051,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":38,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003339"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064138+0000","last_change":"2024-03-30T00:33:15.686814+0000","last_active":"2024-03-30T00:34:39.064138+0000","last_peered":"2024-03-30T00:34:39.064138+0000","last_clean":"2024-03-30T00:34:39.064138+0000","last_became_active":"2024-03-30T00:33:15.686432+0000","last_became_peered":"2024-03-30T00:33:15.686432+0000","last_unstale":"2024-03-30T00:34:39.064138+0000","last_undegraded":"2024-03-30T00:34:39.064138+0000","last_fullsized":"2024-03-30T00:34:39.064138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:35:05.898973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260589+0000","last_change":"2024-03-30T00:33:15.678089+0000","last_active":"2024-03-30T00:34:09.260589+0000","last_peered":"2024-03-30T00:34:09.260589+0000","last_clean":"2024-03-30T00:34:09.260589+0000","last_became_active":"2024-03-30T00:33:15.677254+0000","last_became_peered":"2024-03-30T00:33:15.677254+0000","last_unstale":"2024-03-30T00:34:09.260589+0000","last_undegraded":"2024-03-30T00:34:09.260589+0000","last_fullsized":"2024-03-30T00:34:09.260589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:45.922652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.305494+0000","last_change":"2024-03-30T00:34:10.305494+0000","last_active":"2024-03-30T00:34:10.305494+0000","last_peered":"2024-03-30T00:34:10.305494+0000","last_clean":"2024-03-30T00:34:10.305494+0000","last_became_active":"2024-03-30T00:34:10.305141+0000","last_became_peered":"2024-03-30T00:34:10.305141+0000","last_unstale":"2024-03-30T00:34:10.305494+0000","last_undegraded":"2024-03-30T00:34:10.305494+0000","last_fullsized":"2024-03-30T00:34:10.305494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:25:18.167062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,8],"acting":[0,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.058589+0000","last_change":"2024-03-30T00:33:15.681237+0000","last_active":"2024-03-30T00:34:39.058589+0000","last_peered":"2024-03-30T00:34:39.058589+0000","last_clean":"2024-03-30T00:34:39.058589+0000","last_became_active":"2024-03-30T00:33:15.680669+0000","last_became_peered":"2024-03-30T00:33:15.680669+0000","last_unstale":"2024-03-30T00:34:39.058589+0000","last_undegraded":"2024-03-30T00:34:39.058589+0000","last_fullsized":"2024-03-30T00:34:39.058589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:59.300262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.037005+0000","last_change":"2024-03-30T00:33:15.678825+0000","last_active":"2024-03-30T00:34:39.037005+0000","last_peered":"2024-03-30T00:34:39.037005+0000","last_clean":"2024-03-30T00:34:39.037005+0000","last_became_active":"2024-03-30T00:33:15.676864+0000","last_became_peered":"2024-03-30T00:33:15.676864+0000","last_unstale":"2024-03-30T00:34:39.037005+0000","last_undegraded":"2024-03-30T00:34:39.037005+0000","last_fullsized":"2024-03-30T00:34:39.037005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:12:09.724441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.060702+0000","last_change":"2024-03-30T00:33:15.681866+0000","last_active":"2024-03-30T00:34:39.060702+0000","last_peered":"2024-03-30T00:34:39.060702+0000","last_clean":"2024-03-30T00:34:39.060702+0000","last_became_active":"2024-03-30T00:33:15.678118+0000","last_became_peered":"2024-03-30T00:33:15.678118+0000","last_unstale":"2024-03-30T00:34:39.060702+0000","last_undegraded":"2024-03-30T00:34:39.060702+0000","last_fullsized":"2024-03-30T00:34:39.060702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:24.433027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051906+0000","last_change":"2024-03-30T00:33:15.675999+0000","last_active":"2024-03-30T00:34:39.051906+0000","last_peered":"2024-03-30T00:34:39.051906+0000","last_clean":"2024-03-30T00:34:39.051906+0000","last_became_active":"2024-03-30T00:33:15.675319+0000","last_became_peered":"2024-03-30T00:33:15.675319+0000","last_unstale":"2024-03-30T00:34:39.051906+0000","last_undegraded":"2024-03-30T00:34:39.051906+0000","last_fullsized":"2024-03-30T00:34:39.051906+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:42.668747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257311+0000","last_change":"2024-03-30T00:33:15.679211+0000","last_active":"2024-03-30T00:34:09.257311+0000","last_peered":"2024-03-30T00:34:09.257311+0000","last_clean":"2024-03-30T00:34:09.257311+0000","last_became_active":"2024-03-30T00:33:15.675061+0000","last_became_peered":"2024-03-30T00:33:15.675061+0000","last_unstale":"2024-03-30T00:34:09.257311+0000","last_undegraded":"2024-03-30T00:34:09.257311+0000","last_fullsized":"2024-03-30T00:34:09.257311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:43.973542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064325+0000","last_change":"2024-03-30T00:33:15.680842+0000","last_active":"2024-03-30T00:34:39.064325+0000","last_peered":"2024-03-30T00:34:39.064325+0000","last_clean":"2024-03-30T00:34:39.064325+0000","last_became_active":"2024-03-30T00:33:15.680307+0000","last_became_peered":"2024-03-30T00:33:15.680307+0000","last_unstale":"2024-03-30T00:34:39.064325+0000","last_undegraded":"2024-03-30T00:34:39.064325+0000","last_fullsized":"2024-03-30T00:34:39.064325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:59:40.331779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.060850+0000","last_change":"2024-03-30T00:33:15.692397+0000","last_active":"2024-03-30T00:34:39.060850+0000","last_peered":"2024-03-30T00:34:39.060850+0000","last_clean":"2024-03-30T00:34:39.060850+0000","last_became_active":"2024-03-30T00:33:15.691226+0000","last_became_peered":"2024-03-30T00:33:15.691226+0000","last_unstale":"2024-03-30T00:34:39.060850+0000","last_undegraded":"2024-03-30T00:34:39.060850+0000","last_fullsized":"2024-03-30T00:34:39.060850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:48.816644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.265637+0000","last_change":"2024-03-30T00:33:15.688417+0000","last_active":"2024-03-30T00:34:10.265637+0000","last_peered":"2024-03-30T00:34:10.265637+0000","last_clean":"2024-03-30T00:34:10.265637+0000","last_became_active":"2024-03-30T00:33:15.687755+0000","last_became_peered":"2024-03-30T00:33:15.687755+0000","last_unstale":"2024-03-30T00:34:10.265637+0000","last_undegraded":"2024-03-30T00:34:10.265637+0000","last_fullsized":"2024-03-30T00:34:10.265637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:30:19.782280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.061117+0000","last_change":"2024-03-30T00:33:15.681942+0000","last_active":"2024-03-30T00:34:39.061117+0000","last_peered":"2024-03-30T00:34:39.061117+0000","last_clean":"2024-03-30T00:34:39.061117+0000","last_became_active":"2024-03-30T00:33:15.679854+0000","last_became_peered":"2024-03-30T00:33:15.679854+0000","last_unstale":"2024-03-30T00:34:39.061117+0000","last_undegraded":"2024-03-30T00:34:39.061117+0000","last_fullsized":"2024-03-30T00:34:39.061117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:19.771959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264276+0000","last_change":"2024-03-30T00:33:15.682968+0000","last_active":"2024-03-30T00:34:09.264276+0000","last_peered":"2024-03-30T00:34:09.264276+0000","last_clean":"2024-03-30T00:34:09.264276+0000","last_became_active":"2024-03-30T00:33:15.681216+0000","last_became_peered":"2024-03-30T00:33:15.681216+0000","last_unstale":"2024-03-30T00:34:09.264276+0000","last_undegraded":"2024-03-30T00:34:09.264276+0000","last_fullsized":"2024-03-30T00:34:09.264276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:50.038843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260360+0000","last_change":"2024-03-30T00:33:15.677727+0000","last_active":"2024-03-30T00:34:09.260360+0000","last_peered":"2024-03-30T00:34:09.260360+0000","last_clean":"2024-03-30T00:34:09.260360+0000","last_became_active":"2024-03-30T00:33:15.676758+0000","last_became_peered":"2024-03-30T00:33:15.676758+0000","last_unstale":"2024-03-30T00:34:09.260360+0000","last_undegraded":"2024-03-30T00:34:09.260360+0000","last_fullsized":"2024-03-30T00:34:09.260360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:34.745920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260097+0000","last_change":"2024-03-30T00:33:15.684545+0000","last_active":"2024-03-30T00:34:09.260097+0000","last_peered":"2024-03-30T00:34:09.260097+0000","last_clean":"2024-03-30T00:34:09.260097+0000","last_became_active":"2024-03-30T00:33:15.682989+0000","last_became_peered":"2024-03-30T00:33:15.682989+0000","last_unstale":"2024-03-30T00:34:09.260097+0000","last_undegraded":"2024-03-30T00:34:09.260097+0000","last_fullsized":"2024-03-30T00:34:09.260097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:11.120981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.067066+0000","last_change":"2024-03-30T00:33:15.678122+0000","last_active":"2024-03-30T00:34:39.067066+0000","last_peered":"2024-03-30T00:34:39.067066+0000","last_clean":"2024-03-30T00:34:39.067066+0000","last_became_active":"2024-03-30T00:33:15.677681+0000","last_became_peered":"2024-03-30T00:33:15.677681+0000","last_unstale":"2024-03-30T00:34:39.067066+0000","last_undegraded":"2024-03-30T00:34:39.067066+0000","last_fullsized":"2024-03-30T00:34:39.067066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:50.454125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064339+0000","last_change":"2024-03-30T00:33:15.687689+0000","last_active":"2024-03-30T00:34:39.064339+0000","last_peered":"2024-03-30T00:34:39.064339+0000","last_clean":"2024-03-30T00:34:39.064339+0000","last_became_active":"2024-03-30T00:33:15.686009+0000","last_became_peered":"2024-03-30T00:33:15.686009+0000","last_unstale":"2024-03-30T00:34:39.064339+0000","last_undegraded":"2024-03-30T00:34:39.064339+0000","last_fullsized":"2024-03-30T00:34:39.064339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:17.234749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062258+0000","last_change":"2024-03-30T00:33:15.686053+0000","last_active":"2024-03-30T00:34:39.062258+0000","last_peered":"2024-03-30T00:34:39.062258+0000","last_clean":"2024-03-30T00:34:39.062258+0000","last_became_active":"2024-03-30T00:33:15.685723+0000","last_became_peered":"2024-03-30T00:33:15.685723+0000","last_unstale":"2024-03-30T00:34:39.062258+0000","last_undegraded":"2024-03-30T00:34:39.062258+0000","last_fullsized":"2024-03-30T00:34:39.062258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:27.322834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.050110+0000","last_change":"2024-03-30T00:33:15.674783+0000","last_active":"2024-03-30T00:34:39.050110+0000","last_peered":"2024-03-30T00:34:39.050110+0000","last_clean":"2024-03-30T00:34:39.050110+0000","last_became_active":"2024-03-30T00:33:15.674152+0000","last_became_peered":"2024-03-30T00:33:15.674152+0000","last_unstale":"2024-03-30T00:34:39.050110+0000","last_undegraded":"2024-03-30T00:34:39.050110+0000","last_fullsized":"2024-03-30T00:34:39.050110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:39.517240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.251874+0000","last_change":"2024-03-30T00:33:15.172217+0000","last_active":"2024-03-30T00:34:09.251874+0000","last_peered":"2024-03-30T00:34:09.251874+0000","last_clean":"2024-03-30T00:34:09.251874+0000","last_became_active":"2024-03-30T00:33:15.171718+0000","last_became_peered":"2024-03-30T00:33:15.171718+0000","last_unstale":"2024-03-30T00:34:09.251874+0000","last_undegraded":"2024-03-30T00:34:09.251874+0000","last_fullsized":"2024-03-30T00:34:09.251874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:11.691572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258046+0000","last_change":"2024-03-30T00:33:15.669937+0000","last_active":"2024-03-30T00:34:09.258046+0000","last_peered":"2024-03-30T00:34:09.258046+0000","last_clean":"2024-03-30T00:34:09.258046+0000","last_became_active":"2024-03-30T00:33:15.668704+0000","last_became_peered":"2024-03-30T00:33:15.668704+0000","last_unstale":"2024-03-30T00:34:09.258046+0000","last_undegraded":"2024-03-30T00:34:09.258046+0000","last_fullsized":"2024-03-30T00:34:09.258046+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:23.914755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258142+0000","last_change":"2024-03-30T00:33:15.173663+0000","last_active":"2024-03-30T00:34:09.258142+0000","last_peered":"2024-03-30T00:34:09.258142+0000","last_clean":"2024-03-30T00:34:09.258142+0000","last_became_active":"2024-03-30T00:33:15.173288+0000","last_became_peered":"2024-03-30T00:33:15.173288+0000","last_unstale":"2024-03-30T00:34:09.258142+0000","last_undegraded":"2024-03-30T00:34:09.258142+0000","last_fullsized":"2024-03-30T00:34:09.258142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.435142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260485+0000","last_change":"2024-03-30T00:33:15.686757+0000","last_active":"2024-03-30T00:34:09.260485+0000","last_peered":"2024-03-30T00:34:09.260485+0000","last_clean":"2024-03-30T00:34:09.260485+0000","last_became_active":"2024-03-30T00:33:15.686306+0000","last_became_peered":"2024-03-30T00:33:15.686306+0000","last_unstale":"2024-03-30T00:34:09.260485+0000","last_undegraded":"2024-03-30T00:34:09.260485+0000","last_fullsized":"2024-03-30T00:34:09.260485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:50:21.048212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254756+0000","last_change":"2024-03-30T00:33:15.173402+0000","last_active":"2024-03-30T00:34:09.254756+0000","last_peered":"2024-03-30T00:34:09.254756+0000","last_clean":"2024-03-30T00:34:09.254756+0000","last_became_active":"2024-03-30T00:33:15.172995+0000","last_became_peered":"2024-03-30T00:33:15.172995+0000","last_unstale":"2024-03-30T00:34:09.254756+0000","last_undegraded":"2024-03-30T00:34:09.254756+0000","last_fullsized":"2024-03-30T00:34:09.254756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:25.694792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065176+0000","last_change":"2024-03-30T00:33:15.688292+0000","last_active":"2024-03-30T00:34:39.065176+0000","last_peered":"2024-03-30T00:34:39.065176+0000","last_clean":"2024-03-30T00:34:39.065176+0000","last_became_active":"2024-03-30T00:33:15.682722+0000","last_became_peered":"2024-03-30T00:33:15.682722+0000","last_unstale":"2024-03-30T00:34:39.065176+0000","last_undegraded":"2024-03-30T00:34:39.065176+0000","last_fullsized":"2024-03-30T00:34:39.065176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:52:10.074656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254648+0000","last_change":"2024-03-30T00:33:15.162316+0000","last_active":"2024-03-30T00:34:09.254648+0000","last_peered":"2024-03-30T00:34:09.254648+0000","last_clean":"2024-03-30T00:34:09.254648+0000","last_became_active":"2024-03-30T00:33:15.161805+0000","last_became_peered":"2024-03-30T00:33:15.161805+0000","last_unstale":"2024-03-30T00:34:09.254648+0000","last_undegraded":"2024-03-30T00:34:09.254648+0000","last_fullsized":"2024-03-30T00:34:09.254648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:26:16.470245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062423+0000","last_change":"2024-03-30T00:33:15.675442+0000","last_active":"2024-03-30T00:34:39.062423+0000","last_peered":"2024-03-30T00:34:39.062423+0000","last_clean":"2024-03-30T00:34:39.062423+0000","last_became_active":"2024-03-30T00:33:15.675068+0000","last_became_peered":"2024-03-30T00:33:15.675068+0000","last_unstale":"2024-03-30T00:34:39.062423+0000","last_undegraded":"2024-03-30T00:34:39.062423+0000","last_fullsized":"2024-03-30T00:34:39.062423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:36:31.353262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":105,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:40.249081+0000","last_change":"2024-03-30T00:33:15.172087+0000","last_active":"2024-03-30T00:34:40.249081+0000","last_peered":"2024-03-30T00:34:40.249081+0000","last_clean":"2024-03-30T00:34:40.249081+0000","last_became_active":"2024-03-30T00:33:15.171768+0000","last_became_peered":"2024-03-30T00:33:15.171768+0000","last_unstale":"2024-03-30T00:34:40.249081+0000","last_undegraded":"2024-03-30T00:34:40.249081+0000","last_fullsized":"2024-03-30T00:34:40.249081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:17.198995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1070,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.042702+0000","last_change":"2024-03-30T00:33:15.665219+0000","last_active":"2024-03-30T00:34:39.042702+0000","last_peered":"2024-03-30T00:34:39.042702+0000","last_clean":"2024-03-30T00:34:39.042702+0000","last_became_active":"2024-03-30T00:33:15.664935+0000","last_became_peered":"2024-03-30T00:33:15.664935+0000","last_unstale":"2024-03-30T00:34:39.042702+0000","last_undegraded":"2024-03-30T00:34:39.042702+0000","last_fullsized":"2024-03-30T00:34:39.042702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:37.123937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254914+0000","last_change":"2024-03-30T00:33:15.165668+0000","last_active":"2024-03-30T00:34:09.254914+0000","last_peered":"2024-03-30T00:34:09.254914+0000","last_clean":"2024-03-30T00:34:09.254914+0000","last_became_active":"2024-03-30T00:33:15.164637+0000","last_became_peered":"2024-03-30T00:33:15.164637+0000","last_unstale":"2024-03-30T00:34:09.254914+0000","last_undegraded":"2024-03-30T00:34:09.254914+0000","last_fullsized":"2024-03-30T00:34:09.254914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:47:59.389414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.048514+0000","last_change":"2024-03-30T00:34:10.309779+0000","last_active":"2024-03-30T00:34:39.048514+0000","last_peered":"2024-03-30T00:34:39.048514+0000","last_clean":"2024-03-30T00:34:39.048514+0000","last_became_active":"2024-03-30T00:34:10.309272+0000","last_became_peered":"2024-03-30T00:34:10.309272+0000","last_unstale":"2024-03-30T00:34:39.048514+0000","last_undegraded":"2024-03-30T00:34:39.048514+0000","last_fullsized":"2024-03-30T00:34:39.048514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:22:54.666608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264857+0000","last_change":"2024-03-30T00:33:15.177269+0000","last_active":"2024-03-30T00:34:09.264857+0000","last_peered":"2024-03-30T00:34:09.264857+0000","last_clean":"2024-03-30T00:34:09.264857+0000","last_became_active":"2024-03-30T00:33:15.176937+0000","last_became_peered":"2024-03-30T00:33:15.176937+0000","last_unstale":"2024-03-30T00:34:09.264857+0000","last_undegraded":"2024-03-30T00:34:09.264857+0000","last_fullsized":"2024-03-30T00:34:09.264857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:44.658775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257873+0000","last_change":"2024-03-30T00:33:15.675832+0000","last_active":"2024-03-30T00:34:09.257873+0000","last_peered":"2024-03-30T00:34:09.257873+0000","last_clean":"2024-03-30T00:34:09.257873+0000","last_became_active":"2024-03-30T00:33:15.675481+0000","last_became_peered":"2024-03-30T00:33:15.675481+0000","last_unstale":"2024-03-30T00:34:09.257873+0000","last_undegraded":"2024-03-30T00:34:09.257873+0000","last_fullsized":"2024-03-30T00:34:09.257873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:11.638056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258682+0000","last_change":"2024-03-30T00:33:15.171238+0000","last_active":"2024-03-30T00:34:09.258682+0000","last_peered":"2024-03-30T00:34:09.258682+0000","last_clean":"2024-03-30T00:34:09.258682+0000","last_became_active":"2024-03-30T00:33:15.170931+0000","last_became_peered":"2024-03-30T00:33:15.170931+0000","last_unstale":"2024-03-30T00:34:09.258682+0000","last_undegraded":"2024-03-30T00:34:09.258682+0000","last_fullsized":"2024-03-30T00:34:09.258682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:43:50.178363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.060489+0000","last_change":"2024-03-30T00:33:15.676014+0000","last_active":"2024-03-30T00:34:39.060489+0000","last_peered":"2024-03-30T00:34:39.060489+0000","last_clean":"2024-03-30T00:34:39.060489+0000","last_became_active":"2024-03-30T00:33:15.675553+0000","last_became_peered":"2024-03-30T00:33:15.675553+0000","last_unstale":"2024-03-30T00:34:39.060489+0000","last_undegraded":"2024-03-30T00:34:39.060489+0000","last_fullsized":"2024-03-30T00:34:39.060489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:17.248242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":99,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.246658+0000","last_change":"2024-03-30T00:33:15.177033+0000","last_active":"2024-03-30T00:34:39.246658+0000","last_peered":"2024-03-30T00:34:39.246658+0000","last_clean":"2024-03-30T00:34:39.246658+0000","last_became_active":"2024-03-30T00:33:15.176602+0000","last_became_peered":"2024-03-30T00:33:15.176602+0000","last_unstale":"2024-03-30T00:34:39.246658+0000","last_undegraded":"2024-03-30T00:34:39.246658+0000","last_fullsized":"2024-03-30T00:34:39.246658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:10.804704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065450+0000","last_change":"2024-03-30T00:33:15.683388+0000","last_active":"2024-03-30T00:34:39.065450+0000","last_peered":"2024-03-30T00:34:39.065450+0000","last_clean":"2024-03-30T00:34:39.065450+0000","last_became_active":"2024-03-30T00:33:15.682976+0000","last_became_peered":"2024-03-30T00:33:15.682976+0000","last_unstale":"2024-03-30T00:34:39.065450+0000","last_undegraded":"2024-03-30T00:34:39.065450+0000","last_fullsized":"2024-03-30T00:34:39.065450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:50:34.907265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.256323+0000","last_change":"2024-03-30T00:33:15.167819+0000","last_active":"2024-03-30T00:34:09.256323+0000","last_peered":"2024-03-30T00:34:09.256323+0000","last_clean":"2024-03-30T00:34:09.256323+0000","last_became_active":"2024-03-30T00:33:15.167405+0000","last_became_peered":"2024-03-30T00:33:15.167405+0000","last_unstale":"2024-03-30T00:34:09.256323+0000","last_undegraded":"2024-03-30T00:34:09.256323+0000","last_fullsized":"2024-03-30T00:34:09.256323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:37.793792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260394+0000","last_change":"2024-03-30T00:33:15.690907+0000","last_active":"2024-03-30T00:34:09.260394+0000","last_peered":"2024-03-30T00:34:09.260394+0000","last_clean":"2024-03-30T00:34:09.260394+0000","last_became_active":"2024-03-30T00:33:15.690046+0000","last_became_peered":"2024-03-30T00:33:15.690046+0000","last_unstale":"2024-03-30T00:34:09.260394+0000","last_undegraded":"2024-03-30T00:34:09.260394+0000","last_fullsized":"2024-03-30T00:34:09.260394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:24.319685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258463+0000","last_change":"2024-03-30T00:33:15.166633+0000","last_active":"2024-03-30T00:34:09.258463+0000","last_peered":"2024-03-30T00:34:09.258463+0000","last_clean":"2024-03-30T00:34:09.258463+0000","last_became_active":"2024-03-30T00:33:15.166308+0000","last_became_peered":"2024-03-30T00:33:15.166308+0000","last_unstale":"2024-03-30T00:34:09.258463+0000","last_undegraded":"2024-03-30T00:34:09.258463+0000","last_fullsized":"2024-03-30T00:34:09.258463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:53.102872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065783+0000","last_change":"2024-03-30T00:33:15.691880+0000","last_active":"2024-03-30T00:34:39.065783+0000","last_peered":"2024-03-30T00:34:39.065783+0000","last_clean":"2024-03-30T00:34:39.065783+0000","last_became_active":"2024-03-30T00:33:15.690780+0000","last_became_peered":"2024-03-30T00:33:15.690780+0000","last_unstale":"2024-03-30T00:34:39.065783+0000","last_undegraded":"2024-03-30T00:34:39.065783+0000","last_fullsized":"2024-03-30T00:34:39.065783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:06.532709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":226,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:40.260983+0000","last_change":"2024-03-30T00:33:15.164287+0000","last_active":"2024-03-30T00:34:40.260983+0000","last_peered":"2024-03-30T00:34:40.260983+0000","last_clean":"2024-03-30T00:34:40.260983+0000","last_became_active":"2024-03-30T00:33:15.163557+0000","last_became_peered":"2024-03-30T00:33:15.163557+0000","last_unstale":"2024-03-30T00:34:40.260983+0000","last_undegraded":"2024-03-30T00:34:40.260983+0000","last_fullsized":"2024-03-30T00:34:40.260983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:41.117936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":174,"num_read_kb":154,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062751+0000","last_change":"2024-03-30T00:33:15.682841+0000","last_active":"2024-03-30T00:34:39.062751+0000","last_peered":"2024-03-30T00:34:39.062751+0000","last_clean":"2024-03-30T00:34:39.062751+0000","last_became_active":"2024-03-30T00:33:15.682237+0000","last_became_peered":"2024-03-30T00:33:15.682237+0000","last_unstale":"2024-03-30T00:34:39.062751+0000","last_undegraded":"2024-03-30T00:34:39.062751+0000","last_fullsized":"2024-03-30T00:34:39.062751+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:59:05.719216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.262163+0000","last_change":"2024-03-30T00:33:15.165193+0000","last_active":"2024-03-30T00:34:09.262163+0000","last_peered":"2024-03-30T00:34:09.262163+0000","last_clean":"2024-03-30T00:34:09.262163+0000","last_became_active":"2024-03-30T00:33:15.161741+0000","last_became_peered":"2024-03-30T00:33:15.161741+0000","last_unstale":"2024-03-30T00:34:09.262163+0000","last_undegraded":"2024-03-30T00:34:09.262163+0000","last_fullsized":"2024-03-30T00:34:09.262163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:22:11.891064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264117+0000","last_change":"2024-03-30T00:33:15.680929+0000","last_active":"2024-03-30T00:34:09.264117+0000","last_peered":"2024-03-30T00:34:09.264117+0000","last_clean":"2024-03-30T00:34:09.264117+0000","last_became_active":"2024-03-30T00:33:15.676671+0000","last_became_peered":"2024-03-30T00:33:15.676671+0000","last_unstale":"2024-03-30T00:34:09.264117+0000","last_undegraded":"2024-03-30T00:34:09.264117+0000","last_fullsized":"2024-03-30T00:34:09.264117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:28:33.819135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257972+0000","last_change":"2024-03-30T00:33:15.168029+0000","last_active":"2024-03-30T00:34:09.257972+0000","last_peered":"2024-03-30T00:34:09.257972+0000","last_clean":"2024-03-30T00:34:09.257972+0000","last_became_active":"2024-03-30T00:33:15.167636+0000","last_became_peered":"2024-03-30T00:33:15.167636+0000","last_unstale":"2024-03-30T00:34:09.257972+0000","last_undegraded":"2024-03-30T00:34:09.257972+0000","last_fullsized":"2024-03-30T00:34:09.257972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:25:47.834696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.058578+0000","last_change":"2024-03-30T00:33:15.687977+0000","last_active":"2024-03-30T00:34:39.058578+0000","last_peered":"2024-03-30T00:34:39.058578+0000","last_clean":"2024-03-30T00:34:39.058578+0000","last_became_active":"2024-03-30T00:33:15.687491+0000","last_became_peered":"2024-03-30T00:33:15.687491+0000","last_unstale":"2024-03-30T00:34:39.058578+0000","last_undegraded":"2024-03-30T00:34:39.058578+0000","last_fullsized":"2024-03-30T00:34:39.058578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:08:55.568039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":100,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:38.257303+0000","last_change":"2024-03-30T00:33:15.171826+0000","last_active":"2024-03-30T00:34:38.257303+0000","last_peered":"2024-03-30T00:34:38.257303+0000","last_clean":"2024-03-30T00:34:38.257303+0000","last_became_active":"2024-03-30T00:33:15.170749+0000","last_became_peered":"2024-03-30T00:33:15.170749+0000","last_unstale":"2024-03-30T00:34:38.257303+0000","last_undegraded":"2024-03-30T00:34:38.257303+0000","last_fullsized":"2024-03-30T00:34:38.257303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:57:52.034530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062584+0000","last_change":"2024-03-30T00:34:10.304456+0000","last_active":"2024-03-30T00:34:39.062584+0000","last_peered":"2024-03-30T00:34:39.062584+0000","last_clean":"2024-03-30T00:34:39.062584+0000","last_became_active":"2024-03-30T00:34:10.303934+0000","last_became_peered":"2024-03-30T00:34:10.303934+0000","last_unstale":"2024-03-30T00:34:39.062584+0000","last_undegraded":"2024-03-30T00:34:39.062584+0000","last_fullsized":"2024-03-30T00:34:39.062584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:48:07.792123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254268+0000","last_change":"2024-03-30T00:33:15.171016+0000","last_active":"2024-03-30T00:34:09.254268+0000","last_peered":"2024-03-30T00:34:09.254268+0000","last_clean":"2024-03-30T00:34:09.254268+0000","last_became_active":"2024-03-30T00:33:15.170058+0000","last_became_peered":"2024-03-30T00:33:15.170058+0000","last_unstale":"2024-03-30T00:34:09.254268+0000","last_undegraded":"2024-03-30T00:34:09.254268+0000","last_fullsized":"2024-03-30T00:34:09.254268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:07:39.910286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051568+0000","last_change":"2024-03-30T00:34:10.305753+0000","last_active":"2024-03-30T00:34:39.051568+0000","last_peered":"2024-03-30T00:34:39.051568+0000","last_clean":"2024-03-30T00:34:39.051568+0000","last_became_active":"2024-03-30T00:34:10.305339+0000","last_became_peered":"2024-03-30T00:34:10.305339+0000","last_unstale":"2024-03-30T00:34:39.051568+0000","last_undegraded":"2024-03-30T00:34:39.051568+0000","last_fullsized":"2024-03-30T00:34:39.051568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:41.611151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254799+0000","last_change":"2024-03-30T00:33:15.163633+0000","last_active":"2024-03-30T00:34:09.254799+0000","last_peered":"2024-03-30T00:34:09.254799+0000","last_clean":"2024-03-30T00:34:09.254799+0000","last_became_active":"2024-03-30T00:33:15.161339+0000","last_became_peered":"2024-03-30T00:33:15.161339+0000","last_unstale":"2024-03-30T00:34:09.254799+0000","last_undegraded":"2024-03-30T00:34:09.254799+0000","last_fullsized":"2024-03-30T00:34:09.254799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:15:25.194650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.050235+0000","last_change":"2024-03-30T00:34:10.269652+0000","last_active":"2024-03-30T00:34:39.050235+0000","last_peered":"2024-03-30T00:34:39.050235+0000","last_clean":"2024-03-30T00:34:39.050235+0000","last_became_active":"2024-03-30T00:34:10.269267+0000","last_became_peered":"2024-03-30T00:34:10.269267+0000","last_unstale":"2024-03-30T00:34:39.050235+0000","last_undegraded":"2024-03-30T00:34:39.050235+0000","last_fullsized":"2024-03-30T00:34:39.050235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:13.724322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.251412+0000","last_change":"2024-03-30T00:33:15.177273+0000","last_active":"2024-03-30T00:34:09.251412+0000","last_peered":"2024-03-30T00:34:09.251412+0000","last_clean":"2024-03-30T00:34:09.251412+0000","last_became_active":"2024-03-30T00:33:15.175333+0000","last_became_peered":"2024-03-30T00:33:15.175333+0000","last_unstale":"2024-03-30T00:34:09.251412+0000","last_undegraded":"2024-03-30T00:34:09.251412+0000","last_fullsized":"2024-03-30T00:34:09.251412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:27:03.305671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.046660+0000","last_change":"2024-03-30T00:33:15.674746+0000","last_active":"2024-03-30T00:34:39.046660+0000","last_peered":"2024-03-30T00:34:39.046660+0000","last_clean":"2024-03-30T00:34:39.046660+0000","last_became_active":"2024-03-30T00:33:15.674388+0000","last_became_peered":"2024-03-30T00:33:15.674388+0000","last_unstale":"2024-03-30T00:34:39.046660+0000","last_undegraded":"2024-03-30T00:34:39.046660+0000","last_fullsized":"2024-03-30T00:34:39.046660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:21:15.024181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.256443+0000","last_change":"2024-03-30T00:33:15.173640+0000","last_active":"2024-03-30T00:34:09.256443+0000","last_peered":"2024-03-30T00:34:09.256443+0000","last_clean":"2024-03-30T00:34:09.256443+0000","last_became_active":"2024-03-30T00:33:15.173104+0000","last_became_peered":"2024-03-30T00:33:15.173104+0000","last_unstale":"2024-03-30T00:34:09.256443+0000","last_undegraded":"2024-03-30T00:34:09.256443+0000","last_fullsized":"2024-03-30T00:34:09.256443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:20:14.880730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.056823+0000","last_change":"2024-03-30T00:33:15.677824+0000","last_active":"2024-03-30T00:34:39.056823+0000","last_peered":"2024-03-30T00:34:39.056823+0000","last_clean":"2024-03-30T00:34:39.056823+0000","last_became_active":"2024-03-30T00:33:15.677435+0000","last_became_peered":"2024-03-30T00:33:15.677435+0000","last_unstale":"2024-03-30T00:34:39.056823+0000","last_undegraded":"2024-03-30T00:34:39.056823+0000","last_fullsized":"2024-03-30T00:34:39.056823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:22.992989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.044985+0000","last_change":"2024-03-30T00:34:10.304054+0000","last_active":"2024-03-30T00:34:39.044985+0000","last_peered":"2024-03-30T00:34:39.044985+0000","last_clean":"2024-03-30T00:34:39.044985+0000","last_became_active":"2024-03-30T00:34:10.303653+0000","last_became_peered":"2024-03-30T00:34:10.303653+0000","last_unstale":"2024-03-30T00:34:39.044985+0000","last_undegraded":"2024-03-30T00:34:39.044985+0000","last_fullsized":"2024-03-30T00:34:39.044985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:12:32.378165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":96,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:36.251806+0000","last_change":"2024-03-30T00:33:15.173264+0000","last_active":"2024-03-30T00:34:36.251806+0000","last_peered":"2024-03-30T00:34:36.251806+0000","last_clean":"2024-03-30T00:34:36.251806+0000","last_became_active":"2024-03-30T00:33:15.172932+0000","last_became_peered":"2024-03-30T00:33:15.172932+0000","last_unstale":"2024-03-30T00:34:36.251806+0000","last_undegraded":"2024-03-30T00:34:36.251806+0000","last_fullsized":"2024-03-30T00:34:36.251806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:18:25.079284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257668+0000","last_change":"2024-03-30T00:33:15.686606+0000","last_active":"2024-03-30T00:34:09.257668+0000","last_peered":"2024-03-30T00:34:09.257668+0000","last_clean":"2024-03-30T00:34:09.257668+0000","last_became_active":"2024-03-30T00:33:15.686076+0000","last_became_peered":"2024-03-30T00:33:15.686076+0000","last_unstale":"2024-03-30T00:34:09.257668+0000","last_undegraded":"2024-03-30T00:34:09.257668+0000","last_fullsized":"2024-03-30T00:34:09.257668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:46.962659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.261642+0000","last_change":"2024-03-30T00:33:15.177263+0000","last_active":"2024-03-30T00:34:09.261642+0000","last_peered":"2024-03-30T00:34:09.261642+0000","last_clean":"2024-03-30T00:34:09.261642+0000","last_became_active":"2024-03-30T00:33:15.176743+0000","last_became_peered":"2024-03-30T00:33:15.176743+0000","last_unstale":"2024-03-30T00:34:09.261642+0000","last_undegraded":"2024-03-30T00:34:09.261642+0000","last_fullsized":"2024-03-30T00:34:09.261642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:56:47.491371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.256985+0000","last_change":"2024-03-30T00:32:12.678730+0000","last_active":"2024-03-30T00:34:09.256985+0000","last_peered":"2024-03-30T00:34:09.256985+0000","last_clean":"2024-03-30T00:34:09.256985+0000","last_became_active":"2024-03-30T00:32:12.678256+0000","last_became_peered":"2024-03-30T00:32:12.678256+0000","last_unstale":"2024-03-30T00:34:09.256985+0000","last_undegraded":"2024-03-30T00:34:09.256985+0000","last_fullsized":"2024-03-30T00:34:09.256985+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_clean_scrub_stamp":"2024-03-30T00:29:17.768535+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:16.229072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.304196+0000","last_change":"2024-03-30T00:34:10.304196+0000","last_active":"2024-03-30T00:34:10.304196+0000","last_peered":"2024-03-30T00:34:10.304196+0000","last_clean":"2024-03-30T00:34:10.304196+0000","last_became_active":"2024-03-30T00:34:10.303740+0000","last_became_peered":"2024-03-30T00:34:10.303740+0000","last_unstale":"2024-03-30T00:34:10.304196+0000","last_undegraded":"2024-03-30T00:34:10.304196+0000","last_fullsized":"2024-03-30T00:34:10.304196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:25:10.417989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257070+0000","last_change":"2024-03-30T00:33:15.167490+0000","last_active":"2024-03-30T00:34:09.257070+0000","last_peered":"2024-03-30T00:34:09.257070+0000","last_clean":"2024-03-30T00:34:09.257070+0000","last_became_active":"2024-03-30T00:33:15.166888+0000","last_became_peered":"2024-03-30T00:33:15.166888+0000","last_unstale":"2024-03-30T00:34:09.257070+0000","last_undegraded":"2024-03-30T00:34:09.257070+0000","last_fullsized":"2024-03-30T00:34:09.257070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:00:32.571281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254643+0000","last_change":"2024-03-30T00:33:15.178388+0000","last_active":"2024-03-30T00:34:09.254643+0000","last_peered":"2024-03-30T00:34:09.254643+0000","last_clean":"2024-03-30T00:34:09.254643+0000","last_became_active":"2024-03-30T00:33:15.178160+0000","last_became_peered":"2024-03-30T00:33:15.178160+0000","last_unstale":"2024-03-30T00:34:09.254643+0000","last_undegraded":"2024-03-30T00:34:09.254643+0000","last_fullsized":"2024-03-30T00:34:09.254643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:28:51.567598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264257+0000","last_change":"2024-03-30T00:33:15.675426+0000","last_active":"2024-03-30T00:34:09.264257+0000","last_peered":"2024-03-30T00:34:09.264257+0000","last_clean":"2024-03-30T00:34:09.264257+0000","last_became_active":"2024-03-30T00:33:15.675083+0000","last_became_peered":"2024-03-30T00:33:15.675083+0000","last_unstale":"2024-03-30T00:34:09.264257+0000","last_undegraded":"2024-03-30T00:34:09.264257+0000","last_fullsized":"2024-03-30T00:34:09.264257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:37:03.086331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.251872+0000","last_change":"2024-03-30T00:33:15.171537+0000","last_active":"2024-03-30T00:34:09.251872+0000","last_peered":"2024-03-30T00:34:09.251872+0000","last_clean":"2024-03-30T00:34:09.251872+0000","last_became_active":"2024-03-30T00:33:15.171128+0000","last_became_peered":"2024-03-30T00:33:15.171128+0000","last_unstale":"2024-03-30T00:34:09.251872+0000","last_undegraded":"2024-03-30T00:34:09.251872+0000","last_fullsized":"2024-03-30T00:34:09.251872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:52.062144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257830+0000","last_change":"2024-03-30T00:33:15.674075+0000","last_active":"2024-03-30T00:34:09.257830+0000","last_peered":"2024-03-30T00:34:09.257830+0000","last_clean":"2024-03-30T00:34:09.257830+0000","last_became_active":"2024-03-30T00:33:15.672400+0000","last_became_peered":"2024-03-30T00:33:15.672400+0000","last_unstale":"2024-03-30T00:34:09.257830+0000","last_undegraded":"2024-03-30T00:34:09.257830+0000","last_fullsized":"2024-03-30T00:34:09.257830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:59.309835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.261775+0000","last_change":"2024-03-30T00:33:15.168245+0000","last_active":"2024-03-30T00:34:09.261775+0000","last_peered":"2024-03-30T00:34:09.261775+0000","last_clean":"2024-03-30T00:34:09.261775+0000","last_became_active":"2024-03-30T00:33:15.167852+0000","last_became_peered":"2024-03-30T00:33:15.167852+0000","last_unstale":"2024-03-30T00:34:09.261775+0000","last_undegraded":"2024-03-30T00:34:09.261775+0000","last_fullsized":"2024-03-30T00:34:09.261775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:33:28.780614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.305258+0000","last_change":"2024-03-30T00:34:10.305258+0000","last_active":"2024-03-30T00:34:10.305258+0000","last_peered":"2024-03-30T00:34:10.305258+0000","last_clean":"2024-03-30T00:34:10.305258+0000","last_became_active":"2024-03-30T00:34:10.304913+0000","last_became_peered":"2024-03-30T00:34:10.304913+0000","last_unstale":"2024-03-30T00:34:10.305258+0000","last_undegraded":"2024-03-30T00:34:10.305258+0000","last_fullsized":"2024-03-30T00:34:10.305258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:29:08.202307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.263955+0000","last_change":"2024-03-30T00:33:15.170052+0000","last_active":"2024-03-30T00:34:09.263955+0000","last_peered":"2024-03-30T00:34:09.263955+0000","last_clean":"2024-03-30T00:34:09.263955+0000","last_became_active":"2024-03-30T00:33:15.169706+0000","last_became_peered":"2024-03-30T00:33:15.169706+0000","last_unstale":"2024-03-30T00:34:09.263955+0000","last_undegraded":"2024-03-30T00:34:09.263955+0000","last_fullsized":"2024-03-30T00:34:09.263955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:32.043071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.014521+0000","last_change":"2024-03-30T00:33:15.667061+0000","last_active":"2024-03-30T00:34:39.014521+0000","last_peered":"2024-03-30T00:34:39.014521+0000","last_clean":"2024-03-30T00:34:39.014521+0000","last_became_active":"2024-03-30T00:33:15.666763+0000","last_became_peered":"2024-03-30T00:33:15.666763+0000","last_unstale":"2024-03-30T00:34:39.014521+0000","last_undegraded":"2024-03-30T00:34:39.014521+0000","last_fullsized":"2024-03-30T00:34:39.014521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:46.558297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257420+0000","last_change":"2024-03-30T00:33:15.176152+0000","last_active":"2024-03-30T00:34:09.257420+0000","last_peered":"2024-03-30T00:34:09.257420+0000","last_clean":"2024-03-30T00:34:09.257420+0000","last_became_active":"2024-03-30T00:33:15.175640+0000","last_became_peered":"2024-03-30T00:33:15.175640+0000","last_unstale":"2024-03-30T00:34:09.257420+0000","last_undegraded":"2024-03-30T00:34:09.257420+0000","last_fullsized":"2024-03-30T00:34:09.257420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:58:11.575577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051405+0000","last_change":"2024-03-30T00:33:15.672957+0000","last_active":"2024-03-30T00:34:39.051405+0000","last_peered":"2024-03-30T00:34:39.051405+0000","last_clean":"2024-03-30T00:34:39.051405+0000","last_became_active":"2024-03-30T00:33:15.668868+0000","last_became_peered":"2024-03-30T00:33:15.668868+0000","last_unstale":"2024-03-30T00:34:39.051405+0000","last_undegraded":"2024-03-30T00:34:39.051405+0000","last_fullsized":"2024-03-30T00:34:39.051405+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:50.342693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.301965+0000","last_change":"2024-03-30T00:33:15.160345+0000","last_active":"2024-03-30T00:34:10.301965+0000","last_peered":"2024-03-30T00:34:10.301965+0000","last_clean":"2024-03-30T00:34:10.301965+0000","last_became_active":"2024-03-30T00:33:15.160026+0000","last_became_peered":"2024-03-30T00:33:15.160026+0000","last_unstale":"2024-03-30T00:34:10.301965+0000","last_undegraded":"2024-03-30T00:34:10.301965+0000","last_fullsized":"2024-03-30T00:34:10.301965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:06:51.177680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257738+0000","last_change":"2024-03-30T00:33:15.674058+0000","last_active":"2024-03-30T00:34:09.257738+0000","last_peered":"2024-03-30T00:34:09.257738+0000","last_clean":"2024-03-30T00:34:09.257738+0000","last_became_active":"2024-03-30T00:33:15.673165+0000","last_became_peered":"2024-03-30T00:33:15.673165+0000","last_unstale":"2024-03-30T00:34:09.257738+0000","last_undegraded":"2024-03-30T00:34:09.257738+0000","last_fullsized":"2024-03-30T00:34:09.257738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:14.869963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254400+0000","last_change":"2024-03-30T00:33:15.173178+0000","last_active":"2024-03-30T00:34:09.254400+0000","last_peered":"2024-03-30T00:34:09.254400+0000","last_clean":"2024-03-30T00:34:09.254400+0000","last_became_active":"2024-03-30T00:33:15.172830+0000","last_became_peered":"2024-03-30T00:33:15.172830+0000","last_unstale":"2024-03-30T00:34:09.254400+0000","last_undegraded":"2024-03-30T00:34:09.254400+0000","last_fullsized":"2024-03-30T00:34:09.254400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:49:09.823855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065387+0000","last_change":"2024-03-30T00:33:15.693485+0000","last_active":"2024-03-30T00:34:39.065387+0000","last_peered":"2024-03-30T00:34:39.065387+0000","last_clean":"2024-03-30T00:34:39.065387+0000","last_became_active":"2024-03-30T00:33:15.693066+0000","last_became_peered":"2024-03-30T00:33:15.693066+0000","last_unstale":"2024-03-30T00:34:39.065387+0000","last_undegraded":"2024-03-30T00:34:39.065387+0000","last_fullsized":"2024-03-30T00:34:39.065387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:38.161331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.262703+0000","last_change":"2024-03-30T00:33:15.170808+0000","last_active":"2024-03-30T00:34:09.262703+0000","last_peered":"2024-03-30T00:34:09.262703+0000","last_clean":"2024-03-30T00:34:09.262703+0000","last_became_active":"2024-03-30T00:33:15.170448+0000","last_became_peered":"2024-03-30T00:33:15.170448+0000","last_unstale":"2024-03-30T00:34:09.262703+0000","last_undegraded":"2024-03-30T00:34:09.262703+0000","last_fullsized":"2024-03-30T00:34:09.262703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:08:28.179041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062015+0000","last_change":"2024-03-30T00:33:15.687619+0000","last_active":"2024-03-30T00:34:39.062015+0000","last_peered":"2024-03-30T00:34:39.062015+0000","last_clean":"2024-03-30T00:34:39.062015+0000","last_became_active":"2024-03-30T00:33:15.687313+0000","last_became_peered":"2024-03-30T00:33:15.687313+0000","last_unstale":"2024-03-30T00:34:39.062015+0000","last_undegraded":"2024-03-30T00:34:39.062015+0000","last_fullsized":"2024-03-30T00:34:39.062015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:36:51.182558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258190+0000","last_change":"2024-03-30T00:33:15.168601+0000","last_active":"2024-03-30T00:34:09.258190+0000","last_peered":"2024-03-30T00:34:09.258190+0000","last_clean":"2024-03-30T00:34:09.258190+0000","last_became_active":"2024-03-30T00:33:15.168065+0000","last_became_peered":"2024-03-30T00:33:15.168065+0000","last_unstale":"2024-03-30T00:34:09.258190+0000","last_undegraded":"2024-03-30T00:34:09.258190+0000","last_fullsized":"2024-03-30T00:34:09.258190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:22:36.477280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.063142+0000","last_change":"2024-03-30T00:33:15.678299+0000","last_active":"2024-03-30T00:34:39.063142+0000","last_peered":"2024-03-30T00:34:39.063142+0000","last_clean":"2024-03-30T00:34:39.063142+0000","last_became_active":"2024-03-30T00:33:15.675499+0000","last_became_peered":"2024-03-30T00:33:15.675499+0000","last_unstale":"2024-03-30T00:34:39.063142+0000","last_undegraded":"2024-03-30T00:34:39.063142+0000","last_fullsized":"2024-03-30T00:34:39.063142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:15.317125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264445+0000","last_change":"2024-03-30T00:33:15.172058+0000","last_active":"2024-03-30T00:34:09.264445+0000","last_peered":"2024-03-30T00:34:09.264445+0000","last_clean":"2024-03-30T00:34:09.264445+0000","last_became_active":"2024-03-30T00:33:15.167857+0000","last_became_peered":"2024-03-30T00:33:15.167857+0000","last_unstale":"2024-03-30T00:34:09.264445+0000","last_undegraded":"2024-03-30T00:34:09.264445+0000","last_fullsized":"2024-03-30T00:34:09.264445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:03:35.951559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062087+0000","last_change":"2024-03-30T00:33:15.682519+0000","last_active":"2024-03-30T00:34:39.062087+0000","last_peered":"2024-03-30T00:34:39.062087+0000","last_clean":"2024-03-30T00:34:39.062087+0000","last_became_active":"2024-03-30T00:33:15.681055+0000","last_became_peered":"2024-03-30T00:33:15.681055+0000","last_unstale":"2024-03-30T00:34:39.062087+0000","last_undegraded":"2024-03-30T00:34:39.062087+0000","last_fullsized":"2024-03-30T00:34:39.062087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:38.160151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051654+0000","last_change":"2024-03-30T00:33:15.673411+0000","last_active":"2024-03-30T00:34:39.051654+0000","last_peered":"2024-03-30T00:34:39.051654+0000","last_clean":"2024-03-30T00:34:39.051654+0000","last_became_active":"2024-03-30T00:33:15.672852+0000","last_became_peered":"2024-03-30T00:33:15.672852+0000","last_unstale":"2024-03-30T00:34:39.051654+0000","last_undegraded":"2024-03-30T00:34:39.051654+0000","last_fullsized":"2024-03-30T00:34:39.051654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:55:46.705785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.055100+0000","last_change":"2024-03-30T00:33:15.680624+0000","last_active":"2024-03-30T00:34:39.055100+0000","last_peered":"2024-03-30T00:34:39.055100+0000","last_clean":"2024-03-30T00:34:39.055100+0000","last_became_active":"2024-03-30T00:33:15.679900+0000","last_became_peered":"2024-03-30T00:33:15.679900+0000","last_unstale":"2024-03-30T00:34:39.055100+0000","last_undegraded":"2024-03-30T00:34:39.055100+0000","last_fullsized":"2024-03-30T00:34:39.055100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:44.484608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051062+0000","last_change":"2024-03-30T00:33:15.672612+0000","last_active":"2024-03-30T00:34:39.051062+0000","last_peered":"2024-03-30T00:34:39.051062+0000","last_clean":"2024-03-30T00:34:39.051062+0000","last_became_active":"2024-03-30T00:33:15.671536+0000","last_became_peered":"2024-03-30T00:33:15.671536+0000","last_unstale":"2024-03-30T00:34:39.051062+0000","last_undegraded":"2024-03-30T00:34:39.051062+0000","last_fullsized":"2024-03-30T00:34:39.051062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:40.783988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260508+0000","last_change":"2024-03-30T00:33:15.681577+0000","last_active":"2024-03-30T00:34:09.260508+0000","last_peered":"2024-03-30T00:34:09.260508+0000","last_clean":"2024-03-30T00:34:09.260508+0000","last_became_active":"2024-03-30T00:33:15.681313+0000","last_became_peered":"2024-03-30T00:33:15.681313+0000","last_unstale":"2024-03-30T00:34:09.260508+0000","last_undegraded":"2024-03-30T00:34:09.260508+0000","last_fullsized":"2024-03-30T00:34:09.260508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:42:15.178957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264354+0000","last_change":"2024-03-30T00:33:15.688172+0000","last_active":"2024-03-30T00:34:09.264354+0000","last_peered":"2024-03-30T00:34:09.264354+0000","last_clean":"2024-03-30T00:34:09.264354+0000","last_became_active":"2024-03-30T00:33:15.687860+0000","last_became_peered":"2024-03-30T00:33:15.687860+0000","last_unstale":"2024-03-30T00:34:09.264354+0000","last_undegraded":"2024-03-30T00:34:09.264354+0000","last_fullsized":"2024-03-30T00:34:09.264354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:28.762798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.047772+0000","last_change":"2024-03-30T00:33:15.673868+0000","last_active":"2024-03-30T00:34:39.047772+0000","last_peered":"2024-03-30T00:34:39.047772+0000","last_clean":"2024-03-30T00:34:39.047772+0000","last_became_active":"2024-03-30T00:33:15.673467+0000","last_became_peered":"2024-03-30T00:33:15.673467+0000","last_unstale":"2024-03-30T00:34:39.047772+0000","last_undegraded":"2024-03-30T00:34:39.047772+0000","last_fullsized":"2024-03-30T00:34:39.047772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:07:39.448967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064293+0000","last_change":"2024-03-30T00:33:15.684639+0000","last_active":"2024-03-30T00:34:39.064293+0000","last_peered":"2024-03-30T00:34:39.064293+0000","last_clean":"2024-03-30T00:34:39.064293+0000","last_became_active":"2024-03-30T00:33:15.684357+0000","last_became_peered":"2024-03-30T00:33:15.684357+0000","last_unstale":"2024-03-30T00:34:39.064293+0000","last_undegraded":"2024-03-30T00:34:39.064293+0000","last_fullsized":"2024-03-30T00:34:39.064293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:02:50.995640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257545+0000","last_change":"2024-03-30T00:33:15.680810+0000","last_active":"2024-03-30T00:34:09.257545+0000","last_peered":"2024-03-30T00:34:09.257545+0000","last_clean":"2024-03-30T00:34:09.257545+0000","last_became_active":"2024-03-30T00:33:15.680457+0000","last_became_peered":"2024-03-30T00:33:15.680457+0000","last_unstale":"2024-03-30T00:34:09.257545+0000","last_undegraded":"2024-03-30T00:34:09.257545+0000","last_fullsized":"2024-03-30T00:34:09.257545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:02.575683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.061425+0000","last_change":"2024-03-30T00:34:10.305867+0000","last_active":"2024-03-30T00:34:39.061425+0000","last_peered":"2024-03-30T00:34:39.061425+0000","last_clean":"2024-03-30T00:34:39.061425+0000","last_became_active":"2024-03-30T00:34:10.304998+0000","last_became_peered":"2024-03-30T00:34:10.304998+0000","last_unstale":"2024-03-30T00:34:39.061425+0000","last_undegraded":"2024-03-30T00:34:39.061425+0000","last_fullsized":"2024-03-30T00:34:39.061425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:34.668457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257506+0000","last_change":"2024-03-30T00:33:15.671491+0000","last_active":"2024-03-30T00:34:09.257506+0000","last_peered":"2024-03-30T00:34:09.257506+0000","last_clean":"2024-03-30T00:34:09.257506+0000","last_became_active":"2024-03-30T00:33:15.671110+0000","last_became_peered":"2024-03-30T00:33:15.671110+0000","last_unstale":"2024-03-30T00:34:09.257506+0000","last_undegraded":"2024-03-30T00:34:09.257506+0000","last_fullsized":"2024-03-30T00:34:09.257506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:57.623730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.067543+0000","last_change":"2024-03-30T00:33:15.686795+0000","last_active":"2024-03-30T00:34:39.067543+0000","last_peered":"2024-03-30T00:34:39.067543+0000","last_clean":"2024-03-30T00:34:39.067543+0000","last_became_active":"2024-03-30T00:33:15.685715+0000","last_became_peered":"2024-03-30T00:33:15.685715+0000","last_unstale":"2024-03-30T00:34:39.067543+0000","last_undegraded":"2024-03-30T00:34:39.067543+0000","last_fullsized":"2024-03-30T00:34:39.067543+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:37:53.838823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264242+0000","last_change":"2024-03-30T00:33:15.687369+0000","last_active":"2024-03-30T00:34:09.264242+0000","last_peered":"2024-03-30T00:34:09.264242+0000","last_clean":"2024-03-30T00:34:09.264242+0000","last_became_active":"2024-03-30T00:33:15.686852+0000","last_became_peered":"2024-03-30T00:33:15.686852+0000","last_unstale":"2024-03-30T00:34:09.264242+0000","last_undegraded":"2024-03-30T00:34:09.264242+0000","last_fullsized":"2024-03-30T00:34:09.264242+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:09:21.003739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.058661+0000","last_change":"2024-03-30T00:33:15.676840+0000","last_active":"2024-03-30T00:34:39.058661+0000","last_peered":"2024-03-30T00:34:39.058661+0000","last_clean":"2024-03-30T00:34:39.058661+0000","last_became_active":"2024-03-30T00:33:15.675884+0000","last_became_peered":"2024-03-30T00:33:15.675884+0000","last_unstale":"2024-03-30T00:34:39.058661+0000","last_undegraded":"2024-03-30T00:34:39.058661+0000","last_fullsized":"2024-03-30T00:34:39.058661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:02.900843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73836,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":513,"num_read_kb":332,"num_write":168,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":221508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36120,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709032,"statfs":{"total":95995035648,"available":95958048768,"internally_reserved":0,"allocated":933888,"data_stored":446772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.622,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.872,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":1.167,"15min":1.167},"last":0.503}]},{"osd":1,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.60899999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1020000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.67900000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0229999999999999,"5min":1.623,"15min":1.623},"last":0.90800000000000003}]},{"osd":2,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88600000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.44900000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.028,"5min":1.304,"15min":1.304},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83799999999999997,"15min":0.83599999999999997},"min":{"1min":0.64200000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.0069999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.623},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.052,"15min":1.0880000000000001},"min":{"1min":0.64400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.98699999999999999,"5min":6.6029999999999998,"15min":6.6029999999999998},"last":0.80300000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.71399999999999997,"15min":0.72899999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84999999999999998,"5min":1.486,"15min":1.486},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.099,"5min":1.536,"15min":1.536},"last":0.749}]},{"osd":6,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.81999999999999995,"15min":0.83399999999999996},"min":{"1min":0.504,"5min":0.501,"15min":0.501},"max":{"1min":1.052,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.84099999999999997,"15min":0.86199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93799999999999994,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.96099999999999997},"min":{"1min":0.65100000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.145,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.995,"15min":1.0109999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1359999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.93899999999999995}]},{"osd":8,"last update":"Sat Mar 30 00:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.96499999999999997}]},{"osd":9,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":1.1200000000000001,"15min":1.1679999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0089999999999999,"5min":6.6399999999999997,"15min":6.6399999999999997},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":0.96199999999999997},"min":{"1min":0.71799999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3560000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.91700000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.75900000000000001,"5min":0.754,"15min":0.754},"max":{"1min":1.262,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.95599999999999996,"15min":0.96499999999999997},"min":{"1min":0.73599999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.234,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.85399999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678050,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1107750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.5219999999999998,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.48899999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.56200000000000006}]},{"osd":2,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.46600000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.74399999999999999,"15min":0.73899999999999999},"min":{"1min":0.63400000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.075,"5min":1.171,"15min":1.171},"last":0.60399999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.43099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.497,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1160000000000001,"5min":1.194,"15min":1.194},"last":0.66900000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.83699999999999997,"15min":0.83499999999999996},"min":{"1min":0.58599999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.626},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.80000000000000004,"15min":0.78600000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.79800000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.86699999999999999,"15min":0.83099999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":6.4870000000000001,"5min":6.4870000000000001,"15min":6.4870000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.81699999999999995,"15min":0.80400000000000005},"min":{"1min":0.65100000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.97999999999999998}]},{"osd":8,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.879,"15min":0.86099999999999999},"min":{"1min":0.72499999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88100000000000001,"15min":0.88300000000000001},"min":{"1min":0.66200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1679999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.58499999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.89500000000000002,"15min":0.88300000000000001},"min":{"1min":0.755,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.90600000000000003,"15min":0.89900000000000002},"min":{"1min":0.61199999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":3.508,"5min":3.508,"15min":3.508},"last":0.76100000000000001}]},{"osd":11,"last update":"Sat Mar 30 00:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0820000000000001,"5min":1.526,"15min":1.526},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.91500000000000004,"15min":0.92800000000000005},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.034,"5min":2.262,"15min":2.262},"last":0.93500000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":447008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71299999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.439}]},{"osd":1,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.77800000000000002,"15min":0.78900000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0329999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.82299999999999995,"15min":0.83799999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.018,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.69299999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82799999999999996,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.66000000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.95399999999999996,"5min":1.278,"15min":1.278},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.91800000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.40899999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.86499999999999999,"15min":0.88800000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98499999999999999,"5min":1.476,"15min":1.476},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.85699999999999998,"15min":0.878},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92900000000000005,"5min":1.254,"15min":1.254},"last":0.76400000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.87,"15min":0.88600000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.89600000000000002,"15min":0.92200000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94799999999999995,"5min":1.577,"15min":1.577},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.86899999999999999,"15min":0.89500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0600000000000001,"5min":1.486,"15min":1.486},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.90200000000000002,"15min":0.92900000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.073,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.93899999999999995,"15min":0.96299999999999997},"min":{"1min":0.53100000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0669999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.028,"15min":1.0620000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.083,"5min":2.234,"15min":2.234},"last":0.74399999999999999}]},{"osd":8,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.998,"15min":1.024},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.095,"5min":1.611,"15min":1.611},"last":0.63},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.0369999999999999,"15min":1.075},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.998,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.79100000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.0329999999999999,"5min":1.355,"15min":1.355},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":1.002},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1990000000000001,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.77300000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":446928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":3.8370000000000002,"5min":3.8370000000000002,"15min":3.8370000000000002},"last":0.72299999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.75900000000000001,"15min":0.747},"min":{"1min":0.56699999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81899999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83799999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65700000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.629},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.8570000000000002,"5min":3.8570000000000002,"15min":3.8570000000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.90600000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.78300000000000003,"15min":0.77200000000000002},"min":{"1min":0.628,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0369999999999999,"5min":1.246,"15min":1.246},"last":0.90200000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95199999999999996,"5min":1.026,"15min":1.026},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92200000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.47599999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79900000000000004,"15min":0.81699999999999995},"min":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.98599999999999999,"5min":1.054,"15min":1.054},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.86199999999999999,"15min":0.87},"min":{"1min":0.61399999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.7650000000000001,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.75}]},{"osd":7,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.71699999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.83799999999999997,"15min":0.84599999999999997},"min":{"1min":0.58199999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0469999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83199999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.86299999999999999,"15min":0.84199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.8969999999999998,"5min":3.8969999999999998,"15min":3.8969999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.8250000000000002,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83599999999999997,"15min":0.83399999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.88200000000000001,"15min":0.86599999999999999},"min":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.93999999999999995}]},{"osd":11,"last update":"Sat Mar 30 00:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.85799999999999998}]}]},{"osd":7,"up_from":50,"seq":214748364846,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1544192,"data_stored":1039363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93899999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.746}]},{"osd":2,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89500000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.0049999999999999,"5min":1.111,"15min":1.111},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93400000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68400000000000005,"15min":0.67200000000000004},"min":{"1min":0.47499999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97599999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.81100000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68500000000000005,"15min":0.66300000000000003},"min":{"1min":0.45300000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0209999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94099999999999995,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.84199999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.76900000000000002,"15min":0.78600000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94799999999999995,"5min":1.095,"15min":1.095},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78500000000000003,"15min":0.78700000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.96899999999999997,"5min":1.252,"15min":1.252},"last":0.68600000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78500000000000003,"15min":0.77800000000000002},"min":{"1min":0.58099999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98999999999999999,"5min":1.254,"15min":1.254},"last":0.879},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.78100000000000003,"15min":0.78800000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.88400000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.65400000000000003}]},{"osd":8,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84399999999999997,"15min":0.85199999999999998},"min":{"1min":0.63300000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.026,"5min":1.236,"15min":1.236},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.84199999999999997,"15min":0.83599999999999997},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0640000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.91100000000000003}]},{"osd":9,"last update":"Sat Mar 30 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86899999999999999,"15min":0.877},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.028,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.85499999999999998,"15min":0.86899999999999999},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98799999999999999,"5min":1.284,"15min":1.284},"last":0.89100000000000001}]},{"osd":10,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.88900000000000001,"15min":0.89900000000000002},"min":{"1min":0.68200000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0009999999999999,"5min":1.321,"15min":1.321},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.91100000000000003,"15min":0.91600000000000004},"min":{"1min":0.63300000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.048,"5min":1.3,"15min":1.3},"last":0.96699999999999997}]},{"osd":11,"last update":"Sat Mar 30 00:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.90200000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":937984,"data_stored":446397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64900000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77700000000000002,"5min":0.873,"15min":0.873},"last":0.436}]},{"osd":1,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.46000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.626},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71399999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91100000000000003,"5min":1.002,"15min":1.002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0069999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.54400000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.877,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.497}]},{"osd":4,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.048,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.749,"15min":0.749},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.96899999999999997,"5min":1.238,"15min":1.238},"last":0.749}]},{"osd":5,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0329999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.55900000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1080000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.53800000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80600000000000005,"15min":0.80800000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90500000000000003,"5min":1.105,"15min":1.105},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.99099999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77300000000000002}]},{"osd":8,"last update":"Sat Mar 30 00:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.85799999999999998,"15min":0.89500000000000002},"min":{"1min":0.60399999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.96999999999999997,"5min":1.252,"15min":1.252},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.82799999999999996,"15min":0.85799999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95099999999999996,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.56699999999999995}]},{"osd":9,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.84299999999999997,"15min":0.86499999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.89400000000000002,"15min":0.90900000000000003},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":0.97799999999999998,"5min":1.155,"15min":1.155},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.88600000000000001,"15min":0.89600000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0680000000000001,"5min":1.206,"15min":1.206},"last":0.84199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":1007616,"data_stored":515520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64000000000000001,"15min":0.59699999999999998},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80100000000000005,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.498},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64400000000000002,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:34:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.66200000000000003,"15min":0.58699999999999997},"min":{"1min":0.495,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94199999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.70599999999999996,"15min":0.61799999999999999},"min":{"1min":0.54500000000000004,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0640000000000001,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.85399999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65100000000000002,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.125,"15min":1.125},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64000000000000001,"15min":0.59699999999999998},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.98899999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.65600000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.52700000000000002,"15min":0.48999999999999999},"min":{"1min":0.40500000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89400000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57399999999999995,"15min":0.51800000000000002},"min":{"1min":0.45800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91600000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.80600000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.70099999999999996,"15min":0.65000000000000002},"min":{"1min":0.65400000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.99099999999999999,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.876},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71499999999999997,"15min":0.69599999999999995},"min":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.079,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.52400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93100000000000005,"5min":1.046,"15min":1.046},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68700000000000006,"15min":0.65700000000000003},"min":{"1min":0.47899999999999998,"5min":0.438,"15min":0.438},"max":{"1min":1.099,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.90800000000000003}]},{"osd":8,"last update":"Sat Mar 30 00:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78100000000000003,"15min":0.75900000000000001},"min":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88200000000000001,"15min":0.88300000000000001},"min":{"1min":0.67200000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.159,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.95099999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.873,"15min":0.871},"min":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.125,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.018},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1200000000000001,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.92600000000000005}]},{"osd":10,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1679999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.87,"15min":0.86199999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.98799999999999999}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0409999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":958464,"data_stored":447668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78400000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68899999999999995,"15min":0.60299999999999998},"min":{"1min":0.42399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.619,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.68600000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6819999999999999,"5min":5.8550000000000004,"15min":5.8550000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.80300000000000005},"min":{"1min":0.442,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90800000000000003,"5min":5.282,"15min":5.282},"last":0.75900000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.621}]},{"osd":4,"last update":"Sat Mar 30 00:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.65900000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.58499999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:34:27 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.58799999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.79000000000000004,"15min":0.91300000000000003},"min":{"1min":0.46800000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89300000000000002,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.69199999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.377,"15min":0.377},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.629,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.113,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.48199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.125,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2470000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Mar 30 00:34:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.754,"15min":0.79000000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.84799999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.83299999999999996,"15min":0.874},"min":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.70599999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.84599999999999997,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":1.161,"15min":1.161},"last":0.54000000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.84499999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1899999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.90500000000000003,"15min":0.91200000000000003},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.77200000000000002}]},{"osd":11,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77000000000000002,"15min":0.76300000000000001},"min":{"1min":0.63900000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.749}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":1032192,"data_stored":517318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.623,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82499999999999996,"5min":1.131,"15min":1.131},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.45800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.63800000000000001,"15min":0.59699999999999998},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.501},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.70399999999999996,"15min":0.63800000000000001},"min":{"1min":0.56200000000000006,"5min":0.375,"15min":0.375},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68200000000000005,"15min":0.66800000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93799999999999994,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.45000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.68999999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.91400000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.59899999999999998,"15min":0.53100000000000003},"min":{"1min":0.49299999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0289999999999999,"5min":1.147,"15min":1.147},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.45700000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67900000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77300000000000002,"15min":0.74299999999999999},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0309999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72099999999999997,"15min":0.69699999999999995},"min":{"1min":0.55900000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.85599999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.65700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.83299999999999996}]},{"osd":8,"last update":"Sat Mar 30 00:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.74399999999999999,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97199999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.85699999999999998,"15min":0.83099999999999996},"min":{"1min":0.67000000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.052,"5min":1.321,"15min":1.321},"last":0.71799999999999997}]},{"osd":9,"last update":"Sat Mar 30 00:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95799999999999996,"15min":0.95999999999999996},"min":{"1min":0.68500000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.137,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.90800000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.88200000000000001,"15min":0.872},"min":{"1min":0.77300000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.1439999999999999,"5min":1.204,"15min":1.204},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.004,"15min":0.996},"min":{"1min":0.85499999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.218,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.0429999999999999}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.0129999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":446708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42499999999999999,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46800000000000003,"15min":0.437},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.40200000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.622,"15min":0.58599999999999997},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.56499999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68300000000000005,"15min":0.63600000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:34:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.48999999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.748,"15min":0.69999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.79500000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.65500000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78100000000000003,"15min":0.77400000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.74299999999999999}]},{"osd":8,"last update":"Sat Mar 30 00:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.66700000000000004}]},{"osd":9,"last update":"Sat Mar 30 00:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.87,"15min":0.88400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0209999999999999,"5min":1.099,"15min":1.099},"last":0.502}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.85399999999999998,"15min":0.86599999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.71899999999999997}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.82499999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":942080,"data_stored":447282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.497}]},{"osd":1,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48599999999999999,"15min":0.45200000000000001},"min":{"1min":0.372,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.48699999999999999,"15min":0.45500000000000002},"min":{"1min":0.39200000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71399999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65600000000000003,"15min":0.64300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.55900000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.81200000000000006}]},{"osd":3,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69999999999999996,"15min":0.65800000000000003},"min":{"1min":0.54700000000000004,"5min":0.439,"15min":0.439},"max":{"1min":0.97699999999999998,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68899999999999995,"15min":0.66300000000000003},"min":{"1min":0.52800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83999999999999997,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.94699999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70499999999999996,"15min":0.68300000000000005},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0049999999999999,"5min":1.137,"15min":1.137},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71399999999999997,"15min":0.67600000000000005},"min":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0720000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.76100000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0369999999999999,"5min":1.071,"15min":1.071},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82399999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68999999999999995,"15min":0.66200000000000003},"min":{"1min":0.51500000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.69699999999999995,"15min":0.67400000000000004},"min":{"1min":0.54200000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.70899999999999996}]},{"osd":8,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.83199999999999996,"15min":0.82399999999999995},"min":{"1min":0.64000000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0589999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.79400000000000004,"15min":0.77700000000000002},"min":{"1min":0.64600000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.026,"5min":1.153,"15min":1.153},"last":0.879}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.86499999999999999,"15min":0.88500000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97499999999999998,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.91200000000000003,"15min":0.92900000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0549999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.91400000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.872,"15min":0.87},"min":{"1min":0.69199999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.167,"5min":1.179,"15min":1.179},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.871,"15min":0.879},"min":{"1min":0.57499999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.73299999999999998}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.95999999999999996}]}]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32628,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712524,"statfs":{"total":95995035648,"available":95961624576,"internally_reserved":0,"allocated":1552384,"data_stored":1038268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.71599999999999997,"15min":0.81299999999999994},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88800000000000001,"5min":8.077,"15min":8.077},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.64900000000000002,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.67600000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.64100000000000001,"15min":0.61899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95199999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.69299999999999995,"15min":0.64800000000000002},"min":{"1min":0.52800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.82299999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.45400000000000001},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.503},"min":{"1min":0.38600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85399999999999998,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.47299999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.504},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.753,"15min":0.86699999999999999},"min":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94199999999999995,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":0.54600000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.73199999999999998,"15min":0.67800000000000005},"min":{"1min":0.55300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1419999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69799999999999995,"15min":0.67400000000000004},"min":{"1min":0.54500000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94199999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.748}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64700000000000002,"15min":0.629},"min":{"1min":0.56000000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84999999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.72399999999999998,"15min":0.68600000000000005},"min":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.73099999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.57199999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.72999999999999998,"15min":0.71399999999999997},"min":{"1min":0.623,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.93899999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89000000000000001}]},{"osd":8,"last update":"Sat Mar 30 00:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.77400000000000002,"15min":0.72999999999999998},"min":{"1min":0.65400000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.73699999999999999,"15min":0.70099999999999996},"min":{"1min":0.621,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.70899999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.875,"15min":0.86899999999999999},"min":{"1min":0.70699999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.83499999999999996,"15min":0.83399999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.071,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.84699999999999998}]},{"osd":10,"last update":"Sat Mar 30 00:34:12 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.86499999999999999,"15min":0.85899999999999999},"min":{"1min":0.71199999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.0589999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.872},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.79700000000000004,"15min":0.78900000000000003},"min":{"1min":0.67200000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.83099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:34:44.430 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-30T00:34:44.431 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T00:34:44.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.649+0000 7f01c3c00700 1 -- 172.21.15.99:0/1912771568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01bc102810 msgr2=0x7f01bc1090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:44.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.649+0000 7f01c3c00700 1 --2- 172.21.15.99:0/1912771568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01bc102810 0x7f01bc1090c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f01b400b0d0 tx=0x7f01b400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.649+0000 7f01c3c00700 1 -- 172.21.15.99:0/1912771568 shutdown_connections 2024-03-30T00:34:44.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.649+0000 7f01c3c00700 1 --2- 172.21.15.99:0/1912771568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01bc102810 0x7f01bc1090c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.649+0000 7f01c3c00700 1 --2- 172.21.15.99:0/1912771568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01bc100c80 0x7f01bc1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.649+0000 7f01c3c00700 1 --2- 172.21.15.99:0/1912771568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01bc101e70 0x7f01bc1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.649+0000 7f01c3c00700 1 -- 172.21.15.99:0/1912771568 >> 172.21.15.99:0/1912771568 conn(0x7f01bc0fc1f0 msgr2=0x7f01bc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:44.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.649+0000 7f01c3c00700 1 -- 172.21.15.99:0/1912771568 shutdown_connections 2024-03-30T00:34:44.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.649+0000 7f01c3c00700 1 -- 172.21.15.99:0/1912771568 wait complete. 2024-03-30T00:34:44.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.650+0000 7f01c3c00700 1 Processor -- start 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.650+0000 7f01c3c00700 1 -- start start 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.650+0000 7f01c3c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01bc100c80 0x7f01bc198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.650+0000 7f01c3c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01bc101e70 0x7f01bc199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.650+0000 7f01c3c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01bc102810 0x7f01bc1a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.650+0000 7f01c3c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc1a0930 con 0x7f01bc101e70 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.650+0000 7f01c3c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc1a0a70 con 0x7f01bc100c80 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.650+0000 7f01c3c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc1a0c00 con 0x7f01bc102810 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.650+0000 7f01bbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01bc101e70 0x7f01bc199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01bbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01bc101e70 0x7f01bc199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53004/0 (socket says 172.21.15.99:53004) 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01bbe00700 1 -- 172.21.15.99:0/1363440926 learned_addr learned my addr 172.21.15.99:0/1363440926 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01c1200700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01bc100c80 0x7f01bc198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01c1c00700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01bc102810 0x7f01bc1a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01bbe00700 1 -- 172.21.15.99:0/1363440926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01bc102810 msgr2=0x7f01bc1a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01bbe00700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01bc102810 0x7f01bc1a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01bbe00700 1 -- 172.21.15.99:0/1363440926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01bc100c80 msgr2=0x7f01bc198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01bbe00700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01bc100c80 0x7f01bc198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01bbe00700 1 -- 172.21.15.99:0/1363440926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4009d00 con 0x7f01bc101e70 2024-03-30T00:34:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01bbe00700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01bc101e70 0x7f01bc199390 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f01a800e9a0 tx=0x7f01a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01b9600700 1 -- 172.21.15.99:0/1363440926 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8009790 con 0x7f01bc101e70 2024-03-30T00:34:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.651+0000 7f01b9600700 1 -- 172.21.15.99:0/1363440926 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01a8010430 con 0x7f01bc101e70 2024-03-30T00:34:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.652+0000 7f01c3c00700 1 -- 172.21.15.99:0/1363440926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc1a0ee0 con 0x7f01bc101e70 2024-03-30T00:34:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.652+0000 7f01b9600700 1 -- 172.21.15.99:0/1363440926 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a800cea0 con 0x7f01bc101e70 2024-03-30T00:34:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.652+0000 7f01c3c00700 1 -- 172.21.15.99:0/1363440926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc1a1360 con 0x7f01bc101e70 2024-03-30T00:34:44.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.653+0000 7f01c3c00700 1 -- 172.21.15.99:0/1363440926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01bc067a10 con 0x7f01bc101e70 2024-03-30T00:34:44.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.654+0000 7f01b9600700 1 -- 172.21.15.99:0/1363440926 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f01a80105a0 con 0x7f01bc101e70 2024-03-30T00:34:44.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.656+0000 7f01b9600700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f01a40a1120 0x7f01a40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:44.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.656+0000 7f01b9600700 1 -- 172.21.15.99:0/1363440926 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f01a8014070 con 0x7f01bc101e70 2024-03-30T00:34:44.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.656+0000 7f01c1200700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f01a40a1120 0x7f01a40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:44.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.657+0000 7f01c1200700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f01a40a1120 0x7f01a40a35e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f01bc101cd0 tx=0x7f01b0007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:44.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.659+0000 7f01b9600700 1 -- 172.21.15.99:0/1363440926 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f01a80529f0 con 0x7f01bc101e70 2024-03-30T00:34:44.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:44 smithi161 ceph-mon[42221]: pgmap v302: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-30T00:34:44.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:44 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3595719086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:44.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:44 smithi099 ceph-mon[38079]: pgmap v302: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-30T00:34:44.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:44 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3595719086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:44.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:44 smithi142 ceph-mon[41779]: pgmap v302: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-30T00:34:44.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:44 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3595719086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:44.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.862+0000 7f01c3c00700 1 -- 172.21.15.99:0/1363440926 --> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01bc074700 con 0x7f01a40a1120 2024-03-30T00:34:44.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.872+0000 7f01b9600700 1 -- 172.21.15.99:0/1363440926 <== mgr.14150 v2:172.21.15.99:6800/1501731896 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343408 (secure 0 0 0) 0x7f01bc074700 con 0x7f01a40a1120 2024-03-30T00:34:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:44.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 -- 172.21.15.99:0/1363440926 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f01a40a1120 msgr2=0x7f01a40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:44.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f01a40a1120 0x7f01a40a35e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f01bc101cd0 tx=0x7f01b0007400 comp rx=0 tx=0).stop 2024-03-30T00:34:44.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 -- 172.21.15.99:0/1363440926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01bc101e70 msgr2=0x7f01bc199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:44.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01bc101e70 0x7f01bc199390 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f01a800e9a0 tx=0x7f01a800ecb0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 -- 172.21.15.99:0/1363440926 shutdown_connections 2024-03-30T00:34:44.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01bc102810 0x7f01bc1a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f01a40a1120 0x7f01a40a35e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01bc100c80 0x7f01bc198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 --2- 172.21.15.99:0/1363440926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01bc101e70 0x7f01bc199390 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:44.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 -- 172.21.15.99:0/1363440926 >> 172.21.15.99:0/1363440926 conn(0x7f01bc0fc1f0 msgr2=0x7f01bc072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:44.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 -- 172.21.15.99:0/1363440926 shutdown_connections 2024-03-30T00:34:44.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:44.877+0000 7f01c3c00700 1 -- 172.21.15.99:0/1363440926 wait complete. 2024-03-30T00:34:44.888 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-30T00:34:44.901 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-03-30T00:34:43.029365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664438,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":639,"num_read_kb":441,"num_write":351,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":631,"ondisk_log_size":631,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378260,"kb_used_data":13084,"kb_used_omap":18,"kb_used_meta":365037,"kb_avail":1124563564,"statfs":{"total":1151940427776,"available":1151553089536,"internally_reserved":0,"allocated":13398016,"data_stored":7346982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":373798214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6051,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":38,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003339"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064138+0000","last_change":"2024-03-30T00:33:15.686814+0000","last_active":"2024-03-30T00:34:39.064138+0000","last_peered":"2024-03-30T00:34:39.064138+0000","last_clean":"2024-03-30T00:34:39.064138+0000","last_became_active":"2024-03-30T00:33:15.686432+0000","last_became_peered":"2024-03-30T00:33:15.686432+0000","last_unstale":"2024-03-30T00:34:39.064138+0000","last_undegraded":"2024-03-30T00:34:39.064138+0000","last_fullsized":"2024-03-30T00:34:39.064138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:35:05.898973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260589+0000","last_change":"2024-03-30T00:33:15.678089+0000","last_active":"2024-03-30T00:34:09.260589+0000","last_peered":"2024-03-30T00:34:09.260589+0000","last_clean":"2024-03-30T00:34:09.260589+0000","last_became_active":"2024-03-30T00:33:15.677254+0000","last_became_peered":"2024-03-30T00:33:15.677254+0000","last_unstale":"2024-03-30T00:34:09.260589+0000","last_undegraded":"2024-03-30T00:34:09.260589+0000","last_fullsized":"2024-03-30T00:34:09.260589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:55:45.922652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.305494+0000","last_change":"2024-03-30T00:34:10.305494+0000","last_active":"2024-03-30T00:34:10.305494+0000","last_peered":"2024-03-30T00:34:10.305494+0000","last_clean":"2024-03-30T00:34:10.305494+0000","last_became_active":"2024-03-30T00:34:10.305141+0000","last_became_peered":"2024-03-30T00:34:10.305141+0000","last_unstale":"2024-03-30T00:34:10.305494+0000","last_undegraded":"2024-03-30T00:34:10.305494+0000","last_fullsized":"2024-03-30T00:34:10.305494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:25:18.167062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,8],"acting":[0,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.058589+0000","last_change":"2024-03-30T00:33:15.681237+0000","last_active":"2024-03-30T00:34:39.058589+0000","last_peered":"2024-03-30T00:34:39.058589+0000","last_clean":"2024-03-30T00:34:39.058589+0000","last_became_active":"2024-03-30T00:33:15.680669+0000","last_became_peered":"2024-03-30T00:33:15.680669+0000","last_unstale":"2024-03-30T00:34:39.058589+0000","last_undegraded":"2024-03-30T00:34:39.058589+0000","last_fullsized":"2024-03-30T00:34:39.058589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:59.300262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.037005+0000","last_change":"2024-03-30T00:33:15.678825+0000","last_active":"2024-03-30T00:34:39.037005+0000","last_peered":"2024-03-30T00:34:39.037005+0000","last_clean":"2024-03-30T00:34:39.037005+0000","last_became_active":"2024-03-30T00:33:15.676864+0000","last_became_peered":"2024-03-30T00:33:15.676864+0000","last_unstale":"2024-03-30T00:34:39.037005+0000","last_undegraded":"2024-03-30T00:34:39.037005+0000","last_fullsized":"2024-03-30T00:34:39.037005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:12:09.724441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.060702+0000","last_change":"2024-03-30T00:33:15.681866+0000","last_active":"2024-03-30T00:34:39.060702+0000","last_peered":"2024-03-30T00:34:39.060702+0000","last_clean":"2024-03-30T00:34:39.060702+0000","last_became_active":"2024-03-30T00:33:15.678118+0000","last_became_peered":"2024-03-30T00:33:15.678118+0000","last_unstale":"2024-03-30T00:34:39.060702+0000","last_undegraded":"2024-03-30T00:34:39.060702+0000","last_fullsized":"2024-03-30T00:34:39.060702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:24.433027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051906+0000","last_change":"2024-03-30T00:33:15.675999+0000","last_active":"2024-03-30T00:34:39.051906+0000","last_peered":"2024-03-30T00:34:39.051906+0000","last_clean":"2024-03-30T00:34:39.051906+0000","last_became_active":"2024-03-30T00:33:15.675319+0000","last_became_peered":"2024-03-30T00:33:15.675319+0000","last_unstale":"2024-03-30T00:34:39.051906+0000","last_undegraded":"2024-03-30T00:34:39.051906+0000","last_fullsized":"2024-03-30T00:34:39.051906+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:52:42.668747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257311+0000","last_change":"2024-03-30T00:33:15.679211+0000","last_active":"2024-03-30T00:34:09.257311+0000","last_peered":"2024-03-30T00:34:09.257311+0000","last_clean":"2024-03-30T00:34:09.257311+0000","last_became_active":"2024-03-30T00:33:15.675061+0000","last_became_peered":"2024-03-30T00:33:15.675061+0000","last_unstale":"2024-03-30T00:34:09.257311+0000","last_undegraded":"2024-03-30T00:34:09.257311+0000","last_fullsized":"2024-03-30T00:34:09.257311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:43.973542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064325+0000","last_change":"2024-03-30T00:33:15.680842+0000","last_active":"2024-03-30T00:34:39.064325+0000","last_peered":"2024-03-30T00:34:39.064325+0000","last_clean":"2024-03-30T00:34:39.064325+0000","last_became_active":"2024-03-30T00:33:15.680307+0000","last_became_peered":"2024-03-30T00:33:15.680307+0000","last_unstale":"2024-03-30T00:34:39.064325+0000","last_undegraded":"2024-03-30T00:34:39.064325+0000","last_fullsized":"2024-03-30T00:34:39.064325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:59:40.331779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.060850+0000","last_change":"2024-03-30T00:33:15.692397+0000","last_active":"2024-03-30T00:34:39.060850+0000","last_peered":"2024-03-30T00:34:39.060850+0000","last_clean":"2024-03-30T00:34:39.060850+0000","last_became_active":"2024-03-30T00:33:15.691226+0000","last_became_peered":"2024-03-30T00:33:15.691226+0000","last_unstale":"2024-03-30T00:34:39.060850+0000","last_undegraded":"2024-03-30T00:34:39.060850+0000","last_fullsized":"2024-03-30T00:34:39.060850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:48.816644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.265637+0000","last_change":"2024-03-30T00:33:15.688417+0000","last_active":"2024-03-30T00:34:10.265637+0000","last_peered":"2024-03-30T00:34:10.265637+0000","last_clean":"2024-03-30T00:34:10.265637+0000","last_became_active":"2024-03-30T00:33:15.687755+0000","last_became_peered":"2024-03-30T00:33:15.687755+0000","last_unstale":"2024-03-30T00:34:10.265637+0000","last_undegraded":"2024-03-30T00:34:10.265637+0000","last_fullsized":"2024-03-30T00:34:10.265637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:30:19.782280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.061117+0000","last_change":"2024-03-30T00:33:15.681942+0000","last_active":"2024-03-30T00:34:39.061117+0000","last_peered":"2024-03-30T00:34:39.061117+0000","last_clean":"2024-03-30T00:34:39.061117+0000","last_became_active":"2024-03-30T00:33:15.679854+0000","last_became_peered":"2024-03-30T00:33:15.679854+0000","last_unstale":"2024-03-30T00:34:39.061117+0000","last_undegraded":"2024-03-30T00:34:39.061117+0000","last_fullsized":"2024-03-30T00:34:39.061117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:09:19.771959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264276+0000","last_change":"2024-03-30T00:33:15.682968+0000","last_active":"2024-03-30T00:34:09.264276+0000","last_peered":"2024-03-30T00:34:09.264276+0000","last_clean":"2024-03-30T00:34:09.264276+0000","last_became_active":"2024-03-30T00:33:15.681216+0000","last_became_peered":"2024-03-30T00:33:15.681216+0000","last_unstale":"2024-03-30T00:34:09.264276+0000","last_undegraded":"2024-03-30T00:34:09.264276+0000","last_fullsized":"2024-03-30T00:34:09.264276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:14:50.038843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260360+0000","last_change":"2024-03-30T00:33:15.677727+0000","last_active":"2024-03-30T00:34:09.260360+0000","last_peered":"2024-03-30T00:34:09.260360+0000","last_clean":"2024-03-30T00:34:09.260360+0000","last_became_active":"2024-03-30T00:33:15.676758+0000","last_became_peered":"2024-03-30T00:33:15.676758+0000","last_unstale":"2024-03-30T00:34:09.260360+0000","last_undegraded":"2024-03-30T00:34:09.260360+0000","last_fullsized":"2024-03-30T00:34:09.260360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:41:34.745920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260097+0000","last_change":"2024-03-30T00:33:15.684545+0000","last_active":"2024-03-30T00:34:09.260097+0000","last_peered":"2024-03-30T00:34:09.260097+0000","last_clean":"2024-03-30T00:34:09.260097+0000","last_became_active":"2024-03-30T00:33:15.682989+0000","last_became_peered":"2024-03-30T00:33:15.682989+0000","last_unstale":"2024-03-30T00:34:09.260097+0000","last_undegraded":"2024-03-30T00:34:09.260097+0000","last_fullsized":"2024-03-30T00:34:09.260097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:11.120981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.067066+0000","last_change":"2024-03-30T00:33:15.678122+0000","last_active":"2024-03-30T00:34:39.067066+0000","last_peered":"2024-03-30T00:34:39.067066+0000","last_clean":"2024-03-30T00:34:39.067066+0000","last_became_active":"2024-03-30T00:33:15.677681+0000","last_became_peered":"2024-03-30T00:33:15.677681+0000","last_unstale":"2024-03-30T00:34:39.067066+0000","last_undegraded":"2024-03-30T00:34:39.067066+0000","last_fullsized":"2024-03-30T00:34:39.067066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:50.454125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064339+0000","last_change":"2024-03-30T00:33:15.687689+0000","last_active":"2024-03-30T00:34:39.064339+0000","last_peered":"2024-03-30T00:34:39.064339+0000","last_clean":"2024-03-30T00:34:39.064339+0000","last_became_active":"2024-03-30T00:33:15.686009+0000","last_became_peered":"2024-03-30T00:33:15.686009+0000","last_unstale":"2024-03-30T00:34:39.064339+0000","last_undegraded":"2024-03-30T00:34:39.064339+0000","last_fullsized":"2024-03-30T00:34:39.064339+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:45:17.234749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062258+0000","last_change":"2024-03-30T00:33:15.686053+0000","last_active":"2024-03-30T00:34:39.062258+0000","last_peered":"2024-03-30T00:34:39.062258+0000","last_clean":"2024-03-30T00:34:39.062258+0000","last_became_active":"2024-03-30T00:33:15.685723+0000","last_became_peered":"2024-03-30T00:33:15.685723+0000","last_unstale":"2024-03-30T00:34:39.062258+0000","last_undegraded":"2024-03-30T00:34:39.062258+0000","last_fullsized":"2024-03-30T00:34:39.062258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:17:27.322834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.050110+0000","last_change":"2024-03-30T00:33:15.674783+0000","last_active":"2024-03-30T00:34:39.050110+0000","last_peered":"2024-03-30T00:34:39.050110+0000","last_clean":"2024-03-30T00:34:39.050110+0000","last_became_active":"2024-03-30T00:33:15.674152+0000","last_became_peered":"2024-03-30T00:33:15.674152+0000","last_unstale":"2024-03-30T00:34:39.050110+0000","last_undegraded":"2024-03-30T00:34:39.050110+0000","last_fullsized":"2024-03-30T00:34:39.050110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:39:39.517240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'9","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.251874+0000","last_change":"2024-03-30T00:33:15.172217+0000","last_active":"2024-03-30T00:34:09.251874+0000","last_peered":"2024-03-30T00:34:09.251874+0000","last_clean":"2024-03-30T00:34:09.251874+0000","last_became_active":"2024-03-30T00:33:15.171718+0000","last_became_peered":"2024-03-30T00:33:15.171718+0000","last_unstale":"2024-03-30T00:34:09.251874+0000","last_undegraded":"2024-03-30T00:34:09.251874+0000","last_fullsized":"2024-03-30T00:34:09.251874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:11.691572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258046+0000","last_change":"2024-03-30T00:33:15.669937+0000","last_active":"2024-03-30T00:34:09.258046+0000","last_peered":"2024-03-30T00:34:09.258046+0000","last_clean":"2024-03-30T00:34:09.258046+0000","last_became_active":"2024-03-30T00:33:15.668704+0000","last_became_peered":"2024-03-30T00:33:15.668704+0000","last_unstale":"2024-03-30T00:34:09.258046+0000","last_undegraded":"2024-03-30T00:34:09.258046+0000","last_fullsized":"2024-03-30T00:34:09.258046+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:23.914755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258142+0000","last_change":"2024-03-30T00:33:15.173663+0000","last_active":"2024-03-30T00:34:09.258142+0000","last_peered":"2024-03-30T00:34:09.258142+0000","last_clean":"2024-03-30T00:34:09.258142+0000","last_became_active":"2024-03-30T00:33:15.173288+0000","last_became_peered":"2024-03-30T00:33:15.173288+0000","last_unstale":"2024-03-30T00:34:09.258142+0000","last_undegraded":"2024-03-30T00:34:09.258142+0000","last_fullsized":"2024-03-30T00:34:09.258142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:29:14.435142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260485+0000","last_change":"2024-03-30T00:33:15.686757+0000","last_active":"2024-03-30T00:34:09.260485+0000","last_peered":"2024-03-30T00:34:09.260485+0000","last_clean":"2024-03-30T00:34:09.260485+0000","last_became_active":"2024-03-30T00:33:15.686306+0000","last_became_peered":"2024-03-30T00:33:15.686306+0000","last_unstale":"2024-03-30T00:34:09.260485+0000","last_undegraded":"2024-03-30T00:34:09.260485+0000","last_fullsized":"2024-03-30T00:34:09.260485+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:50:21.048212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254756+0000","last_change":"2024-03-30T00:33:15.173402+0000","last_active":"2024-03-30T00:34:09.254756+0000","last_peered":"2024-03-30T00:34:09.254756+0000","last_clean":"2024-03-30T00:34:09.254756+0000","last_became_active":"2024-03-30T00:33:15.172995+0000","last_became_peered":"2024-03-30T00:33:15.172995+0000","last_unstale":"2024-03-30T00:34:09.254756+0000","last_undegraded":"2024-03-30T00:34:09.254756+0000","last_fullsized":"2024-03-30T00:34:09.254756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:03:25.694792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065176+0000","last_change":"2024-03-30T00:33:15.688292+0000","last_active":"2024-03-30T00:34:39.065176+0000","last_peered":"2024-03-30T00:34:39.065176+0000","last_clean":"2024-03-30T00:34:39.065176+0000","last_became_active":"2024-03-30T00:33:15.682722+0000","last_became_peered":"2024-03-30T00:33:15.682722+0000","last_unstale":"2024-03-30T00:34:39.065176+0000","last_undegraded":"2024-03-30T00:34:39.065176+0000","last_fullsized":"2024-03-30T00:34:39.065176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:52:10.074656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254648+0000","last_change":"2024-03-30T00:33:15.162316+0000","last_active":"2024-03-30T00:34:09.254648+0000","last_peered":"2024-03-30T00:34:09.254648+0000","last_clean":"2024-03-30T00:34:09.254648+0000","last_became_active":"2024-03-30T00:33:15.161805+0000","last_became_peered":"2024-03-30T00:33:15.161805+0000","last_unstale":"2024-03-30T00:34:09.254648+0000","last_undegraded":"2024-03-30T00:34:09.254648+0000","last_fullsized":"2024-03-30T00:34:09.254648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:26:16.470245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062423+0000","last_change":"2024-03-30T00:33:15.675442+0000","last_active":"2024-03-30T00:34:39.062423+0000","last_peered":"2024-03-30T00:34:39.062423+0000","last_clean":"2024-03-30T00:34:39.062423+0000","last_became_active":"2024-03-30T00:33:15.675068+0000","last_became_peered":"2024-03-30T00:33:15.675068+0000","last_unstale":"2024-03-30T00:34:39.062423+0000","last_undegraded":"2024-03-30T00:34:39.062423+0000","last_fullsized":"2024-03-30T00:34:39.062423+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:36:31.353262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":105,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:40.249081+0000","last_change":"2024-03-30T00:33:15.172087+0000","last_active":"2024-03-30T00:34:40.249081+0000","last_peered":"2024-03-30T00:34:40.249081+0000","last_clean":"2024-03-30T00:34:40.249081+0000","last_became_active":"2024-03-30T00:33:15.171768+0000","last_became_peered":"2024-03-30T00:33:15.171768+0000","last_unstale":"2024-03-30T00:34:40.249081+0000","last_undegraded":"2024-03-30T00:34:40.249081+0000","last_fullsized":"2024-03-30T00:34:40.249081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:49:17.198995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1070,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.042702+0000","last_change":"2024-03-30T00:33:15.665219+0000","last_active":"2024-03-30T00:34:39.042702+0000","last_peered":"2024-03-30T00:34:39.042702+0000","last_clean":"2024-03-30T00:34:39.042702+0000","last_became_active":"2024-03-30T00:33:15.664935+0000","last_became_peered":"2024-03-30T00:33:15.664935+0000","last_unstale":"2024-03-30T00:34:39.042702+0000","last_undegraded":"2024-03-30T00:34:39.042702+0000","last_fullsized":"2024-03-30T00:34:39.042702+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:12:37.123937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254914+0000","last_change":"2024-03-30T00:33:15.165668+0000","last_active":"2024-03-30T00:34:09.254914+0000","last_peered":"2024-03-30T00:34:09.254914+0000","last_clean":"2024-03-30T00:34:09.254914+0000","last_became_active":"2024-03-30T00:33:15.164637+0000","last_became_peered":"2024-03-30T00:33:15.164637+0000","last_unstale":"2024-03-30T00:34:09.254914+0000","last_undegraded":"2024-03-30T00:34:09.254914+0000","last_fullsized":"2024-03-30T00:34:09.254914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:47:59.389414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.048514+0000","last_change":"2024-03-30T00:34:10.309779+0000","last_active":"2024-03-30T00:34:39.048514+0000","last_peered":"2024-03-30T00:34:39.048514+0000","last_clean":"2024-03-30T00:34:39.048514+0000","last_became_active":"2024-03-30T00:34:10.309272+0000","last_became_peered":"2024-03-30T00:34:10.309272+0000","last_unstale":"2024-03-30T00:34:39.048514+0000","last_undegraded":"2024-03-30T00:34:39.048514+0000","last_fullsized":"2024-03-30T00:34:39.048514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:22:54.666608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264857+0000","last_change":"2024-03-30T00:33:15.177269+0000","last_active":"2024-03-30T00:34:09.264857+0000","last_peered":"2024-03-30T00:34:09.264857+0000","last_clean":"2024-03-30T00:34:09.264857+0000","last_became_active":"2024-03-30T00:33:15.176937+0000","last_became_peered":"2024-03-30T00:33:15.176937+0000","last_unstale":"2024-03-30T00:34:09.264857+0000","last_undegraded":"2024-03-30T00:34:09.264857+0000","last_fullsized":"2024-03-30T00:34:09.264857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:06:44.658775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257873+0000","last_change":"2024-03-30T00:33:15.675832+0000","last_active":"2024-03-30T00:34:09.257873+0000","last_peered":"2024-03-30T00:34:09.257873+0000","last_clean":"2024-03-30T00:34:09.257873+0000","last_became_active":"2024-03-30T00:33:15.675481+0000","last_became_peered":"2024-03-30T00:33:15.675481+0000","last_unstale":"2024-03-30T00:34:09.257873+0000","last_undegraded":"2024-03-30T00:34:09.257873+0000","last_fullsized":"2024-03-30T00:34:09.257873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:04:11.638056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258682+0000","last_change":"2024-03-30T00:33:15.171238+0000","last_active":"2024-03-30T00:34:09.258682+0000","last_peered":"2024-03-30T00:34:09.258682+0000","last_clean":"2024-03-30T00:34:09.258682+0000","last_became_active":"2024-03-30T00:33:15.170931+0000","last_became_peered":"2024-03-30T00:33:15.170931+0000","last_unstale":"2024-03-30T00:34:09.258682+0000","last_undegraded":"2024-03-30T00:34:09.258682+0000","last_fullsized":"2024-03-30T00:34:09.258682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:43:50.178363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.060489+0000","last_change":"2024-03-30T00:33:15.676014+0000","last_active":"2024-03-30T00:34:39.060489+0000","last_peered":"2024-03-30T00:34:39.060489+0000","last_clean":"2024-03-30T00:34:39.060489+0000","last_became_active":"2024-03-30T00:33:15.675553+0000","last_became_peered":"2024-03-30T00:33:15.675553+0000","last_unstale":"2024-03-30T00:34:39.060489+0000","last_undegraded":"2024-03-30T00:34:39.060489+0000","last_fullsized":"2024-03-30T00:34:39.060489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:10:17.248242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":99,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.246658+0000","last_change":"2024-03-30T00:33:15.177033+0000","last_active":"2024-03-30T00:34:39.246658+0000","last_peered":"2024-03-30T00:34:39.246658+0000","last_clean":"2024-03-30T00:34:39.246658+0000","last_became_active":"2024-03-30T00:33:15.176602+0000","last_became_peered":"2024-03-30T00:33:15.176602+0000","last_unstale":"2024-03-30T00:34:39.246658+0000","last_undegraded":"2024-03-30T00:34:39.246658+0000","last_fullsized":"2024-03-30T00:34:39.246658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:14:10.804704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065450+0000","last_change":"2024-03-30T00:33:15.683388+0000","last_active":"2024-03-30T00:34:39.065450+0000","last_peered":"2024-03-30T00:34:39.065450+0000","last_clean":"2024-03-30T00:34:39.065450+0000","last_became_active":"2024-03-30T00:33:15.682976+0000","last_became_peered":"2024-03-30T00:33:15.682976+0000","last_unstale":"2024-03-30T00:34:39.065450+0000","last_undegraded":"2024-03-30T00:34:39.065450+0000","last_fullsized":"2024-03-30T00:34:39.065450+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:50:34.907265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.256323+0000","last_change":"2024-03-30T00:33:15.167819+0000","last_active":"2024-03-30T00:34:09.256323+0000","last_peered":"2024-03-30T00:34:09.256323+0000","last_clean":"2024-03-30T00:34:09.256323+0000","last_became_active":"2024-03-30T00:33:15.167405+0000","last_became_peered":"2024-03-30T00:33:15.167405+0000","last_unstale":"2024-03-30T00:34:09.256323+0000","last_undegraded":"2024-03-30T00:34:09.256323+0000","last_fullsized":"2024-03-30T00:34:09.256323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:25:37.793792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260394+0000","last_change":"2024-03-30T00:33:15.690907+0000","last_active":"2024-03-30T00:34:09.260394+0000","last_peered":"2024-03-30T00:34:09.260394+0000","last_clean":"2024-03-30T00:34:09.260394+0000","last_became_active":"2024-03-30T00:33:15.690046+0000","last_became_peered":"2024-03-30T00:33:15.690046+0000","last_unstale":"2024-03-30T00:34:09.260394+0000","last_undegraded":"2024-03-30T00:34:09.260394+0000","last_fullsized":"2024-03-30T00:34:09.260394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:24.319685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258463+0000","last_change":"2024-03-30T00:33:15.166633+0000","last_active":"2024-03-30T00:34:09.258463+0000","last_peered":"2024-03-30T00:34:09.258463+0000","last_clean":"2024-03-30T00:34:09.258463+0000","last_became_active":"2024-03-30T00:33:15.166308+0000","last_became_peered":"2024-03-30T00:33:15.166308+0000","last_unstale":"2024-03-30T00:34:09.258463+0000","last_undegraded":"2024-03-30T00:34:09.258463+0000","last_fullsized":"2024-03-30T00:34:09.258463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:21:53.102872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065783+0000","last_change":"2024-03-30T00:33:15.691880+0000","last_active":"2024-03-30T00:34:39.065783+0000","last_peered":"2024-03-30T00:34:39.065783+0000","last_clean":"2024-03-30T00:34:39.065783+0000","last_became_active":"2024-03-30T00:33:15.690780+0000","last_became_peered":"2024-03-30T00:33:15.690780+0000","last_unstale":"2024-03-30T00:34:39.065783+0000","last_undegraded":"2024-03-30T00:34:39.065783+0000","last_fullsized":"2024-03-30T00:34:39.065783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:06.532709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":226,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:40.260983+0000","last_change":"2024-03-30T00:33:15.164287+0000","last_active":"2024-03-30T00:34:40.260983+0000","last_peered":"2024-03-30T00:34:40.260983+0000","last_clean":"2024-03-30T00:34:40.260983+0000","last_became_active":"2024-03-30T00:33:15.163557+0000","last_became_peered":"2024-03-30T00:33:15.163557+0000","last_unstale":"2024-03-30T00:34:40.260983+0000","last_undegraded":"2024-03-30T00:34:40.260983+0000","last_fullsized":"2024-03-30T00:34:40.260983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:41.117936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":174,"num_read_kb":154,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062751+0000","last_change":"2024-03-30T00:33:15.682841+0000","last_active":"2024-03-30T00:34:39.062751+0000","last_peered":"2024-03-30T00:34:39.062751+0000","last_clean":"2024-03-30T00:34:39.062751+0000","last_became_active":"2024-03-30T00:33:15.682237+0000","last_became_peered":"2024-03-30T00:33:15.682237+0000","last_unstale":"2024-03-30T00:34:39.062751+0000","last_undegraded":"2024-03-30T00:34:39.062751+0000","last_fullsized":"2024-03-30T00:34:39.062751+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:59:05.719216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'6","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.262163+0000","last_change":"2024-03-30T00:33:15.165193+0000","last_active":"2024-03-30T00:34:09.262163+0000","last_peered":"2024-03-30T00:34:09.262163+0000","last_clean":"2024-03-30T00:34:09.262163+0000","last_became_active":"2024-03-30T00:33:15.161741+0000","last_became_peered":"2024-03-30T00:33:15.161741+0000","last_unstale":"2024-03-30T00:34:09.262163+0000","last_undegraded":"2024-03-30T00:34:09.262163+0000","last_fullsized":"2024-03-30T00:34:09.262163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:22:11.891064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264117+0000","last_change":"2024-03-30T00:33:15.680929+0000","last_active":"2024-03-30T00:34:09.264117+0000","last_peered":"2024-03-30T00:34:09.264117+0000","last_clean":"2024-03-30T00:34:09.264117+0000","last_became_active":"2024-03-30T00:33:15.676671+0000","last_became_peered":"2024-03-30T00:33:15.676671+0000","last_unstale":"2024-03-30T00:34:09.264117+0000","last_undegraded":"2024-03-30T00:34:09.264117+0000","last_fullsized":"2024-03-30T00:34:09.264117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:28:33.819135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257972+0000","last_change":"2024-03-30T00:33:15.168029+0000","last_active":"2024-03-30T00:34:09.257972+0000","last_peered":"2024-03-30T00:34:09.257972+0000","last_clean":"2024-03-30T00:34:09.257972+0000","last_became_active":"2024-03-30T00:33:15.167636+0000","last_became_peered":"2024-03-30T00:33:15.167636+0000","last_unstale":"2024-03-30T00:34:09.257972+0000","last_undegraded":"2024-03-30T00:34:09.257972+0000","last_fullsized":"2024-03-30T00:34:09.257972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:25:47.834696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.058578+0000","last_change":"2024-03-30T00:33:15.687977+0000","last_active":"2024-03-30T00:34:39.058578+0000","last_peered":"2024-03-30T00:34:39.058578+0000","last_clean":"2024-03-30T00:34:39.058578+0000","last_became_active":"2024-03-30T00:33:15.687491+0000","last_became_peered":"2024-03-30T00:33:15.687491+0000","last_unstale":"2024-03-30T00:34:39.058578+0000","last_undegraded":"2024-03-30T00:34:39.058578+0000","last_fullsized":"2024-03-30T00:34:39.058578+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:08:55.568039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":100,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:38.257303+0000","last_change":"2024-03-30T00:33:15.171826+0000","last_active":"2024-03-30T00:34:38.257303+0000","last_peered":"2024-03-30T00:34:38.257303+0000","last_clean":"2024-03-30T00:34:38.257303+0000","last_became_active":"2024-03-30T00:33:15.170749+0000","last_became_peered":"2024-03-30T00:33:15.170749+0000","last_unstale":"2024-03-30T00:34:38.257303+0000","last_undegraded":"2024-03-30T00:34:38.257303+0000","last_fullsized":"2024-03-30T00:34:38.257303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:57:52.034530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062584+0000","last_change":"2024-03-30T00:34:10.304456+0000","last_active":"2024-03-30T00:34:39.062584+0000","last_peered":"2024-03-30T00:34:39.062584+0000","last_clean":"2024-03-30T00:34:39.062584+0000","last_became_active":"2024-03-30T00:34:10.303934+0000","last_became_peered":"2024-03-30T00:34:10.303934+0000","last_unstale":"2024-03-30T00:34:39.062584+0000","last_undegraded":"2024-03-30T00:34:39.062584+0000","last_fullsized":"2024-03-30T00:34:39.062584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:48:07.792123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'15","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254268+0000","last_change":"2024-03-30T00:33:15.171016+0000","last_active":"2024-03-30T00:34:09.254268+0000","last_peered":"2024-03-30T00:34:09.254268+0000","last_clean":"2024-03-30T00:34:09.254268+0000","last_became_active":"2024-03-30T00:33:15.170058+0000","last_became_peered":"2024-03-30T00:33:15.170058+0000","last_unstale":"2024-03-30T00:34:09.254268+0000","last_undegraded":"2024-03-30T00:34:09.254268+0000","last_fullsized":"2024-03-30T00:34:09.254268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:07:39.910286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051568+0000","last_change":"2024-03-30T00:34:10.305753+0000","last_active":"2024-03-30T00:34:39.051568+0000","last_peered":"2024-03-30T00:34:39.051568+0000","last_clean":"2024-03-30T00:34:39.051568+0000","last_became_active":"2024-03-30T00:34:10.305339+0000","last_became_peered":"2024-03-30T00:34:10.305339+0000","last_unstale":"2024-03-30T00:34:39.051568+0000","last_undegraded":"2024-03-30T00:34:39.051568+0000","last_fullsized":"2024-03-30T00:34:39.051568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:23:41.611151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254799+0000","last_change":"2024-03-30T00:33:15.163633+0000","last_active":"2024-03-30T00:34:09.254799+0000","last_peered":"2024-03-30T00:34:09.254799+0000","last_clean":"2024-03-30T00:34:09.254799+0000","last_became_active":"2024-03-30T00:33:15.161339+0000","last_became_peered":"2024-03-30T00:33:15.161339+0000","last_unstale":"2024-03-30T00:34:09.254799+0000","last_undegraded":"2024-03-30T00:34:09.254799+0000","last_fullsized":"2024-03-30T00:34:09.254799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:15:25.194650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.050235+0000","last_change":"2024-03-30T00:34:10.269652+0000","last_active":"2024-03-30T00:34:39.050235+0000","last_peered":"2024-03-30T00:34:39.050235+0000","last_clean":"2024-03-30T00:34:39.050235+0000","last_became_active":"2024-03-30T00:34:10.269267+0000","last_became_peered":"2024-03-30T00:34:10.269267+0000","last_unstale":"2024-03-30T00:34:39.050235+0000","last_undegraded":"2024-03-30T00:34:39.050235+0000","last_fullsized":"2024-03-30T00:34:39.050235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:29:13.724322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.251412+0000","last_change":"2024-03-30T00:33:15.177273+0000","last_active":"2024-03-30T00:34:09.251412+0000","last_peered":"2024-03-30T00:34:09.251412+0000","last_clean":"2024-03-30T00:34:09.251412+0000","last_became_active":"2024-03-30T00:33:15.175333+0000","last_became_peered":"2024-03-30T00:33:15.175333+0000","last_unstale":"2024-03-30T00:34:09.251412+0000","last_undegraded":"2024-03-30T00:34:09.251412+0000","last_fullsized":"2024-03-30T00:34:09.251412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:27:03.305671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.046660+0000","last_change":"2024-03-30T00:33:15.674746+0000","last_active":"2024-03-30T00:34:39.046660+0000","last_peered":"2024-03-30T00:34:39.046660+0000","last_clean":"2024-03-30T00:34:39.046660+0000","last_became_active":"2024-03-30T00:33:15.674388+0000","last_became_peered":"2024-03-30T00:33:15.674388+0000","last_unstale":"2024-03-30T00:34:39.046660+0000","last_undegraded":"2024-03-30T00:34:39.046660+0000","last_fullsized":"2024-03-30T00:34:39.046660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:21:15.024181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.256443+0000","last_change":"2024-03-30T00:33:15.173640+0000","last_active":"2024-03-30T00:34:09.256443+0000","last_peered":"2024-03-30T00:34:09.256443+0000","last_clean":"2024-03-30T00:34:09.256443+0000","last_became_active":"2024-03-30T00:33:15.173104+0000","last_became_peered":"2024-03-30T00:33:15.173104+0000","last_unstale":"2024-03-30T00:34:09.256443+0000","last_undegraded":"2024-03-30T00:34:09.256443+0000","last_fullsized":"2024-03-30T00:34:09.256443+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:20:14.880730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.056823+0000","last_change":"2024-03-30T00:33:15.677824+0000","last_active":"2024-03-30T00:34:39.056823+0000","last_peered":"2024-03-30T00:34:39.056823+0000","last_clean":"2024-03-30T00:34:39.056823+0000","last_became_active":"2024-03-30T00:33:15.677435+0000","last_became_peered":"2024-03-30T00:33:15.677435+0000","last_unstale":"2024-03-30T00:34:39.056823+0000","last_undegraded":"2024-03-30T00:34:39.056823+0000","last_fullsized":"2024-03-30T00:34:39.056823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:22.992989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.044985+0000","last_change":"2024-03-30T00:34:10.304054+0000","last_active":"2024-03-30T00:34:39.044985+0000","last_peered":"2024-03-30T00:34:39.044985+0000","last_clean":"2024-03-30T00:34:39.044985+0000","last_became_active":"2024-03-30T00:34:10.303653+0000","last_became_peered":"2024-03-30T00:34:10.303653+0000","last_unstale":"2024-03-30T00:34:39.044985+0000","last_undegraded":"2024-03-30T00:34:39.044985+0000","last_fullsized":"2024-03-30T00:34:39.044985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:12:32.378165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":96,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:36.251806+0000","last_change":"2024-03-30T00:33:15.173264+0000","last_active":"2024-03-30T00:34:36.251806+0000","last_peered":"2024-03-30T00:34:36.251806+0000","last_clean":"2024-03-30T00:34:36.251806+0000","last_became_active":"2024-03-30T00:33:15.172932+0000","last_became_peered":"2024-03-30T00:33:15.172932+0000","last_unstale":"2024-03-30T00:34:36.251806+0000","last_undegraded":"2024-03-30T00:34:36.251806+0000","last_fullsized":"2024-03-30T00:34:36.251806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:18:25.079284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257668+0000","last_change":"2024-03-30T00:33:15.686606+0000","last_active":"2024-03-30T00:34:09.257668+0000","last_peered":"2024-03-30T00:34:09.257668+0000","last_clean":"2024-03-30T00:34:09.257668+0000","last_became_active":"2024-03-30T00:33:15.686076+0000","last_became_peered":"2024-03-30T00:33:15.686076+0000","last_unstale":"2024-03-30T00:34:09.257668+0000","last_undegraded":"2024-03-30T00:34:09.257668+0000","last_fullsized":"2024-03-30T00:34:09.257668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:46.962659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.261642+0000","last_change":"2024-03-30T00:33:15.177263+0000","last_active":"2024-03-30T00:34:09.261642+0000","last_peered":"2024-03-30T00:34:09.261642+0000","last_clean":"2024-03-30T00:34:09.261642+0000","last_became_active":"2024-03-30T00:33:15.176743+0000","last_became_peered":"2024-03-30T00:33:15.176743+0000","last_unstale":"2024-03-30T00:34:09.261642+0000","last_undegraded":"2024-03-30T00:34:09.261642+0000","last_fullsized":"2024-03-30T00:34:09.261642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:56:47.491371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":324,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.256985+0000","last_change":"2024-03-30T00:32:12.678730+0000","last_active":"2024-03-30T00:34:09.256985+0000","last_peered":"2024-03-30T00:34:09.256985+0000","last_clean":"2024-03-30T00:34:09.256985+0000","last_became_active":"2024-03-30T00:32:12.678256+0000","last_became_peered":"2024-03-30T00:32:12.678256+0000","last_unstale":"2024-03-30T00:34:09.256985+0000","last_undegraded":"2024-03-30T00:34:09.256985+0000","last_fullsized":"2024-03-30T00:34:09.256985+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:29:17.768535+0000","last_clean_scrub_stamp":"2024-03-30T00:29:17.768535+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:14:16.229072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.304196+0000","last_change":"2024-03-30T00:34:10.304196+0000","last_active":"2024-03-30T00:34:10.304196+0000","last_peered":"2024-03-30T00:34:10.304196+0000","last_clean":"2024-03-30T00:34:10.304196+0000","last_became_active":"2024-03-30T00:34:10.303740+0000","last_became_peered":"2024-03-30T00:34:10.303740+0000","last_unstale":"2024-03-30T00:34:10.304196+0000","last_undegraded":"2024-03-30T00:34:10.304196+0000","last_fullsized":"2024-03-30T00:34:10.304196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:25:10.417989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257070+0000","last_change":"2024-03-30T00:33:15.167490+0000","last_active":"2024-03-30T00:34:09.257070+0000","last_peered":"2024-03-30T00:34:09.257070+0000","last_clean":"2024-03-30T00:34:09.257070+0000","last_became_active":"2024-03-30T00:33:15.166888+0000","last_became_peered":"2024-03-30T00:33:15.166888+0000","last_unstale":"2024-03-30T00:34:09.257070+0000","last_undegraded":"2024-03-30T00:34:09.257070+0000","last_fullsized":"2024-03-30T00:34:09.257070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:00:32.571281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254643+0000","last_change":"2024-03-30T00:33:15.178388+0000","last_active":"2024-03-30T00:34:09.254643+0000","last_peered":"2024-03-30T00:34:09.254643+0000","last_clean":"2024-03-30T00:34:09.254643+0000","last_became_active":"2024-03-30T00:33:15.178160+0000","last_became_peered":"2024-03-30T00:33:15.178160+0000","last_unstale":"2024-03-30T00:34:09.254643+0000","last_undegraded":"2024-03-30T00:34:09.254643+0000","last_fullsized":"2024-03-30T00:34:09.254643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:28:51.567598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264257+0000","last_change":"2024-03-30T00:33:15.675426+0000","last_active":"2024-03-30T00:34:09.264257+0000","last_peered":"2024-03-30T00:34:09.264257+0000","last_clean":"2024-03-30T00:34:09.264257+0000","last_became_active":"2024-03-30T00:33:15.675083+0000","last_became_peered":"2024-03-30T00:33:15.675083+0000","last_unstale":"2024-03-30T00:34:09.264257+0000","last_undegraded":"2024-03-30T00:34:09.264257+0000","last_fullsized":"2024-03-30T00:34:09.264257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:37:03.086331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'6","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.251872+0000","last_change":"2024-03-30T00:33:15.171537+0000","last_active":"2024-03-30T00:34:09.251872+0000","last_peered":"2024-03-30T00:34:09.251872+0000","last_clean":"2024-03-30T00:34:09.251872+0000","last_became_active":"2024-03-30T00:33:15.171128+0000","last_became_peered":"2024-03-30T00:33:15.171128+0000","last_unstale":"2024-03-30T00:34:09.251872+0000","last_undegraded":"2024-03-30T00:34:09.251872+0000","last_fullsized":"2024-03-30T00:34:09.251872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:16:52.062144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257830+0000","last_change":"2024-03-30T00:33:15.674075+0000","last_active":"2024-03-30T00:34:09.257830+0000","last_peered":"2024-03-30T00:34:09.257830+0000","last_clean":"2024-03-30T00:34:09.257830+0000","last_became_active":"2024-03-30T00:33:15.672400+0000","last_became_peered":"2024-03-30T00:33:15.672400+0000","last_unstale":"2024-03-30T00:34:09.257830+0000","last_undegraded":"2024-03-30T00:34:09.257830+0000","last_fullsized":"2024-03-30T00:34:09.257830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:59.309835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.261775+0000","last_change":"2024-03-30T00:33:15.168245+0000","last_active":"2024-03-30T00:34:09.261775+0000","last_peered":"2024-03-30T00:34:09.261775+0000","last_clean":"2024-03-30T00:34:09.261775+0000","last_became_active":"2024-03-30T00:33:15.167852+0000","last_became_peered":"2024-03-30T00:33:15.167852+0000","last_unstale":"2024-03-30T00:34:09.261775+0000","last_undegraded":"2024-03-30T00:34:09.261775+0000","last_fullsized":"2024-03-30T00:34:09.261775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:33:28.780614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.305258+0000","last_change":"2024-03-30T00:34:10.305258+0000","last_active":"2024-03-30T00:34:10.305258+0000","last_peered":"2024-03-30T00:34:10.305258+0000","last_clean":"2024-03-30T00:34:10.305258+0000","last_became_active":"2024-03-30T00:34:10.304913+0000","last_became_peered":"2024-03-30T00:34:10.304913+0000","last_unstale":"2024-03-30T00:34:10.305258+0000","last_undegraded":"2024-03-30T00:34:10.305258+0000","last_fullsized":"2024-03-30T00:34:10.305258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:29:08.202307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.263955+0000","last_change":"2024-03-30T00:33:15.170052+0000","last_active":"2024-03-30T00:34:09.263955+0000","last_peered":"2024-03-30T00:34:09.263955+0000","last_clean":"2024-03-30T00:34:09.263955+0000","last_became_active":"2024-03-30T00:33:15.169706+0000","last_became_peered":"2024-03-30T00:33:15.169706+0000","last_unstale":"2024-03-30T00:34:09.263955+0000","last_undegraded":"2024-03-30T00:34:09.263955+0000","last_fullsized":"2024-03-30T00:34:09.263955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:36:32.043071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.014521+0000","last_change":"2024-03-30T00:33:15.667061+0000","last_active":"2024-03-30T00:34:39.014521+0000","last_peered":"2024-03-30T00:34:39.014521+0000","last_clean":"2024-03-30T00:34:39.014521+0000","last_became_active":"2024-03-30T00:33:15.666763+0000","last_became_peered":"2024-03-30T00:33:15.666763+0000","last_unstale":"2024-03-30T00:34:39.014521+0000","last_undegraded":"2024-03-30T00:34:39.014521+0000","last_fullsized":"2024-03-30T00:34:39.014521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:46.558297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257420+0000","last_change":"2024-03-30T00:33:15.176152+0000","last_active":"2024-03-30T00:34:09.257420+0000","last_peered":"2024-03-30T00:34:09.257420+0000","last_clean":"2024-03-30T00:34:09.257420+0000","last_became_active":"2024-03-30T00:33:15.175640+0000","last_became_peered":"2024-03-30T00:33:15.175640+0000","last_unstale":"2024-03-30T00:34:09.257420+0000","last_undegraded":"2024-03-30T00:34:09.257420+0000","last_fullsized":"2024-03-30T00:34:09.257420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:58:11.575577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051405+0000","last_change":"2024-03-30T00:33:15.672957+0000","last_active":"2024-03-30T00:34:39.051405+0000","last_peered":"2024-03-30T00:34:39.051405+0000","last_clean":"2024-03-30T00:34:39.051405+0000","last_became_active":"2024-03-30T00:33:15.668868+0000","last_became_peered":"2024-03-30T00:33:15.668868+0000","last_unstale":"2024-03-30T00:34:39.051405+0000","last_undegraded":"2024-03-30T00:34:39.051405+0000","last_fullsized":"2024-03-30T00:34:39.051405+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:50.342693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:10.301965+0000","last_change":"2024-03-30T00:33:15.160345+0000","last_active":"2024-03-30T00:34:10.301965+0000","last_peered":"2024-03-30T00:34:10.301965+0000","last_clean":"2024-03-30T00:34:10.301965+0000","last_became_active":"2024-03-30T00:33:15.160026+0000","last_became_peered":"2024-03-30T00:33:15.160026+0000","last_unstale":"2024-03-30T00:34:10.301965+0000","last_undegraded":"2024-03-30T00:34:10.301965+0000","last_fullsized":"2024-03-30T00:34:10.301965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:06:51.177680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257738+0000","last_change":"2024-03-30T00:33:15.674058+0000","last_active":"2024-03-30T00:34:09.257738+0000","last_peered":"2024-03-30T00:34:09.257738+0000","last_clean":"2024-03-30T00:34:09.257738+0000","last_became_active":"2024-03-30T00:33:15.673165+0000","last_became_peered":"2024-03-30T00:33:15.673165+0000","last_unstale":"2024-03-30T00:34:09.257738+0000","last_undegraded":"2024-03-30T00:34:09.257738+0000","last_fullsized":"2024-03-30T00:34:09.257738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:00:14.869963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.254400+0000","last_change":"2024-03-30T00:33:15.173178+0000","last_active":"2024-03-30T00:34:09.254400+0000","last_peered":"2024-03-30T00:34:09.254400+0000","last_clean":"2024-03-30T00:34:09.254400+0000","last_became_active":"2024-03-30T00:33:15.172830+0000","last_became_peered":"2024-03-30T00:33:15.172830+0000","last_unstale":"2024-03-30T00:34:09.254400+0000","last_undegraded":"2024-03-30T00:34:09.254400+0000","last_fullsized":"2024-03-30T00:34:09.254400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T00:49:09.823855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.065387+0000","last_change":"2024-03-30T00:33:15.693485+0000","last_active":"2024-03-30T00:34:39.065387+0000","last_peered":"2024-03-30T00:34:39.065387+0000","last_clean":"2024-03-30T00:34:39.065387+0000","last_became_active":"2024-03-30T00:33:15.693066+0000","last_became_peered":"2024-03-30T00:33:15.693066+0000","last_unstale":"2024-03-30T00:34:39.065387+0000","last_undegraded":"2024-03-30T00:34:39.065387+0000","last_fullsized":"2024-03-30T00:34:39.065387+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:31:38.161331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.262703+0000","last_change":"2024-03-30T00:33:15.170808+0000","last_active":"2024-03-30T00:34:09.262703+0000","last_peered":"2024-03-30T00:34:09.262703+0000","last_clean":"2024-03-30T00:34:09.262703+0000","last_became_active":"2024-03-30T00:33:15.170448+0000","last_became_peered":"2024-03-30T00:33:15.170448+0000","last_unstale":"2024-03-30T00:34:09.262703+0000","last_undegraded":"2024-03-30T00:34:09.262703+0000","last_fullsized":"2024-03-30T00:34:09.262703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:08:28.179041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062015+0000","last_change":"2024-03-30T00:33:15.687619+0000","last_active":"2024-03-30T00:34:39.062015+0000","last_peered":"2024-03-30T00:34:39.062015+0000","last_clean":"2024-03-30T00:34:39.062015+0000","last_became_active":"2024-03-30T00:33:15.687313+0000","last_became_peered":"2024-03-30T00:33:15.687313+0000","last_unstale":"2024-03-30T00:34:39.062015+0000","last_undegraded":"2024-03-30T00:34:39.062015+0000","last_fullsized":"2024-03-30T00:34:39.062015+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:36:51.182558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.258190+0000","last_change":"2024-03-30T00:33:15.168601+0000","last_active":"2024-03-30T00:34:09.258190+0000","last_peered":"2024-03-30T00:34:09.258190+0000","last_clean":"2024-03-30T00:34:09.258190+0000","last_became_active":"2024-03-30T00:33:15.168065+0000","last_became_peered":"2024-03-30T00:33:15.168065+0000","last_unstale":"2024-03-30T00:34:09.258190+0000","last_undegraded":"2024-03-30T00:34:09.258190+0000","last_fullsized":"2024-03-30T00:34:09.258190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:22:36.477280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.063142+0000","last_change":"2024-03-30T00:33:15.678299+0000","last_active":"2024-03-30T00:34:39.063142+0000","last_peered":"2024-03-30T00:34:39.063142+0000","last_clean":"2024-03-30T00:34:39.063142+0000","last_became_active":"2024-03-30T00:33:15.675499+0000","last_became_peered":"2024-03-30T00:33:15.675499+0000","last_unstale":"2024-03-30T00:34:39.063142+0000","last_undegraded":"2024-03-30T00:34:39.063142+0000","last_fullsized":"2024-03-30T00:34:39.063142+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:01:15.317125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264445+0000","last_change":"2024-03-30T00:33:15.172058+0000","last_active":"2024-03-30T00:34:09.264445+0000","last_peered":"2024-03-30T00:34:09.264445+0000","last_clean":"2024-03-30T00:34:09.264445+0000","last_became_active":"2024-03-30T00:33:15.167857+0000","last_became_peered":"2024-03-30T00:33:15.167857+0000","last_unstale":"2024-03-30T00:34:09.264445+0000","last_undegraded":"2024-03-30T00:34:09.264445+0000","last_fullsized":"2024-03-30T00:34:09.264445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:14.135315+0000","last_clean_scrub_stamp":"2024-03-30T00:33:14.135315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:03:35.951559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.062087+0000","last_change":"2024-03-30T00:33:15.682519+0000","last_active":"2024-03-30T00:34:39.062087+0000","last_peered":"2024-03-30T00:34:39.062087+0000","last_clean":"2024-03-30T00:34:39.062087+0000","last_became_active":"2024-03-30T00:33:15.681055+0000","last_became_peered":"2024-03-30T00:33:15.681055+0000","last_unstale":"2024-03-30T00:34:39.062087+0000","last_undegraded":"2024-03-30T00:34:39.062087+0000","last_fullsized":"2024-03-30T00:34:39.062087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:54:38.160151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051654+0000","last_change":"2024-03-30T00:33:15.673411+0000","last_active":"2024-03-30T00:34:39.051654+0000","last_peered":"2024-03-30T00:34:39.051654+0000","last_clean":"2024-03-30T00:34:39.051654+0000","last_became_active":"2024-03-30T00:33:15.672852+0000","last_became_peered":"2024-03-30T00:33:15.672852+0000","last_unstale":"2024-03-30T00:34:39.051654+0000","last_undegraded":"2024-03-30T00:34:39.051654+0000","last_fullsized":"2024-03-30T00:34:39.051654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:55:46.705785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.055100+0000","last_change":"2024-03-30T00:33:15.680624+0000","last_active":"2024-03-30T00:34:39.055100+0000","last_peered":"2024-03-30T00:34:39.055100+0000","last_clean":"2024-03-30T00:34:39.055100+0000","last_became_active":"2024-03-30T00:33:15.679900+0000","last_became_peered":"2024-03-30T00:33:15.679900+0000","last_unstale":"2024-03-30T00:34:39.055100+0000","last_undegraded":"2024-03-30T00:34:39.055100+0000","last_fullsized":"2024-03-30T00:34:39.055100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:44.484608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.051062+0000","last_change":"2024-03-30T00:33:15.672612+0000","last_active":"2024-03-30T00:34:39.051062+0000","last_peered":"2024-03-30T00:34:39.051062+0000","last_clean":"2024-03-30T00:34:39.051062+0000","last_became_active":"2024-03-30T00:33:15.671536+0000","last_became_peered":"2024-03-30T00:33:15.671536+0000","last_unstale":"2024-03-30T00:34:39.051062+0000","last_undegraded":"2024-03-30T00:34:39.051062+0000","last_fullsized":"2024-03-30T00:34:39.051062+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:52:40.783988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.260508+0000","last_change":"2024-03-30T00:33:15.681577+0000","last_active":"2024-03-30T00:34:09.260508+0000","last_peered":"2024-03-30T00:34:09.260508+0000","last_clean":"2024-03-30T00:34:09.260508+0000","last_became_active":"2024-03-30T00:33:15.681313+0000","last_became_peered":"2024-03-30T00:33:15.681313+0000","last_unstale":"2024-03-30T00:34:09.260508+0000","last_undegraded":"2024-03-30T00:34:09.260508+0000","last_fullsized":"2024-03-30T00:34:09.260508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T01:42:15.178957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264354+0000","last_change":"2024-03-30T00:33:15.688172+0000","last_active":"2024-03-30T00:34:09.264354+0000","last_peered":"2024-03-30T00:34:09.264354+0000","last_clean":"2024-03-30T00:34:09.264354+0000","last_became_active":"2024-03-30T00:33:15.687860+0000","last_became_peered":"2024-03-30T00:33:15.687860+0000","last_unstale":"2024-03-30T00:34:09.264354+0000","last_undegraded":"2024-03-30T00:34:09.264354+0000","last_fullsized":"2024-03-30T00:34:09.264354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:08:28.762798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.047772+0000","last_change":"2024-03-30T00:33:15.673868+0000","last_active":"2024-03-30T00:34:39.047772+0000","last_peered":"2024-03-30T00:34:39.047772+0000","last_clean":"2024-03-30T00:34:39.047772+0000","last_became_active":"2024-03-30T00:33:15.673467+0000","last_became_peered":"2024-03-30T00:33:15.673467+0000","last_unstale":"2024-03-30T00:34:39.047772+0000","last_undegraded":"2024-03-30T00:34:39.047772+0000","last_fullsized":"2024-03-30T00:34:39.047772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:07:39.448967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.064293+0000","last_change":"2024-03-30T00:33:15.684639+0000","last_active":"2024-03-30T00:34:39.064293+0000","last_peered":"2024-03-30T00:34:39.064293+0000","last_clean":"2024-03-30T00:34:39.064293+0000","last_became_active":"2024-03-30T00:33:15.684357+0000","last_became_peered":"2024-03-30T00:33:15.684357+0000","last_unstale":"2024-03-30T00:34:39.064293+0000","last_undegraded":"2024-03-30T00:34:39.064293+0000","last_fullsized":"2024-03-30T00:34:39.064293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:02:50.995640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257545+0000","last_change":"2024-03-30T00:33:15.680810+0000","last_active":"2024-03-30T00:34:09.257545+0000","last_peered":"2024-03-30T00:34:09.257545+0000","last_clean":"2024-03-30T00:34:09.257545+0000","last_became_active":"2024-03-30T00:33:15.680457+0000","last_became_peered":"2024-03-30T00:33:15.680457+0000","last_unstale":"2024-03-30T00:34:09.257545+0000","last_undegraded":"2024-03-30T00:34:09.257545+0000","last_fullsized":"2024-03-30T00:34:09.257545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:55:02.575683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.061425+0000","last_change":"2024-03-30T00:34:10.305867+0000","last_active":"2024-03-30T00:34:39.061425+0000","last_peered":"2024-03-30T00:34:39.061425+0000","last_clean":"2024-03-30T00:34:39.061425+0000","last_became_active":"2024-03-30T00:34:10.304998+0000","last_became_peered":"2024-03-30T00:34:10.304998+0000","last_unstale":"2024-03-30T00:34:39.061425+0000","last_undegraded":"2024-03-30T00:34:39.061425+0000","last_fullsized":"2024-03-30T00:34:39.061425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:43:34.668457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.257506+0000","last_change":"2024-03-30T00:33:15.671491+0000","last_active":"2024-03-30T00:34:09.257506+0000","last_peered":"2024-03-30T00:34:09.257506+0000","last_clean":"2024-03-30T00:34:09.257506+0000","last_became_active":"2024-03-30T00:33:15.671110+0000","last_became_peered":"2024-03-30T00:33:15.671110+0000","last_unstale":"2024-03-30T00:34:09.257506+0000","last_undegraded":"2024-03-30T00:34:09.257506+0000","last_fullsized":"2024-03-30T00:34:09.257506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:57.623730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.067543+0000","last_change":"2024-03-30T00:33:15.686795+0000","last_active":"2024-03-30T00:34:39.067543+0000","last_peered":"2024-03-30T00:34:39.067543+0000","last_clean":"2024-03-30T00:34:39.067543+0000","last_became_active":"2024-03-30T00:33:15.685715+0000","last_became_peered":"2024-03-30T00:33:15.685715+0000","last_unstale":"2024-03-30T00:34:39.067543+0000","last_undegraded":"2024-03-30T00:34:39.067543+0000","last_fullsized":"2024-03-30T00:34:39.067543+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:37:53.838823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T00:34:09.264242+0000","last_change":"2024-03-30T00:33:15.687369+0000","last_active":"2024-03-30T00:34:09.264242+0000","last_peered":"2024-03-30T00:34:09.264242+0000","last_clean":"2024-03-30T00:34:09.264242+0000","last_became_active":"2024-03-30T00:33:15.686852+0000","last_became_peered":"2024-03-30T00:33:15.686852+0000","last_unstale":"2024-03-30T00:34:09.264242+0000","last_undegraded":"2024-03-30T00:34:09.264242+0000","last_fullsized":"2024-03-30T00:34:09.264242+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:09:21.003739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T00:34:39.058661+0000","last_change":"2024-03-30T00:33:15.676840+0000","last_active":"2024-03-30T00:34:39.058661+0000","last_peered":"2024-03-30T00:34:39.058661+0000","last_clean":"2024-03-30T00:34:39.058661+0000","last_became_active":"2024-03-30T00:33:15.675884+0000","last_became_peered":"2024-03-30T00:33:15.675884+0000","last_unstale":"2024-03-30T00:34:39.058661+0000","last_undegraded":"2024-03-30T00:34:39.058661+0000","last_fullsized":"2024-03-30T00:34:39.058661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T00:33:15.137580+0000","last_clean_scrub_stamp":"2024-03-30T00:33:15.137580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:50:02.900843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73836,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":513,"num_read_kb":332,"num_write":168,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":221508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36120,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709032,"statfs":{"total":95995035648,"available":95958048768,"internally_reserved":0,"allocated":933888,"data_stored":446772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.622,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.872,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.621,"15min":0.625},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":1.167,"15min":1.167},"last":0.503}]},{"osd":1,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77700000000000002,"15min":0.77400000000000002},"min":{"1min":0.60899999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1020000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.78200000000000003,"15min":0.77600000000000002},"min":{"1min":0.67900000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.0229999999999999,"5min":1.623,"15min":1.623},"last":0.90800000000000003}]},{"osd":2,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88600000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.44900000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.028,"5min":1.304,"15min":1.304},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":4,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83799999999999997,"15min":0.83599999999999997},"min":{"1min":0.64200000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.0069999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.623},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.052,"15min":1.0880000000000001},"min":{"1min":0.64400000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.98699999999999999,"5min":6.6029999999999998,"15min":6.6029999999999998},"last":0.80300000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.71399999999999997,"15min":0.72899999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.84999999999999998,"5min":1.486,"15min":1.486},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.099,"5min":1.536,"15min":1.536},"last":0.749}]},{"osd":6,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.81999999999999995,"15min":0.83399999999999996},"min":{"1min":0.504,"5min":0.501,"15min":0.501},"max":{"1min":1.052,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.84099999999999997,"15min":0.86199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93799999999999994,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.95399999999999996,"15min":0.96099999999999997},"min":{"1min":0.65100000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.145,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.995,"15min":1.0109999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.1359999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.93899999999999995}]},{"osd":8,"last update":"Sat Mar 30 00:34:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.96499999999999997}]},{"osd":9,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":1.1200000000000001,"15min":1.1679999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0089999999999999,"5min":6.6399999999999997,"15min":6.6399999999999997},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.95599999999999996,"15min":0.96199999999999997},"min":{"1min":0.71799999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3560000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.91700000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.998,"15min":1.0049999999999999},"min":{"1min":0.75900000000000001,"5min":0.754,"15min":0.754},"max":{"1min":1.262,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.95599999999999996,"15min":0.96499999999999997},"min":{"1min":0.73599999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.234,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.85399999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678050,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1107750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.5219999999999998,"5min":3.5219999999999998,"15min":3.5219999999999998},"last":0.48899999999999999}]},{"osd":1,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.58999999999999997,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.309,"15min":0.309},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.56200000000000006}]},{"osd":2,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.75900000000000001,"15min":0.751},"min":{"1min":0.46600000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.69899999999999995}]},{"osd":3,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70999999999999996,"15min":0.71399999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.74399999999999999,"15min":0.73899999999999999},"min":{"1min":0.63400000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.075,"5min":1.171,"15min":1.171},"last":0.60399999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.43099999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.497,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1160000000000001,"5min":1.194,"15min":1.194},"last":0.66900000000000004}]},{"osd":5,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.83699999999999997,"15min":0.83499999999999996},"min":{"1min":0.58599999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":0.626},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.80000000000000004,"15min":0.78600000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.79800000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.86699999999999999,"15min":0.83099999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":6.4870000000000001,"5min":6.4870000000000001,"15min":6.4870000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.81699999999999995,"15min":0.80400000000000005},"min":{"1min":0.65100000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:33:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.97999999999999998}]},{"osd":8,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.879,"15min":0.86099999999999999},"min":{"1min":0.72499999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88100000000000001,"15min":0.88300000000000001},"min":{"1min":0.66200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.1679999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.58499999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.89500000000000002,"15min":0.88300000000000001},"min":{"1min":0.755,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.90600000000000003,"15min":0.89900000000000002},"min":{"1min":0.61199999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":3.508,"5min":3.508,"15min":3.508},"last":0.76100000000000001}]},{"osd":11,"last update":"Sat Mar 30 00:34:34 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.0820000000000001,"5min":1.526,"15min":1.526},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.91500000000000004,"15min":0.92800000000000005},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.034,"5min":2.262,"15min":2.262},"last":0.93500000000000005}]}]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":447008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.57399999999999995,"15min":0.59299999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.57999999999999996,"15min":0.59799999999999998},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71299999999999997,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.439}]},{"osd":1,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.77800000000000002,"15min":0.78900000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0329999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.82299999999999995,"15min":0.83799999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.018,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.69299999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73899999999999999,"15min":0.75900000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83799999999999997,"5min":1,"15min":1},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82799999999999996,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.66000000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.73499999999999999,"15min":0.753},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.95399999999999996,"5min":1.278,"15min":1.278},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.91800000000000004,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.40899999999999997}]},{"osd":4,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.86499999999999999,"15min":0.88800000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.98499999999999999,"5min":1.476,"15min":1.476},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.85699999999999998,"15min":0.878},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92900000000000005,"5min":1.254,"15min":1.254},"last":0.76400000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.87,"15min":0.88600000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.89600000000000002,"15min":0.92200000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94799999999999995,"5min":1.577,"15min":1.577},"last":0.68400000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.86899999999999999,"15min":0.89500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0600000000000001,"5min":1.486,"15min":1.486},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.90200000000000002,"15min":0.92900000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.073,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.93899999999999995,"15min":0.96299999999999997},"min":{"1min":0.53100000000000003,"5min":0.502,"15min":0.502},"max":{"1min":1.0669999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.028,"15min":1.0620000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.083,"5min":2.234,"15min":2.234},"last":0.74399999999999999}]},{"osd":8,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.998,"15min":1.024},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.095,"5min":1.611,"15min":1.611},"last":0.63},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.0369999999999999,"15min":1.075},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.998,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":0.79100000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:34:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.93600000000000005,"15min":0.95899999999999996},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.0329999999999999,"5min":1.355,"15min":1.355},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":1.002},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.1990000000000001,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.77300000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":446928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":3.8370000000000002,"5min":3.8370000000000002,"15min":3.8370000000000002},"last":0.72299999999999998}]},{"osd":1,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.70799999999999996,"15min":0.69399999999999995},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.75900000000000001,"15min":0.747},"min":{"1min":0.56699999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81899999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88200000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83799999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65700000000000003}]},{"osd":3,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.629},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73199999999999998,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.8570000000000002,"5min":3.8570000000000002,"15min":3.8570000000000002},"last":0.60099999999999998}]},{"osd":4,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.72499999999999998,"15min":0.72199999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.90600000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.78300000000000003,"15min":0.77200000000000002},"min":{"1min":0.628,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0369999999999999,"5min":1.246,"15min":1.246},"last":0.90200000000000002}]},{"osd":5,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.53100000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95199999999999996,"5min":1.026,"15min":1.026},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92200000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.47599999999999998}]},{"osd":6,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79900000000000004,"15min":0.81699999999999995},"min":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.98599999999999999,"5min":1.054,"15min":1.054},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.86199999999999999,"15min":0.87},"min":{"1min":0.61399999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":3.7650000000000001,"5min":3.7650000000000001,"15min":3.7650000000000001},"last":0.75}]},{"osd":7,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.88200000000000001,"15min":0.89000000000000001},"min":{"1min":0.71699999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.83799999999999997,"15min":0.84599999999999997},"min":{"1min":0.58199999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0469999999999999,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.83199999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.86299999999999999,"15min":0.84199999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.8969999999999998,"5min":3.8969999999999998,"15min":3.8969999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.89900000000000002,"15min":0.88100000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.8250000000000002,"5min":3.8250000000000002,"15min":3.8250000000000002},"last":0.77900000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83599999999999997,"15min":0.83399999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.88200000000000001,"15min":0.86599999999999999},"min":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.93999999999999995}]},{"osd":11,"last update":"Sat Mar 30 00:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.85799999999999998}]}]},{"osd":7,"up_from":50,"seq":214748364846,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1544192,"data_stored":1039363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53200000000000003,"15min":0.52300000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93899999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.746}]},{"osd":2,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89500000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.0049999999999999,"5min":1.111,"15min":1.111},"last":0.61199999999999999}]},{"osd":3,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93400000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68400000000000005,"15min":0.67200000000000004},"min":{"1min":0.47499999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97599999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.81100000000000005}]},{"osd":4,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68500000000000005,"15min":0.66300000000000003},"min":{"1min":0.45300000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0209999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94099999999999995,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.84199999999999997}]},{"osd":5,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.76900000000000002,"15min":0.78600000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.94799999999999995,"5min":1.095,"15min":1.095},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.78500000000000003,"15min":0.78700000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.96899999999999997,"5min":1.252,"15min":1.252},"last":0.68600000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:34:05 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78500000000000003,"15min":0.77800000000000002},"min":{"1min":0.58099999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98999999999999999,"5min":1.254,"15min":1.254},"last":0.879},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.78100000000000003,"15min":0.78800000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.88400000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.65400000000000003}]},{"osd":8,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84399999999999997,"15min":0.85199999999999998},"min":{"1min":0.63300000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.026,"5min":1.236,"15min":1.236},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.84199999999999997,"15min":0.83599999999999997},"min":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0640000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.91100000000000003}]},{"osd":9,"last update":"Sat Mar 30 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86899999999999999,"15min":0.877},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.028,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.85499999999999998,"15min":0.86899999999999999},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.98799999999999999,"5min":1.284,"15min":1.284},"last":0.89100000000000001}]},{"osd":10,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.88900000000000001,"15min":0.89900000000000002},"min":{"1min":0.68200000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0009999999999999,"5min":1.321,"15min":1.321},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.91100000000000003,"15min":0.91600000000000004},"min":{"1min":0.63300000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.048,"5min":1.3,"15min":1.3},"last":0.96699999999999997}]},{"osd":11,"last update":"Sat Mar 30 00:34:16 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.90200000000000002}]}]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":937984,"data_stored":446397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.48799999999999999,"15min":0.49199999999999999},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64900000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77700000000000002,"5min":0.873,"15min":0.873},"last":0.436}]},{"osd":1,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.625},"min":{"1min":0.46000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.626},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71399999999999997}]},{"osd":2,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91100000000000003,"5min":1.002,"15min":1.002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0069999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.54400000000000004}]},{"osd":3,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.877,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.38800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.84999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.497}]},{"osd":4,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.048,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.749,"15min":0.749},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.495},"max":{"1min":0.96899999999999997,"5min":1.238,"15min":1.238},"last":0.749}]},{"osd":5,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.78500000000000003,"15min":0.78300000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0329999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.55900000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1080000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.53800000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80600000000000005,"15min":0.80800000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90500000000000003,"5min":1.105,"15min":1.105},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.99099999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77300000000000002}]},{"osd":8,"last update":"Sat Mar 30 00:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.85799999999999998,"15min":0.89500000000000002},"min":{"1min":0.60399999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.96999999999999997,"5min":1.252,"15min":1.252},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.82799999999999996,"15min":0.85799999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95099999999999996,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.56699999999999995}]},{"osd":9,"last update":"Sat Mar 30 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.84299999999999997,"15min":0.86499999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.85799999999999998,"15min":0.877},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.89400000000000002,"15min":0.90900000000000003},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":0.97799999999999998,"5min":1.155,"15min":1.155},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.88600000000000001,"15min":0.89600000000000002},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.0680000000000001,"5min":1.206,"15min":1.206},"last":0.84199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":1007616,"data_stored":515520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.64000000000000001,"15min":0.59699999999999998},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80100000000000005,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.498},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64400000000000002,"15min":0.62},"min":{"1min":0.41099999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Sat Mar 30 00:34:28 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.66200000000000003,"15min":0.58699999999999997},"min":{"1min":0.495,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.94199999999999995,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.70599999999999996,"15min":0.61799999999999999},"min":{"1min":0.54500000000000004,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0640000000000001,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.85399999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.65100000000000002,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.125,"15min":1.125},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64000000000000001,"15min":0.59699999999999998},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.98899999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.65600000000000003}]},{"osd":4,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.52700000000000002,"15min":0.48999999999999999},"min":{"1min":0.40500000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.89400000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57399999999999995,"15min":0.51800000000000002},"min":{"1min":0.45800000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91600000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.80600000000000005}]},{"osd":5,"last update":"Sat Mar 30 00:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.70099999999999996,"15min":0.65000000000000002},"min":{"1min":0.65400000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72199999999999998,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.99099999999999999,"5min":3.3039999999999998,"15min":3.3039999999999998},"last":0.74299999999999999}]},{"osd":6,"last update":"Sat Mar 30 00:33:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69899999999999995,"15min":0.69499999999999995},"min":{"1min":0.49399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.876},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71499999999999997,"15min":0.69599999999999995},"min":{"1min":0.57899999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.079,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.52400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93100000000000005,"5min":1.046,"15min":1.046},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68700000000000006,"15min":0.65700000000000003},"min":{"1min":0.47899999999999998,"5min":0.438,"15min":0.438},"max":{"1min":1.099,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.90800000000000003}]},{"osd":8,"last update":"Sat Mar 30 00:34:33 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78100000000000003,"15min":0.75900000000000001},"min":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88200000000000001,"15min":0.88300000000000001},"min":{"1min":0.67200000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.159,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.95099999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.873,"15min":0.871},"min":{"1min":0.64600000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.125,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.018},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1200000000000001,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.92600000000000005}]},{"osd":10,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1679999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.87,"15min":0.86199999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.98799999999999999}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0409999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":958464,"data_stored":447668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 00:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.42299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78400000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68899999999999995,"15min":0.60299999999999998},"min":{"1min":0.42399999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.619,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.68600000000000005}]},{"osd":2,"last update":"Sat Mar 30 00:34:24 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.71799999999999997,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6819999999999999,"5min":5.8550000000000004,"15min":5.8550000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69499999999999995,"15min":0.80300000000000005},"min":{"1min":0.442,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90800000000000003,"5min":5.282,"15min":5.282},"last":0.75900000000000001}]},{"osd":3,"last update":"Sat Mar 30 00:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.621}]},{"osd":4,"last update":"Sat Mar 30 00:34:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.65900000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.58499999999999996}]},{"osd":5,"last update":"Sat Mar 30 00:34:27 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.68400000000000005,"15min":0.67000000000000004},"min":{"1min":0.58799999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.79000000000000004,"15min":0.91300000000000003},"min":{"1min":0.46800000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89300000000000002,"5min":6.0220000000000002,"15min":6.0220000000000002},"last":0.69199999999999995}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.377,"15min":0.377},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.629,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.113,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.48199999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.74099999999999999,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.125,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2470000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.54900000000000004}]},{"osd":8,"last update":"Sat Mar 30 00:34:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.754,"15min":0.79000000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.84799999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.83299999999999996,"15min":0.874},"min":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.70599999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.84599999999999997,"15min":0.86899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":1.161,"15min":1.161},"last":0.54000000000000004}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.82999999999999996,"15min":0.84499999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1899999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.90500000000000003,"15min":0.91200000000000003},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.77200000000000002}]},{"osd":11,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77000000000000002,"15min":0.76300000000000001},"min":{"1min":0.63900000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.749}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":1032192,"data_stored":517318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.623,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82499999999999996,"5min":1.131,"15min":1.131},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.45800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997}]},{"osd":1,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.63800000000000001,"15min":0.59699999999999998},"min":{"1min":0.53100000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.501},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.70399999999999996,"15min":0.63800000000000001},"min":{"1min":0.56200000000000006,"5min":0.375,"15min":0.375},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Sat Mar 30 00:33:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68200000000000005,"15min":0.66800000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93799999999999994,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.45000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.68999999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.91400000000000003}]},{"osd":5,"last update":"Sat Mar 30 00:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.59899999999999998,"15min":0.53100000000000003},"min":{"1min":0.49299999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0289999999999999,"5min":1.147,"15min":1.147},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.60999999999999999,"15min":0.56200000000000006},"min":{"1min":0.45700000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89400000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67900000000000005}]},{"osd":6,"last update":"Sat Mar 30 00:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77300000000000002,"15min":0.74299999999999999},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0309999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.72099999999999997,"15min":0.69699999999999995},"min":{"1min":0.55900000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.85599999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:33:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.65700000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.80600000000000005,"15min":0.79400000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.83299999999999996}]},{"osd":8,"last update":"Sat Mar 30 00:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.74399999999999999,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97199999999999998,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.85699999999999998,"15min":0.83099999999999996},"min":{"1min":0.67000000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.052,"5min":1.321,"15min":1.321},"last":0.71799999999999997}]},{"osd":9,"last update":"Sat Mar 30 00:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95799999999999996,"15min":0.95999999999999996},"min":{"1min":0.68500000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.137,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.90800000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.88200000000000001,"15min":0.872},"min":{"1min":0.77300000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.1439999999999999,"5min":1.204,"15min":1.204},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.004,"15min":0.996},"min":{"1min":0.85499999999999998,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.218,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.0429999999999999}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.0129999999999999}]}]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":446708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.42499999999999999,"15min":0.40100000000000002},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46800000000000003,"15min":0.437},"min":{"1min":0.33900000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.40200000000000002}]},{"osd":1,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.622,"15min":0.58599999999999997},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Mar 30 00:33:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68600000000000005,"15min":0.67800000000000005},"min":{"1min":0.49299999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.46000000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.56499999999999995}]},{"osd":4,"last update":"Sat Mar 30 00:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68300000000000005,"15min":0.63600000000000001},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Sat Mar 30 00:34:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71399999999999997,"15min":0.68999999999999995},"min":{"1min":0.48999999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.748,"15min":0.69999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.79500000000000004}]},{"osd":6,"last update":"Sat Mar 30 00:33:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96299999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67000000000000004,"15min":0.65500000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Sat Mar 30 00:34:07 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78100000000000003,"15min":0.77400000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.74299999999999999}]},{"osd":8,"last update":"Sat Mar 30 00:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.66700000000000004}]},{"osd":9,"last update":"Sat Mar 30 00:33:52 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.87,"15min":0.88400000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.77800000000000002,"15min":0.79100000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0209999999999999,"5min":1.099,"15min":1.099},"last":0.502}]},{"osd":10,"last update":"Sat Mar 30 00:34:11 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.85399999999999998,"15min":0.86599999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.71899999999999997}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.82499999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":942080,"data_stored":447282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42599999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.59899999999999998,"15min":0.56699999999999995},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.497}]},{"osd":1,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48599999999999999,"15min":0.45200000000000001},"min":{"1min":0.372,"5min":0.314,"15min":0.314},"max":{"1min":0.68000000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.48699999999999999,"15min":0.45500000000000002},"min":{"1min":0.39200000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71399999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.49299999999999999}]},{"osd":2,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65600000000000003,"15min":0.64300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.67600000000000005,"15min":0.65100000000000002},"min":{"1min":0.55900000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.81200000000000006}]},{"osd":3,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.69999999999999996,"15min":0.65800000000000003},"min":{"1min":0.54700000000000004,"5min":0.439,"15min":0.439},"max":{"1min":0.97699999999999998,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.68899999999999995,"15min":0.66300000000000003},"min":{"1min":0.52800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83999999999999997,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.94699999999999995}]},{"osd":5,"last update":"Sat Mar 30 00:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70499999999999996,"15min":0.68300000000000005},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0049999999999999,"5min":1.137,"15min":1.137},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71399999999999997,"15min":0.67600000000000005},"min":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0720000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.76100000000000001}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0369999999999999,"5min":1.071,"15min":1.071},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82399999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Sat Mar 30 00:34:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68999999999999995,"15min":0.66200000000000003},"min":{"1min":0.51500000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.69699999999999995,"15min":0.67400000000000004},"min":{"1min":0.54200000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.70899999999999996}]},{"osd":8,"last update":"Sat Mar 30 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.83199999999999996,"15min":0.82399999999999995},"min":{"1min":0.64000000000000001,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0589999999999999,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.79400000000000004,"15min":0.77700000000000002},"min":{"1min":0.64600000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.026,"5min":1.153,"15min":1.153},"last":0.879}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.86499999999999999,"15min":0.88500000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97499999999999998,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.91200000000000003,"15min":0.92900000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0549999999999999,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.91400000000000003}]},{"osd":10,"last update":"Sat Mar 30 00:34:13 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.872,"15min":0.87},"min":{"1min":0.69199999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.167,"5min":1.179,"15min":1.179},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.871,"15min":0.879},"min":{"1min":0.57499999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.73299999999999998}]},{"osd":11,"last update":"Sat Mar 30 00:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.95999999999999996}]}]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32628,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712524,"statfs":{"total":95995035648,"available":95961624576,"internally_reserved":0,"allocated":1552384,"data_stored":1038268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.71599999999999997,"15min":0.81299999999999994},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88800000000000001,"5min":8.077,"15min":8.077},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.64900000000000002,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.67600000000000005}]},{"osd":1,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.64100000000000001,"15min":0.61899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95199999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.69299999999999995,"15min":0.64800000000000002},"min":{"1min":0.52800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.82299999999999995}]},{"osd":2,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.47999999999999998,"15min":0.45400000000000001},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.503},"min":{"1min":0.38600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85399999999999998,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.47299999999999998}]},{"osd":3,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.73699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.504},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.753,"15min":0.86699999999999999},"min":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94199999999999995,"5min":8.0559999999999992,"15min":8.0559999999999992},"last":0.54600000000000004}]},{"osd":4,"last update":"Sat Mar 30 00:34:26 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.73199999999999998,"15min":0.67800000000000005},"min":{"1min":0.55300000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1419999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69799999999999995,"15min":0.67400000000000004},"min":{"1min":0.54500000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94199999999999995,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.748}]},{"osd":6,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64700000000000002,"15min":0.629},"min":{"1min":0.56000000000000005,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84999999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.72399999999999998,"15min":0.68600000000000005},"min":{"1min":0.59999999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.73099999999999998}]},{"osd":7,"last update":"Sat Mar 30 00:34:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.57199999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.72999999999999998,"15min":0.71399999999999997},"min":{"1min":0.623,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.93899999999999995,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.89000000000000001}]},{"osd":8,"last update":"Sat Mar 30 00:34:31 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.77400000000000002,"15min":0.72999999999999998},"min":{"1min":0.65400000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.73699999999999999,"15min":0.70099999999999996},"min":{"1min":0.621,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.70899999999999996}]},{"osd":9,"last update":"Sat Mar 30 00:33:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.875,"15min":0.86899999999999999},"min":{"1min":0.70699999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.83499999999999996,"15min":0.83399999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.071,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.84699999999999998}]},{"osd":10,"last update":"Sat Mar 30 00:34:12 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.86499999999999999,"15min":0.85899999999999999},"min":{"1min":0.71199999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.0589999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.872},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.79700000000000004,"15min":0.78900000000000003},"min":{"1min":0.67200000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.83099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T00:34:44.908 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:34:45.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.127+0000 7f9ddb200700 1 -- 172.21.15.99:0/3349194258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ddc101720 msgr2=0x7f9ddc103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:45.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.127+0000 7f9ddb200700 1 --2- 172.21.15.99:0/3349194258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ddc101720 0x7f9ddc103b10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009990 tx=0x7f9dcc009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.127+0000 7f9ddb200700 1 -- 172.21.15.99:0/3349194258 shutdown_connections 2024-03-30T00:34:45.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.127+0000 7f9ddb200700 1 --2- 172.21.15.99:0/3349194258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ddc106980 0x7f9ddc108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.127+0000 7f9ddb200700 1 --2- 172.21.15.99:0/3349194258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ddc101720 0x7f9ddc103b10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.127+0000 7f9ddb200700 1 --2- 172.21.15.99:0/3349194258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc104050 0x7f9ddc106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.127+0000 7f9ddb200700 1 -- 172.21.15.99:0/3349194258 >> 172.21.15.99:0/3349194258 conn(0x7f9ddc0fb090 msgr2=0x7f9ddc0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:45.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.127+0000 7f9ddb200700 1 -- 172.21.15.99:0/3349194258 shutdown_connections 2024-03-30T00:34:45.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.127+0000 7f9ddb200700 1 -- 172.21.15.99:0/3349194258 wait complete. 2024-03-30T00:34:45.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.128+0000 7f9ddb200700 1 Processor -- start 2024-03-30T00:34:45.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.128+0000 7f9ddb200700 1 -- start start 2024-03-30T00:34:45.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.128+0000 7f9ddb200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ddc101720 0x7f9ddc101040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:45.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.128+0000 7f9ddb200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc104050 0x7f9ddc0ff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.128+0000 7f9ddb200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ddc106980 0x7f9ddc0ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.128+0000 7f9ddb200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc1003b0 con 0x7f9ddc101720 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.128+0000 7f9ddb200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc100520 con 0x7f9ddc106980 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.128+0000 7f9ddb200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc1a0b80 con 0x7f9ddc104050 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dd9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ddc101720 0x7f9ddc101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dd9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ddc101720 0x7f9ddc101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56462/0 (socket says 172.21.15.99:56462) 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dd9e00700 1 -- 172.21.15.99:0/210854334 learned_addr learned my addr 172.21.15.99:0/210854334 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dda800700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ddc106980 0x7f9ddc0ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dd9400700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc104050 0x7f9ddc0ff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dd9e00700 1 -- 172.21.15.99:0/210854334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc104050 msgr2=0x7f9ddc0ff6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dd9e00700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc104050 0x7f9ddc0ff6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dd9e00700 1 -- 172.21.15.99:0/210854334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ddc106980 msgr2=0x7f9ddc0ffc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dd9e00700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ddc106980 0x7f9ddc0ffc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dd9e00700 1 -- 172.21.15.99:0/210854334 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc0095f0 con 0x7f9ddc101720 2024-03-30T00:34:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dd9400700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc104050 0x7f9ddc0ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.129+0000 7f9dd9e00700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ddc101720 0x7f9ddc101040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc005740 tx=0x7f9dcc005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.130+0000 7f9dd2a00700 1 -- 172.21.15.99:0/210854334 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc01d070 con 0x7f9ddc101720 2024-03-30T00:34:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.130+0000 7f9dd2a00700 1 -- 172.21.15.99:0/210854334 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9dcc00beb0 con 0x7f9ddc101720 2024-03-30T00:34:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.130+0000 7f9ddb200700 1 -- 172.21.15.99:0/210854334 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ddc1a0db0 con 0x7f9ddc101720 2024-03-30T00:34:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.130+0000 7f9dd2a00700 1 -- 172.21.15.99:0/210854334 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc021410 con 0x7f9ddc101720 2024-03-30T00:34:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.130+0000 7f9ddb200700 1 -- 172.21.15.99:0/210854334 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ddc1a11d0 con 0x7f9ddc101720 2024-03-30T00:34:45.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.131+0000 7f9ddb200700 1 -- 172.21.15.99:0/210854334 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ddc0fcc10 con 0x7f9ddc101720 2024-03-30T00:34:45.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.133+0000 7f9dd2a00700 1 -- 172.21.15.99:0/210854334 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f9dcc00f720 con 0x7f9ddc101720 2024-03-30T00:34:45.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.134+0000 7f9dd2a00700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9dc00a11a0 0x7f9dc00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:45.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.134+0000 7f9dd2a00700 1 -- 172.21.15.99:0/210854334 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9dcc0cdce0 con 0x7f9ddc101720 2024-03-30T00:34:45.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.134+0000 7f9dd9400700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9dc00a11a0 0x7f9dc00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:45.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.135+0000 7f9dd9400700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9dc00a11a0 0x7f9dc00a3660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009be0 tx=0x7f9dc8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:45.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.138+0000 7f9dd2a00700 1 -- 172.21.15.99:0/210854334 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9dcc097490 con 0x7f9ddc101720 2024-03-30T00:34:45.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.385+0000 7f9ddb200700 1 -- 172.21.15.99:0/210854334 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ddc0679a0 con 0x7f9ddc101720 2024-03-30T00:34:45.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.386+0000 7f9dd2a00700 1 -- 172.21.15.99:0/210854334 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22615 (secure 0 0 0) 0x7f9dcc09aab0 con 0x7f9ddc101720 2024-03-30T00:34:45.393 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:45.393 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:34:10.248126+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8},{"from":5,"to":7}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 -- 172.21.15.99:0/210854334 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9dc00a11a0 msgr2=0x7f9dc00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9dc00a11a0 0x7f9dc00a3660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009be0 tx=0x7f9dc8009260 comp rx=0 tx=0).stop 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 -- 172.21.15.99:0/210854334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ddc101720 msgr2=0x7f9ddc101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ddc101720 0x7f9ddc101040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc005740 tx=0x7f9dcc005770 comp rx=0 tx=0).stop 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 -- 172.21.15.99:0/210854334 shutdown_connections 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9dc00a11a0 0x7f9dc00a3660 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ddc106980 0x7f9ddc0ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ddc101720 0x7f9ddc101040 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 --2- 172.21.15.99:0/210854334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc104050 0x7f9ddc0ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 -- 172.21.15.99:0/210854334 >> 172.21.15.99:0/210854334 conn(0x7f9ddc0fb090 msgr2=0x7f9ddc1021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 -- 172.21.15.99:0/210854334 shutdown_connections 2024-03-30T00:34:45.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.389+0000 7f9ddb200700 1 -- 172.21.15.99:0/210854334 wait complete. 2024-03-30T00:34:45.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:33:14.085184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:33:15.090699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:34:45.416 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:34:45.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.603+0000 7f9878400700 1 -- 172.21.15.99:0/2783649152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9870102910 msgr2=0x7f98701091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:45.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.603+0000 7f9878400700 1 --2- 172.21.15.99:0/2783649152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9870102910 0x7f98701091c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f986800b0d0 tx=0x7f986800b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.603+0000 7f9878400700 1 -- 172.21.15.99:0/2783649152 shutdown_connections 2024-03-30T00:34:45.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.603+0000 7f9878400700 1 --2- 172.21.15.99:0/2783649152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9870102910 0x7f98701091c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.603+0000 7f9878400700 1 --2- 172.21.15.99:0/2783649152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9870100d80 0x7f98701011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.603+0000 7f9878400700 1 --2- 172.21.15.99:0/2783649152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9870101f70 0x7f98701023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.603+0000 7f9878400700 1 -- 172.21.15.99:0/2783649152 >> 172.21.15.99:0/2783649152 conn(0x7f98700fc2f0 msgr2=0x7f98700fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:45.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.603+0000 7f9878400700 1 -- 172.21.15.99:0/2783649152 shutdown_connections 2024-03-30T00:34:45.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.603+0000 7f9878400700 1 -- 172.21.15.99:0/2783649152 wait complete. 2024-03-30T00:34:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.604+0000 7f9878400700 1 Processor -- start 2024-03-30T00:34:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.604+0000 7f9878400700 1 -- start start 2024-03-30T00:34:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.604+0000 7f9878400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9870100d80 0x7f9870198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.604+0000 7f9878400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9870101f70 0x7f98701994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.604+0000 7f9878400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9870102910 0x7f98701a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9878400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98701a0a80 con 0x7f9870100d80 2024-03-30T00:34:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9878400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98701a0bc0 con 0x7f9870102910 2024-03-30T00:34:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9878400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98701a0d50 con 0x7f9870101f70 2024-03-30T00:34:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9875a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9870100d80 0x7f9870198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9875a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9870100d80 0x7f9870198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56478/0 (socket says 172.21.15.99:56478) 2024-03-30T00:34:45.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9875a00700 1 -- 172.21.15.99:0/2283274360 learned_addr learned my addr 172.21.15.99:0/2283274360 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9876400700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9870102910 0x7f98701a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9875000700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9870101f70 0x7f98701994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9875a00700 1 -- 172.21.15.99:0/2283274360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9870101f70 msgr2=0x7f98701994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9875a00700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9870101f70 0x7f98701994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9875a00700 1 -- 172.21.15.99:0/2283274360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9870102910 msgr2=0x7f98701a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9875a00700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9870102910 0x7f98701a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9875a00700 1 -- 172.21.15.99:0/2283274360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9868009d00 con 0x7f9870100d80 2024-03-30T00:34:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9875000700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9870101f70 0x7f98701994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.605+0000 7f9875a00700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9870100d80 0x7f9870198fa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f986400e970 tx=0x7f986400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.606+0000 7f986e000700 1 -- 172.21.15.99:0/2283274360 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864009790 con 0x7f9870100d80 2024-03-30T00:34:45.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.606+0000 7f986e000700 1 -- 172.21.15.99:0/2283274360 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f986400eec0 con 0x7f9870100d80 2024-03-30T00:34:45.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.606+0000 7f9878400700 1 -- 172.21.15.99:0/2283274360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98701a0fd0 con 0x7f9870100d80 2024-03-30T00:34:45.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.606+0000 7f986e000700 1 -- 172.21.15.99:0/2283274360 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864003c10 con 0x7f9870100d80 2024-03-30T00:34:45.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.606+0000 7f9878400700 1 -- 172.21.15.99:0/2283274360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98701a14a0 con 0x7f9870100d80 2024-03-30T00:34:45.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.607+0000 7f9878400700 1 -- 172.21.15.99:0/2283274360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9870067af0 con 0x7f9870100d80 2024-03-30T00:34:45.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.609+0000 7f986e000700 1 -- 172.21.15.99:0/2283274360 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f9864003db0 con 0x7f9870100d80 2024-03-30T00:34:45.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.610+0000 7f986e000700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f98580a11a0 0x7f98580a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:45.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.610+0000 7f986e000700 1 -- 172.21.15.99:0/2283274360 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f9864014070 con 0x7f9870100d80 2024-03-30T00:34:45.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.610+0000 7f9875000700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f98580a11a0 0x7f98580a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:45.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.611+0000 7f9875000700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f98580a11a0 0x7f98580a3660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f985c005950 tx=0x7f985c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:45.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.614+0000 7f986e000700 1 -- 172.21.15.99:0/2283274360 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9864096240 con 0x7f9870100d80 2024-03-30T00:34:45.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:45 smithi161 ceph-mon[42221]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:45.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:45 smithi161 ceph-mon[42221]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:45.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:45 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/210854334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:45.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:45 smithi099 ceph-mon[38079]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:45.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:45 smithi099 ceph-mon[38079]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:45.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:45 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/210854334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:45.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:45 smithi142 ceph-mon[41779]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:45.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:45 smithi142 ceph-mon[41779]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:45.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:45 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/210854334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:45.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.819+0000 7f9878400700 1 -- 172.21.15.99:0/2283274360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98701a1900 con 0x7f9870100d80 2024-03-30T00:34:45.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.819+0000 7f986e000700 1 -- 172.21.15.99:0/2283274360 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9864099860 con 0x7f9870100d80 2024-03-30T00:34:45.826 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:34:45.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.821+0000 7f9878400700 1 -- 172.21.15.99:0/2283274360 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f98580a11a0 msgr2=0x7f98580a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:45.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.821+0000 7f9878400700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f98580a11a0 0x7f98580a3660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f985c005950 tx=0x7f985c0058e0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.821+0000 7f9878400700 1 -- 172.21.15.99:0/2283274360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9870100d80 msgr2=0x7f9870198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:45.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.821+0000 7f9878400700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9870100d80 0x7f9870198fa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f986400e970 tx=0x7f986400ec80 comp rx=0 tx=0).stop 2024-03-30T00:34:45.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.822+0000 7f9878400700 1 -- 172.21.15.99:0/2283274360 shutdown_connections 2024-03-30T00:34:45.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.822+0000 7f9878400700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f98580a11a0 0x7f98580a3660 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.822+0000 7f9878400700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9870102910 0x7f98701a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.822+0000 7f9878400700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9870100d80 0x7f9870198fa0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.822+0000 7f9878400700 1 --2- 172.21.15.99:0/2283274360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9870101f70 0x7f98701994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:45.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.822+0000 7f9878400700 1 -- 172.21.15.99:0/2283274360 >> 172.21.15.99:0/2283274360 conn(0x7f98700fc2f0 msgr2=0x7f9870072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:45.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.822+0000 7f9878400700 1 -- 172.21.15.99:0/2283274360 shutdown_connections 2024-03-30T00:34:45.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:45.822+0000 7f9878400700 1 -- 172.21.15.99:0/2283274360 wait complete. 2024-03-30T00:34:45.847 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:34:46.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.036+0000 7f93dd600700 1 -- 172.21.15.99:0/3853354266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100d80 msgr2=0x7f93d81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.036+0000 7f93dd600700 1 --2- 172.21.15.99:0/3853354266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100d80 0x7f93d81011a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f93c80099c0 tx=0x7f93c8009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.036+0000 7f93dd600700 1 -- 172.21.15.99:0/3853354266 shutdown_connections 2024-03-30T00:34:46.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.036+0000 7f93dd600700 1 --2- 172.21.15.99:0/3853354266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102910 0x7f93d81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.036+0000 7f93dd600700 1 --2- 172.21.15.99:0/3853354266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100d80 0x7f93d81011a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.036+0000 7f93dd600700 1 --2- 172.21.15.99:0/3853354266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8101f70 0x7f93d81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.036+0000 7f93dd600700 1 -- 172.21.15.99:0/3853354266 >> 172.21.15.99:0/3853354266 conn(0x7f93d80fc2f0 msgr2=0x7f93d80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:46.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.036+0000 7f93dd600700 1 -- 172.21.15.99:0/3853354266 shutdown_connections 2024-03-30T00:34:46.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.036+0000 7f93dd600700 1 -- 172.21.15.99:0/3853354266 wait complete. 2024-03-30T00:34:46.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.037+0000 7f93dd600700 1 Processor -- start 2024-03-30T00:34:46.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.037+0000 7f93dd600700 1 -- start start 2024-03-30T00:34:46.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.037+0000 7f93dd600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8100d80 0x7f93d8198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.037+0000 7f93dd600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8101f70 0x7f93d81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.037+0000 7f93dd600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102910 0x7f93d81a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93dd600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d81a09c0 con 0x7f93d8101f70 2024-03-30T00:34:46.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93dd600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d81a0b00 con 0x7f93d8100d80 2024-03-30T00:34:46.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93dd600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d81a0c90 con 0x7f93d8102910 2024-03-30T00:34:46.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d6000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8101f70 0x7f93d81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d6000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8101f70 0x7f93d81993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56504/0 (socket says 172.21.15.99:56504) 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d6000700 1 -- 172.21.15.99:0/769778269 learned_addr learned my addr 172.21.15.99:0/769778269 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d6a00700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8100d80 0x7f93d8198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d7400700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102910 0x7f93d81a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d6000700 1 -- 172.21.15.99:0/769778269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102910 msgr2=0x7f93d81a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d6000700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102910 0x7f93d81a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d6000700 1 -- 172.21.15.99:0/769778269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8100d80 msgr2=0x7f93d8198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d6000700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8100d80 0x7f93d8198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d6000700 1 -- 172.21.15.99:0/769778269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c80095f0 con 0x7f93d8101f70 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d7400700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102910 0x7f93d81a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.038+0000 7f93d6000700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8101f70 0x7f93d81993d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f93c400e9a0 tx=0x7f93c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.039+0000 7f93cf400700 1 -- 172.21.15.99:0/769778269 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c4009790 con 0x7f93d8101f70 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.039+0000 7f93cf400700 1 -- 172.21.15.99:0/769778269 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93c400cbd0 con 0x7f93d8101f70 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.039+0000 7f93dd600700 1 -- 172.21.15.99:0/769778269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d81a0f70 con 0x7f93d8101f70 2024-03-30T00:34:46.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.039+0000 7f93cf400700 1 -- 172.21.15.99:0/769778269 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c4018680 con 0x7f93d8101f70 2024-03-30T00:34:46.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.039+0000 7f93dd600700 1 -- 172.21.15.99:0/769778269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d81a13f0 con 0x7f93d8101f70 2024-03-30T00:34:46.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.040+0000 7f93dd600700 1 -- 172.21.15.99:0/769778269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d8067af0 con 0x7f93d8101f70 2024-03-30T00:34:46.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.042+0000 7f93cf400700 1 -- 172.21.15.99:0/769778269 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f93c400cd40 con 0x7f93d8101f70 2024-03-30T00:34:46.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.043+0000 7f93cf400700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 0x7f93bc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.043+0000 7f93cf400700 1 -- 172.21.15.99:0/769778269 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f93c4014070 con 0x7f93d8101f70 2024-03-30T00:34:46.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.043+0000 7f93d6a00700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 0x7f93bc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.044+0000 7f93d6a00700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 0x7f93bc0a35e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009990 tx=0x7f93c8003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:46.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.047+0000 7f93cf400700 1 -- 172.21.15.99:0/769778269 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f93c40529f0 con 0x7f93d8101f70 2024-03-30T00:34:46.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.251+0000 7f93dd600700 1 -- 172.21.15.99:0/769778269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93d81a1720 con 0x7f93d8101f70 2024-03-30T00:34:46.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.252+0000 7f93cf400700 1 -- 172.21.15.99:0/769778269 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f93c40962c0 con 0x7f93d8101f70 2024-03-30T00:34:46.258 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-30T00:34:46.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.254+0000 7f93dd600700 1 -- 172.21.15.99:0/769778269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 msgr2=0x7f93bc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.254+0000 7f93dd600700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 0x7f93bc0a35e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009990 tx=0x7f93c8003680 comp rx=0 tx=0).stop 2024-03-30T00:34:46.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.254+0000 7f93dd600700 1 -- 172.21.15.99:0/769778269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8101f70 msgr2=0x7f93d81993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.254+0000 7f93dd600700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8101f70 0x7f93d81993d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f93c400e9a0 tx=0x7f93c400ecb0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.254+0000 7f93dd600700 1 -- 172.21.15.99:0/769778269 shutdown_connections 2024-03-30T00:34:46.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.254+0000 7f93dd600700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102910 0x7f93d81a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.254+0000 7f93dd600700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 0x7f93bc0a35e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.254+0000 7f93dd600700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8100d80 0x7f93d8198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.254+0000 7f93dd600700 1 --2- 172.21.15.99:0/769778269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8101f70 0x7f93d81993d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.254+0000 7f93dd600700 1 -- 172.21.15.99:0/769778269 >> 172.21.15.99:0/769778269 conn(0x7f93d80fc2f0 msgr2=0x7f93d81077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:46.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.254+0000 7f93dd600700 1 -- 172.21.15.99:0/769778269 shutdown_connections 2024-03-30T00:34:46.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.255+0000 7f93dd600700 1 -- 172.21.15.99:0/769778269 wait complete. 2024-03-30T00:34:46.280 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:34:46.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.468+0000 7f21e2600700 1 -- 172.21.15.99:0/635299319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21dc102910 msgr2=0x7f21dc1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.468+0000 7f21e2600700 1 --2- 172.21.15.99:0/635299319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21dc102910 0x7f21dc1091c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f21d400b0d0 tx=0x7f21d400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.468+0000 7f21e2600700 1 -- 172.21.15.99:0/635299319 shutdown_connections 2024-03-30T00:34:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.468+0000 7f21e2600700 1 --2- 172.21.15.99:0/635299319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21dc102910 0x7f21dc1091c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.468+0000 7f21e2600700 1 --2- 172.21.15.99:0/635299319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21dc100d80 0x7f21dc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.468+0000 7f21e2600700 1 --2- 172.21.15.99:0/635299319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc101f70 0x7f21dc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.468+0000 7f21e2600700 1 -- 172.21.15.99:0/635299319 >> 172.21.15.99:0/635299319 conn(0x7f21dc0fc2f0 msgr2=0x7f21dc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.469+0000 7f21e2600700 1 -- 172.21.15.99:0/635299319 shutdown_connections 2024-03-30T00:34:46.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.469+0000 7f21e2600700 1 -- 172.21.15.99:0/635299319 wait complete. 2024-03-30T00:34:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21e2600700 1 Processor -- start 2024-03-30T00:34:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21e2600700 1 -- start start 2024-03-30T00:34:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21e2600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21dc100d80 0x7f21dc196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21e2600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc101f70 0x7f21dc197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21e2600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21dc102910 0x7f21dc19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21e2600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc19e830 con 0x7f21dc102910 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21e2600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc19e9a0 con 0x7f21dc100d80 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21e2600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc19eb60 con 0x7f21dc101f70 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21dbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21dc102910 0x7f21dc19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21dbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21dc102910 0x7f21dc19e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56508/0 (socket says 172.21.15.99:56508) 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21dbe00700 1 -- 172.21.15.99:0/3834954953 learned_addr learned my addr 172.21.15.99:0/3834954953 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21db400700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21dc100d80 0x7f21dc196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.470+0000 7f21daa00700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc101f70 0x7f21dc197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21dbe00700 1 -- 172.21.15.99:0/3834954953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc101f70 msgr2=0x7f21dc197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21dbe00700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc101f70 0x7f21dc197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21dbe00700 1 -- 172.21.15.99:0/3834954953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21dc100d80 msgr2=0x7f21dc196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21dbe00700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21dc100d80 0x7f21dc196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21dbe00700 1 -- 172.21.15.99:0/3834954953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d4009d00 con 0x7f21dc102910 2024-03-30T00:34:46.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21db400700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21dc100d80 0x7f21dc196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:34:46.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21dbe00700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21dc102910 0x7f21dc19e260 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f21d40079c0 tx=0x7f21d400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:46.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21d3e00700 1 -- 172.21.15.99:0/3834954953 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d400e040 con 0x7f21dc102910 2024-03-30T00:34:46.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21e2600700 1 -- 172.21.15.99:0/3834954953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21dc19ede0 con 0x7f21dc102910 2024-03-30T00:34:46.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21d3e00700 1 -- 172.21.15.99:0/3834954953 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21d400ceb0 con 0x7f21dc102910 2024-03-30T00:34:46.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21e2600700 1 -- 172.21.15.99:0/3834954953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc19f200 con 0x7f21dc102910 2024-03-30T00:34:46.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.471+0000 7f21d3e00700 1 -- 172.21.15.99:0/3834954953 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d401b650 con 0x7f21dc102910 2024-03-30T00:34:46.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.473+0000 7f21e2600700 1 -- 172.21.15.99:0/3834954953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21dc108200 con 0x7f21dc102910 2024-03-30T00:34:46.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.474+0000 7f21d3e00700 1 -- 172.21.15.99:0/3834954953 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f21d401d480 con 0x7f21dc102910 2024-03-30T00:34:46.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.476+0000 7f21d3e00700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f21c00a1120 0x7f21c00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.476+0000 7f21d3e00700 1 -- 172.21.15.99:0/3834954953 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f21d40ce100 con 0x7f21dc102910 2024-03-30T00:34:46.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.476+0000 7f21db400700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f21c00a1120 0x7f21c00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.476+0000 7f21db400700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f21c00a1120 0x7f21c00a35e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0fe6d0 tx=0x7f21cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:46.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.479+0000 7f21d3e00700 1 -- 172.21.15.99:0/3834954953 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f21d40978b0 con 0x7f21dc102910 2024-03-30T00:34:46.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.684+0000 7f21e2600700 1 -- 172.21.15.99:0/3834954953 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f21dc107be0 con 0x7f21dc102910 2024-03-30T00:34:46.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.685+0000 7f21d3e00700 1 -- 172.21.15.99:0/3834954953 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f21d409aed0 con 0x7f21dc102910 2024-03-30T00:34:46.691 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-30T00:34:46.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.687+0000 7f21e2600700 1 -- 172.21.15.99:0/3834954953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f21c00a1120 msgr2=0x7f21c00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.687+0000 7f21e2600700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f21c00a1120 0x7f21c00a35e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0fe6d0 tx=0x7f21cc009260 comp rx=0 tx=0).stop 2024-03-30T00:34:46.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.687+0000 7f21e2600700 1 -- 172.21.15.99:0/3834954953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21dc102910 msgr2=0x7f21dc19e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.687+0000 7f21e2600700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21dc102910 0x7f21dc19e260 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f21d40079c0 tx=0x7f21d400cab0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.687+0000 7f21e2600700 1 -- 172.21.15.99:0/3834954953 shutdown_connections 2024-03-30T00:34:46.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.687+0000 7f21e2600700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21dc102910 0x7f21dc19e260 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.687+0000 7f21e2600700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f21c00a1120 0x7f21c00a35e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.687+0000 7f21e2600700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21dc100d80 0x7f21dc196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.687+0000 7f21e2600700 1 --2- 172.21.15.99:0/3834954953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc101f70 0x7f21dc197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.687+0000 7f21e2600700 1 -- 172.21.15.99:0/3834954953 >> 172.21.15.99:0/3834954953 conn(0x7f21dc0fc2f0 msgr2=0x7f21dc105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:46.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.688+0000 7f21e2600700 1 -- 172.21.15.99:0/3834954953 shutdown_connections 2024-03-30T00:34:46.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.688+0000 7f21e2600700 1 -- 172.21.15.99:0/3834954953 wait complete. 2024-03-30T00:34:46.710 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:46 smithi099 ceph-mon[38079]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:46.710 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:46 smithi099 ceph-mon[38079]: pgmap v303: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:46.710 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:46 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2283274360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:46.710 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:46 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/769778269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:46.712 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:34:46.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:46 smithi161 ceph-mon[42221]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:46.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:46 smithi161 ceph-mon[42221]: pgmap v303: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:46.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:46 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2283274360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:46.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:46 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/769778269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:46.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:46 smithi142 ceph-mon[41779]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T00:34:46.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:46 smithi142 ceph-mon[41779]: pgmap v303: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:46.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:46 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2283274360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:34:46.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:46 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/769778269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:34:46.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.897+0000 7fe1a8200700 1 -- 172.21.15.99:0/3138870426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1a0102810 msgr2=0x7fe1a01090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.897+0000 7fe1a8200700 1 --2- 172.21.15.99:0/3138870426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1a0102810 0x7fe1a01090c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe1980099c0 tx=0x7fe198009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.897+0000 7fe1a8200700 1 -- 172.21.15.99:0/3138870426 shutdown_connections 2024-03-30T00:34:46.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.897+0000 7fe1a8200700 1 --2- 172.21.15.99:0/3138870426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1a0102810 0x7fe1a01090c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.897+0000 7fe1a8200700 1 --2- 172.21.15.99:0/3138870426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a0100c80 0x7fe1a01010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.897+0000 7fe1a8200700 1 --2- 172.21.15.99:0/3138870426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1a0101e70 0x7fe1a01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.897+0000 7fe1a8200700 1 -- 172.21.15.99:0/3138870426 >> 172.21.15.99:0/3138870426 conn(0x7fe1a00fc1f0 msgr2=0x7fe1a00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:46.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.897+0000 7fe1a8200700 1 -- 172.21.15.99:0/3138870426 shutdown_connections 2024-03-30T00:34:46.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.897+0000 7fe1a8200700 1 -- 172.21.15.99:0/3138870426 wait complete. 2024-03-30T00:34:46.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.898+0000 7fe1a8200700 1 Processor -- start 2024-03-30T00:34:46.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.898+0000 7fe1a8200700 1 -- start start 2024-03-30T00:34:46.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a8200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1a0100c80 0x7fe1a0198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a8200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a0101e70 0x7fe1a0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a8200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1a0102810 0x7fe1a01a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a8200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a01a0810 con 0x7fe1a0100c80 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a8200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a01a0980 con 0x7fe1a0101e70 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a8200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a01a0b40 con 0x7fe1a0102810 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1a0100c80 0x7fe1a0198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1a0100c80 0x7fe1a0198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56534/0 (socket says 172.21.15.99:56534) 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a5800700 1 -- 172.21.15.99:0/3142348780 learned_addr learned my addr 172.21.15.99:0/3142348780 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a6200700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1a0102810 0x7fe1a01a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a4e00700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a0101e70 0x7fe1a0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a5800700 1 -- 172.21.15.99:0/3142348780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1a0102810 msgr2=0x7fe1a01a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a5800700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1a0102810 0x7fe1a01a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a5800700 1 -- 172.21.15.99:0/3142348780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a0101e70 msgr2=0x7fe1a0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a5800700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a0101e70 0x7fe1a0199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a5800700 1 -- 172.21.15.99:0/3142348780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980095f0 con 0x7fe1a0100c80 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a6200700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1a0102810 0x7fe1a01a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:46.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a4e00700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a0101e70 0x7fe1a0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:46.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.899+0000 7fe1a5800700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1a0100c80 0x7fe1a0198d40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe194007f10 tx=0x7fe19400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:46.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.900+0000 7fe19e000700 1 -- 172.21.15.99:0/3142348780 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194017070 con 0x7fe1a0100c80 2024-03-30T00:34:46.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.900+0000 7fe19e000700 1 -- 172.21.15.99:0/3142348780 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe19400f040 con 0x7fe1a0100c80 2024-03-30T00:34:46.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.900+0000 7fe1a8200700 1 -- 172.21.15.99:0/3142348780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a01a0e20 con 0x7fe1a0100c80 2024-03-30T00:34:46.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.900+0000 7fe19e000700 1 -- 172.21.15.99:0/3142348780 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194004620 con 0x7fe1a0100c80 2024-03-30T00:34:46.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.900+0000 7fe1a8200700 1 -- 172.21.15.99:0/3142348780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a01a11f0 con 0x7fe1a0100c80 2024-03-30T00:34:46.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.901+0000 7fe1a8200700 1 -- 172.21.15.99:0/3142348780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a0067a10 con 0x7fe1a0100c80 2024-03-30T00:34:46.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.903+0000 7fe19e000700 1 -- 172.21.15.99:0/3142348780 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fe194004780 con 0x7fe1a0100c80 2024-03-30T00:34:46.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.904+0000 7fe19e000700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1880a1120 0x7fe1880a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:46.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.904+0000 7fe19e000700 1 -- 172.21.15.99:0/3142348780 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fe1940cc170 con 0x7fe1a0100c80 2024-03-30T00:34:46.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.904+0000 7fe1a4e00700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1880a1120 0x7fe1880a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:46.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.905+0000 7fe1a4e00700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1880a1120 0x7fe1880a35e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe18c005d90 tx=0x7fe18c005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:46.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:46.908+0000 7fe19e000700 1 -- 172.21.15.99:0/3142348780 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe194095890 con 0x7fe1a0100c80 2024-03-30T00:34:47.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.161+0000 7fe1a8200700 1 -- 172.21.15.99:0/3142348780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1a010c3a0 con 0x7fe1a0100c80 2024-03-30T00:34:47.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.161+0000 7fe19e000700 1 -- 172.21.15.99:0/3142348780 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10860 (secure 0 0 0) 0x7fe194098eb0 con 0x7fe1a0100c80 2024-03-30T00:34:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:47.168 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:34:39.366251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 -- 172.21.15.99:0/3142348780 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1880a1120 msgr2=0x7fe1880a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1880a1120 0x7fe1880a35e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe18c005d90 tx=0x7fe18c005d00 comp rx=0 tx=0).stop 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 -- 172.21.15.99:0/3142348780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1a0100c80 msgr2=0x7fe1a0198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1a0100c80 0x7fe1a0198d40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe194007f10 tx=0x7fe19400d3b0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 -- 172.21.15.99:0/3142348780 shutdown_connections 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1a0102810 0x7fe1a01a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1a0100c80 0x7fe1a0198d40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe1880a1120 0x7fe1880a35e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 --2- 172.21.15.99:0/3142348780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1a0101e70 0x7fe1a0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 -- 172.21.15.99:0/3142348780 >> 172.21.15.99:0/3142348780 conn(0x7fe1a00fc1f0 msgr2=0x7fe1a0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 -- 172.21.15.99:0/3142348780 shutdown_connections 2024-03-30T00:34:47.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.164+0000 7fe1a8200700 1 -- 172.21.15.99:0/3142348780 wait complete. 2024-03-30T00:34:47.172 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2024-03-30T00:34:47.189 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:34:47.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.374+0000 7f93dd400700 1 -- 172.21.15.99:0/37230268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100c80 msgr2=0x7f93d81010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.374+0000 7f93dd400700 1 --2- 172.21.15.99:0/37230268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100c80 0x7f93d81010a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f93c80099c0 tx=0x7f93c8009cd0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.374+0000 7f93dd400700 1 -- 172.21.15.99:0/37230268 shutdown_connections 2024-03-30T00:34:47.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.374+0000 7f93dd400700 1 --2- 172.21.15.99:0/37230268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8102810 0x7f93d81090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.374+0000 7f93dd400700 1 --2- 172.21.15.99:0/37230268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100c80 0x7f93d81010a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.374+0000 7f93dd400700 1 --2- 172.21.15.99:0/37230268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8101e70 0x7f93d81022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.374+0000 7f93dd400700 1 -- 172.21.15.99:0/37230268 >> 172.21.15.99:0/37230268 conn(0x7f93d80fc1f0 msgr2=0x7f93d80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:47.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.374+0000 7f93dd400700 1 -- 172.21.15.99:0/37230268 shutdown_connections 2024-03-30T00:34:47.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.374+0000 7f93dd400700 1 -- 172.21.15.99:0/37230268 wait complete. 2024-03-30T00:34:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.375+0000 7f93dd400700 1 Processor -- start 2024-03-30T00:34:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.375+0000 7f93dd400700 1 -- start start 2024-03-30T00:34:47.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93dd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100c80 0x7f93d8198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93dd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8101e70 0x7f93d8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93dd400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102810 0x7f93d81a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93dd400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d81a08f0 con 0x7f93d8100c80 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93dd400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d81a0a30 con 0x7f93d8101e70 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93dd400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d81a0bc0 con 0x7f93d8102810 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93d6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100c80 0x7f93d8198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93d6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100c80 0x7f93d8198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56554/0 (socket says 172.21.15.99:56554) 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93d6800700 1 -- 172.21.15.99:0/2392115327 learned_addr learned my addr 172.21.15.99:0/2392115327 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93d7200700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102810 0x7f93d81a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93d5e00700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8101e70 0x7f93d8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93d6800700 1 -- 172.21.15.99:0/2392115327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102810 msgr2=0x7f93d81a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93d6800700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102810 0x7f93d81a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93d6800700 1 -- 172.21.15.99:0/2392115327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8101e70 msgr2=0x7f93d8199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93d6800700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8101e70 0x7f93d8199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93d6800700 1 -- 172.21.15.99:0/2392115327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c80095f0 con 0x7f93d8100c80 2024-03-30T00:34:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.376+0000 7f93d5e00700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8101e70 0x7f93d8199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:34:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.377+0000 7f93d6800700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100c80 0x7f93d8198dc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009fd0 tx=0x7f93c800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.377+0000 7f93cf400700 1 -- 172.21.15.99:0/2392115327 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c801d070 con 0x7f93d8100c80 2024-03-30T00:34:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.377+0000 7f93cf400700 1 -- 172.21.15.99:0/2392115327 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93c800bd10 con 0x7f93d8100c80 2024-03-30T00:34:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.377+0000 7f93cf400700 1 -- 172.21.15.99:0/2392115327 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8004cb0 con 0x7f93d8100c80 2024-03-30T00:34:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.377+0000 7f93dd400700 1 -- 172.21.15.99:0/2392115327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d81a0e40 con 0x7f93d8100c80 2024-03-30T00:34:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.377+0000 7f93dd400700 1 -- 172.21.15.99:0/2392115327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d81a11b0 con 0x7f93d8100c80 2024-03-30T00:34:47.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.378+0000 7f93dd400700 1 -- 172.21.15.99:0/2392115327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d8067a80 con 0x7f93d8100c80 2024-03-30T00:34:47.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.380+0000 7f93cf400700 1 -- 172.21.15.99:0/2392115327 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f93c8004030 con 0x7f93d8100c80 2024-03-30T00:34:47.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.381+0000 7f93cf400700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 0x7f93bc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.381+0000 7f93cf400700 1 -- 172.21.15.99:0/2392115327 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7f93c80cd830 con 0x7f93d8100c80 2024-03-30T00:34:47.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.381+0000 7f93d5e00700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 0x7f93bc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.382+0000 7f93d5e00700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 0x7f93bc0a35e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009be0 tx=0x7f93c4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:47.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.383+0000 7f93cf400700 1 -- 172.21.15.99:0/2392115327 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f93c8096fe0 con 0x7f93d8100c80 2024-03-30T00:34:47.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:47 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3834954953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:47.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:47 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3142348780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:47.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.627+0000 7f93dd400700 1 -- 172.21.15.99:0/2392115327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93d810c390 con 0x7f93d8100c80 2024-03-30T00:34:47.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.629+0000 7f93cf400700 1 -- 172.21.15.99:0/2392115327 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22615 (secure 0 0 0) 0x7f93c809a600 con 0x7f93d8100c80 2024-03-30T00:34:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:34:47.636 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:34:10.248126+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8},{"from":5,"to":7}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:34:47.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.631+0000 7f93dd400700 1 -- 172.21.15.99:0/2392115327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 msgr2=0x7f93bc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.631+0000 7f93dd400700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 0x7f93bc0a35e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009be0 tx=0x7f93c4009260 comp rx=0 tx=0).stop 2024-03-30T00:34:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.631+0000 7f93dd400700 1 -- 172.21.15.99:0/2392115327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100c80 msgr2=0x7f93d8198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.631+0000 7f93dd400700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100c80 0x7f93d8198dc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009fd0 tx=0x7f93c800fab0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.632+0000 7f93dd400700 1 -- 172.21.15.99:0/2392115327 shutdown_connections 2024-03-30T00:34:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.632+0000 7f93dd400700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93d8102810 0x7f93d81a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.632+0000 7f93dd400700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93d8100c80 0x7f93d8198dc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.632+0000 7f93dd400700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f93bc0a1120 0x7f93bc0a35e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.632+0000 7f93dd400700 1 --2- 172.21.15.99:0/2392115327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93d8101e70 0x7f93d8199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.632+0000 7f93dd400700 1 -- 172.21.15.99:0/2392115327 >> 172.21.15.99:0/2392115327 conn(0x7f93d80fc1f0 msgr2=0x7f93d8107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.632+0000 7f93dd400700 1 -- 172.21.15.99:0/2392115327 shutdown_connections 2024-03-30T00:34:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.632+0000 7f93dd400700 1 -- 172.21.15.99:0/2392115327 wait complete. 2024-03-30T00:34:47.657 DEBUG:tasks.fwd_scrub:Yielding 2024-03-30T00:34:47.657 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-30T00:34:47.688 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi099.front.sepia.ceph.com 2024-03-30T00:34:47.689 DEBUG:teuthology.orchestra.run.smithi099:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-30T00:34:47.689 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-30T00:34:47.689 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T00:34:47.689 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T00:34:47.693 INFO:teuthology.run_tasks:Running task workunit... 2024-03-30T00:34:47.706 INFO:tasks.workunit:Pulling workunits from ref 470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:34:47.706 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-30T00:34:47.706 INFO:tasks.workunit:timeout=3h 2024-03-30T00:34:47.706 INFO:tasks.workunit:cleanup=True 2024-03-30T00:34:47.706 DEBUG:teuthology.orchestra.run.smithi099:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-30T00:34:47.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:47 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3834954953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:47.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:47 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3142348780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:47.755 INFO:teuthology.orchestra.run.smithi099.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-30T00:34:47.755 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-30T00:34:47.755 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 7ah/122d Inode: 1 Links: 2 2024-03-30T00:34:47.756 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T00:34:47.756 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-30T00:34:47.756 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-30 00:33:22.094561588 +0000 2024-03-30T00:34:47.756 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-30 00:33:22.094561588 +0000 2024-03-30T00:34:47.756 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-30 00:34:37.902928644 +0000 2024-03-30T00:34:47.756 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2024-03-30T00:34:47.756 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-30T00:34:47.756 DEBUG:teuthology.orchestra.run.smithi099:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-30T00:34:47.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:47 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3834954953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:34:47.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:47 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3142348780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:34:47.876 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T00:34:47.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.900+0000 7fd213800700 1 -- 172.21.15.99:0/3272136850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd20c102910 msgr2=0x7fd20c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.900+0000 7fd213800700 1 --2- 172.21.15.99:0/3272136850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd20c102910 0x7fd20c1091c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd20400b0d0 tx=0x7fd20400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.900+0000 7fd213800700 1 -- 172.21.15.99:0/3272136850 shutdown_connections 2024-03-30T00:34:47.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.900+0000 7fd213800700 1 --2- 172.21.15.99:0/3272136850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd20c102910 0x7fd20c1091c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.900+0000 7fd213800700 1 --2- 172.21.15.99:0/3272136850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd20c100d80 0x7fd20c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.900+0000 7fd213800700 1 --2- 172.21.15.99:0/3272136850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd20c101f70 0x7fd20c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.900+0000 7fd213800700 1 -- 172.21.15.99:0/3272136850 >> 172.21.15.99:0/3272136850 conn(0x7fd20c0fc2f0 msgr2=0x7fd20c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:47.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.900+0000 7fd213800700 1 -- 172.21.15.99:0/3272136850 shutdown_connections 2024-03-30T00:34:47.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.900+0000 7fd213800700 1 -- 172.21.15.99:0/3272136850 wait complete. 2024-03-30T00:34:47.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.901+0000 7fd213800700 1 Processor -- start 2024-03-30T00:34:47.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.901+0000 7fd213800700 1 -- start start 2024-03-30T00:34:47.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.901+0000 7fd213800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd20c100d80 0x7fd20c198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.901+0000 7fd213800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd20c101f70 0x7fd20c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd213800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd20c102910 0x7fd20c1a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd213800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c1a0aa0 con 0x7fd20c102910 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd213800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c1a0be0 con 0x7fd20c101f70 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd213800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20c1a0d70 con 0x7fd20c100d80 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd211800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd20c102910 0x7fd20c1a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd211800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd20c102910 0x7fd20c1a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56572/0 (socket says 172.21.15.99:56572) 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd211800700 1 -- 172.21.15.99:0/3170569084 learned_addr learned my addr 172.21.15.99:0/3170569084 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd210e00700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd20c100d80 0x7fd20c198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd20be00700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd20c101f70 0x7fd20c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd211800700 1 -- 172.21.15.99:0/3170569084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd20c100d80 msgr2=0x7fd20c198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd211800700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd20c100d80 0x7fd20c198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd211800700 1 -- 172.21.15.99:0/3170569084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd20c101f70 msgr2=0x7fd20c1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd211800700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd20c101f70 0x7fd20c1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.902+0000 7fd211800700 1 -- 172.21.15.99:0/3170569084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204009d00 con 0x7fd20c102910 2024-03-30T00:34:47.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.903+0000 7fd211800700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd20c102910 0x7fd20c1a04d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd204007b60 tx=0x7fd204009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:47.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.903+0000 7fd209600700 1 -- 172.21.15.99:0/3170569084 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20400e040 con 0x7fd20c102910 2024-03-30T00:34:47.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.903+0000 7fd209600700 1 -- 172.21.15.99:0/3170569084 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd20401dd20 con 0x7fd20c102910 2024-03-30T00:34:47.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.903+0000 7fd213800700 1 -- 172.21.15.99:0/3170569084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20c1a0ff0 con 0x7fd20c102910 2024-03-30T00:34:47.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.903+0000 7fd209600700 1 -- 172.21.15.99:0/3170569084 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204012dd0 con 0x7fd20c102910 2024-03-30T00:34:47.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.903+0000 7fd213800700 1 -- 172.21.15.99:0/3170569084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20c1a13c0 con 0x7fd20c102910 2024-03-30T00:34:47.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.907+0000 7fd209600700 1 -- 172.21.15.99:0/3170569084 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fd20401d890 con 0x7fd20c102910 2024-03-30T00:34:47.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.908+0000 7fd22726fb80 1 Processor -- start 2024-03-30T00:34:47.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.908+0000 7fd22726fb80 1 -- start start 2024-03-30T00:34:47.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.908+0000 7fd209600700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd1f40a1120 0x7fd1f40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.908+0000 7fd209600700 1 -- 172.21.15.99:0/3170569084 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fd2040ce840 con 0x7fd20c102910 2024-03-30T00:34:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.908+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e910850 0x55a26e910c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.908+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e911a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.908+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e912050 0x55a26e9124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.908+0000 7fd22726fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e868a10 con 0x55a26e9111b0 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.908+0000 7fd22726fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e8663d0 con 0x55a26e912050 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.908+0000 7fd22726fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e861cc0 con 0x55a26e910850 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.908+0000 7fd20be00700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd1f40a1120 0x7fd1f40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd20be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e911a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd20be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e911a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56580/0 (socket says 172.21.15.99:56580) 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd20be00700 1 -- 172.21.15.99:0/3735855941 learned_addr learned my addr 172.21.15.99:0/3735855941 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd210e00700 1 --2- 172.21.15.99:0/3735855941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e910850 0x55a26e910c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd211800700 1 --2- 172.21.15.99:0/3735855941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e912050 0x55a26e9124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd20be00700 1 -- 172.21.15.99:0/3735855941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e910850 msgr2=0x55a26e910c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd20be00700 1 --2- 172.21.15.99:0/3735855941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e910850 0x55a26e910c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd20be00700 1 -- 172.21.15.99:0/3735855941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e912050 msgr2=0x55a26e9124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd20be00700 1 --2- 172.21.15.99:0/3735855941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e912050 0x55a26e9124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd20be00700 1 -- 172.21.15.99:0/3735855941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a26e6e3430 con 0x55a26e9111b0 2024-03-30T00:34:47.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd210e00700 1 --2- 172.21.15.99:0/3735855941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e910850 0x55a26e910c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:34:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd20be00700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd1f40a1120 0x7fd1f40a35e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8012a20 tx=0x7fd1f800bc30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.909+0000 7fd20be00700 1 --2- 172.21.15.99:0/3735855941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e911a70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800bfc0 tx=0x7fd1f801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1653b6554a80cdc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.910+0000 7fd1fe000700 1 -- 172.21.15.99:0/3735855941 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8010e80 con 0x55a26e9111b0 2024-03-30T00:34:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.910+0000 7fd1fe000700 1 -- 172.21.15.99:0/3735855941 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1f80208c0 con 0x55a26e9111b0 2024-03-30T00:34:47.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.910+0000 7fd1fe000700 1 -- 172.21.15.99:0/3735855941 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f801e5f0 con 0x55a26e9111b0 2024-03-30T00:34:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.910+0000 7fd22726fb80 1 -- 172.21.15.99:0/3735855941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 msgr2=0x55a26e911a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.910+0000 7fd22726fb80 1 --2- 172.21.15.99:0/3735855941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e911a70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800bfc0 tx=0x7fd1f801c000 comp rx=0 tx=0).stop 2024-03-30T00:34:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.910+0000 7fd22726fb80 1 -- 172.21.15.99:0/3735855941 shutdown_connections 2024-03-30T00:34:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.910+0000 7fd22726fb80 1 --2- 172.21.15.99:0/3735855941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e912050 0x55a26e9124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.910+0000 7fd22726fb80 1 --2- 172.21.15.99:0/3735855941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e910850 0x55a26e910c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.910+0000 7fd22726fb80 1 --2- 172.21.15.99:0/3735855941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e911a70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.910+0000 7fd22726fb80 1 -- 172.21.15.99:0/3735855941 >> 172.21.15.99:0/3735855941 conn(0x55a26e86f720 msgr2=0x55a26e90ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.911+0000 7fd22726fb80 1 -- 172.21.15.99:0/3735855941 shutdown_connections 2024-03-30T00:34:47.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.911+0000 7fd22726fb80 1 -- 172.21.15.99:0/3735855941 wait complete. 2024-03-30T00:34:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.911+0000 7fd22726fb80 1 -- start start 2024-03-30T00:34:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.911+0000 7fd22726fb80 1 Processor -- start 2024-03-30T00:34:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e910850 0x55a26e91a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e91abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e912050 0x55a26e921c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd22726fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e7fde00 con 0x55a26e9111b0 2024-03-30T00:34:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd22726fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e7fdaa0 con 0x55a26e910850 2024-03-30T00:34:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd22726fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e6e5720 con 0x55a26e912050 2024-03-30T00:34:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd22726fb80 10 client.0 _ll_register_callbacks cb 0x55a26e8074f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd20be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e91abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd20be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e91abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56582/0 (socket says 172.21.15.99:56582) 2024-03-30T00:34:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd20be00700 1 -- 172.21.15.99:0/1373217468 learned_addr learned my addr 172.21.15.99:0/1373217468 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd211800700 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e912050 0x55a26e921c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd210e00700 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e910850 0x55a26e91a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd20be00700 1 -- 172.21.15.99:0/1373217468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e912050 msgr2=0x55a26e921c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd20be00700 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e912050 0x55a26e921c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd20be00700 1 -- 172.21.15.99:0/1373217468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e910850 msgr2=0x55a26e91a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:47.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd20be00700 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e910850 0x55a26e91a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd20be00700 1 -- 172.21.15.99:0/1373217468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20000b8c0 con 0x55a26e9111b0 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.912+0000 7fd210e00700 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e910850 0x55a26e91a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.913+0000 7fd20be00700 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e91abe0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8005950 tx=0x7fd1f801e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68e21536025d306 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.913+0000 7fd1f2a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.913+0000 7fd1f2a00700 1 -- 172.21.15.99:0/1373217468 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8020af0 con 0x55a26e9111b0 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.913+0000 7fd1f2a00700 1 -- 172.21.15.99:0/1373217468 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fd1f8020c50 con 0x55a26e9111b0 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.913+0000 7fd22726fb80 1 -- 172.21.15.99:0/1373217468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a26e822400 con 0x55a26e9111b0 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.913+0000 7fd1f2a00700 1 -- 172.21.15.99:0/1373217468 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1f8020430 con 0x55a26e9111b0 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.913+0000 7fd22726fb80 10 client.15465 fetch_fsmap learned FSMap version 24 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.913+0000 7fd22726fb80 1 -- 172.21.15.99:0/1373217468 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fd1f801a040 con 0x55a26e9111b0 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.914+0000 7fd1f2a00700 1 -- 172.21.15.99:0/1373217468 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a26e822400 con 0x55a26e9111b0 2024-03-30T00:34:47.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.914+0000 7fd1f2a00700 1 -- 172.21.15.99:0/1373217468 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd1f801e8c0 con 0x55a26e9111b0 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.914+0000 7fd22726fb80 10 client.15465 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.914+0000 7fd22726fb80 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.914+0000 7fd22726fb80 20 client.15465 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.914+0000 7fd22726fb80 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a26e926400 0x55a26e928820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.914+0000 7fd22726fb80 4 client.15465 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.914+0000 7fd22726fb80 1 -- 172.21.15.99:0/1373217468 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a26e85a780 con 0x55a26e926400 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.914+0000 7fd210e00700 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a26e926400 0x55a26e928820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.915+0000 7fd210e00700 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a26e926400 0x55a26e928820 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe4fc9fdd79a3c3 server_cookie=1c43ae844dde9e72 in_seq=0 out_seq=0 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.915+0000 7fd1f2a00700 10 client.15465 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd1f2a00700 1 -- 172.21.15.99:0/1373217468 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a26e85a780 con 0x55a26e926400 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd1f2a00700 10 client.15465 handle_command_reply: tid=0 2024-03-30T00:34:47.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd22726fb80 1 client.15465 shutdown 2024-03-30T00:34:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd22726fb80 1 -- 172.21.15.99:0/1373217468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 msgr2=0x55a26e91abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd22726fb80 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e91abe0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8005950 tx=0x7fd1f801e6a0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd22726fb80 1 -- 172.21.15.99:0/1373217468 shutdown_connections 2024-03-30T00:34:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd22726fb80 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a26e926400 0x55a26e928820 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd22726fb80 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e912050 0x55a26e921c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd22726fb80 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e910850 0x55a26e91a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd22726fb80 1 --2- 172.21.15.99:0/1373217468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e9111b0 0x55a26e91abe0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd1f2a00700 0 client.15465 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd22726fb80 1 -- 172.21.15.99:0/1373217468 >> 172.21.15.99:0/1373217468 conn(0x55a26e86f720 msgr2=0x55a26e90f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd22726fb80 1 -- 172.21.15.99:0/1373217468 shutdown_connections 2024-03-30T00:34:47.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.916+0000 7fd22726fb80 1 -- 172.21.15.99:0/1373217468 wait complete. 2024-03-30T00:34:47.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.917+0000 7fd22726fb80 20 client.15465 trim_cache size 0 max 16384 2024-03-30T00:34:47.929 INFO:tasks.workunit.client.0.smithi099.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-30T00:34:47.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.934+0000 7fd22726fb80 1 Processor -- start 2024-03-30T00:34:47.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.934+0000 7fd22726fb80 1 -- start start 2024-03-30T00:34:47.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.934+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e910850 0x55a26e91a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.934+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e9111b0 0x55a26e91abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.934+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e912050 0x55a26e921be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.934+0000 7fd22726fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e90c850 con 0x55a26e910850 2024-03-30T00:34:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.934+0000 7fd22726fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e90c530 con 0x55a26e912050 2024-03-30T00:34:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.934+0000 7fd22726fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e91b5a0 con 0x55a26e9111b0 2024-03-30T00:34:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.935+0000 7fd210e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e910850 0x55a26e91a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.935+0000 7fd210e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e910850 0x55a26e91a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56588/0 (socket says 172.21.15.99:56588) 2024-03-30T00:34:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.935+0000 7fd210e00700 1 -- 172.21.15.99:0/3856739998 learned_addr learned my addr 172.21.15.99:0/3856739998 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.935+0000 7fd20be00700 1 --2- 172.21.15.99:0/3856739998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e9111b0 0x55a26e91abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.935+0000 7fd211800700 1 --2- 172.21.15.99:0/3856739998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e912050 0x55a26e921be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.935+0000 7fd20be00700 1 -- 172.21.15.99:0/3856739998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e912050 msgr2=0x55a26e921be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.935+0000 7fd20be00700 1 --2- 172.21.15.99:0/3856739998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e912050 0x55a26e921be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.935+0000 7fd20be00700 1 -- 172.21.15.99:0/3856739998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e910850 msgr2=0x55a26e91a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:47.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.935+0000 7fd20be00700 1 --2- 172.21.15.99:0/3856739998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e910850 0x55a26e91a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.935+0000 7fd20be00700 1 -- 172.21.15.99:0/3856739998 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a26e68fbe0 con 0x55a26e9111b0 2024-03-30T00:34:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.935+0000 7fd210e00700 1 --2- 172.21.15.99:0/3856739998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e910850 0x55a26e91a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:34:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.936+0000 7fd20be00700 1 --2- 172.21.15.99:0/3856739998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e9111b0 0x55a26e91abc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8016010 tx=0x7fd1f8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17f0249a4e25fe5e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.936+0000 7fd1fea00700 1 -- 172.21.15.99:0/3856739998 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8041070 con 0x55a26e9111b0 2024-03-30T00:34:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.936+0000 7fd1fea00700 1 -- 172.21.15.99:0/3856739998 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1f802d070 con 0x55a26e9111b0 2024-03-30T00:34:47.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.936+0000 7fd1fea00700 1 -- 172.21.15.99:0/3856739998 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8020d50 con 0x55a26e9111b0 2024-03-30T00:34:47.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.937+0000 7fd22726fb80 1 -- 172.21.15.99:0/3856739998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e9111b0 msgr2=0x55a26e91abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.937+0000 7fd22726fb80 1 --2- 172.21.15.99:0/3856739998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e9111b0 0x55a26e91abc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8016010 tx=0x7fd1f8005880 comp rx=0 tx=0).stop 2024-03-30T00:34:47.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.937+0000 7fd22726fb80 1 -- 172.21.15.99:0/3856739998 shutdown_connections 2024-03-30T00:34:47.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.937+0000 7fd22726fb80 1 --2- 172.21.15.99:0/3856739998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e912050 0x55a26e921be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.937+0000 7fd22726fb80 1 --2- 172.21.15.99:0/3856739998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e910850 0x55a26e91a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.937+0000 7fd22726fb80 1 --2- 172.21.15.99:0/3856739998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e9111b0 0x55a26e91abc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.937+0000 7fd22726fb80 1 -- 172.21.15.99:0/3856739998 >> 172.21.15.99:0/3856739998 conn(0x55a26e86f720 msgr2=0x55a26e919380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:47.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.937+0000 7fd22726fb80 1 -- 172.21.15.99:0/3856739998 shutdown_connections 2024-03-30T00:34:47.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.937+0000 7fd22726fb80 1 -- 172.21.15.99:0/3856739998 wait complete. 2024-03-30T00:34:47.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.937+0000 7fd22726fb80 1 -- start start 2024-03-30T00:34:47.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.937+0000 7fd22726fb80 1 Processor -- start 2024-03-30T00:34:47.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e910850 0x55a26e9213c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e9111b0 0x55a26e921900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd22726fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e912050 0x55a26e97a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd22726fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e97ac90 con 0x55a26e912050 2024-03-30T00:34:47.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd22726fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e97ae00 con 0x55a26e910850 2024-03-30T00:34:47.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd22726fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a26e97afc0 con 0x55a26e9111b0 2024-03-30T00:34:47.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd22726fb80 10 client.0 _ll_register_callbacks cb 0x55a26e8074f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:47.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd211800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e912050 0x55a26e97a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd211800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e912050 0x55a26e97a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56602/0 (socket says 172.21.15.99:56602) 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd211800700 1 -- 172.21.15.99:0/2695539907 learned_addr learned my addr 172.21.15.99:0/2695539907 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd210e00700 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e910850 0x55a26e9213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd20be00700 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e9111b0 0x55a26e921900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd211800700 1 -- 172.21.15.99:0/2695539907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e9111b0 msgr2=0x55a26e921900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd211800700 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e9111b0 0x55a26e921900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.938+0000 7fd211800700 1 -- 172.21.15.99:0/2695539907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e910850 msgr2=0x55a26e9213c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.939+0000 7fd211800700 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e910850 0x55a26e9213c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.939+0000 7fd211800700 1 -- 172.21.15.99:0/2695539907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd200004040 con 0x55a26e912050 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.939+0000 7fd211800700 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e912050 0x55a26e97a750 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd2040a3b70 tx=0x7fd204009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b40abad588dc309 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.939+0000 7fd1f2a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:47.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.939+0000 7fd1f2a00700 1 -- 172.21.15.99:0/2695539907 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20409b3a0 con 0x55a26e912050 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.939+0000 7fd1f2a00700 1 -- 172.21.15.99:0/2695539907 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fd20409c3b0 con 0x55a26e912050 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.939+0000 7fd22726fb80 1 -- 172.21.15.99:0/2695539907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1f80208c0 con 0x55a26e912050 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.939+0000 7fd1f2a00700 1 -- 172.21.15.99:0/2695539907 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd20409bcb0 con 0x55a26e912050 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.939+0000 7fd22726fb80 10 client.15471 fetch_fsmap learned FSMap version 24 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.939+0000 7fd22726fb80 1 -- 172.21.15.99:0/2695539907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fd1f801a040 con 0x55a26e912050 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.939+0000 7fd1f2a00700 1 -- 172.21.15.99:0/2695539907 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1f80208c0 con 0x55a26e912050 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.940+0000 7fd1f2a00700 1 -- 172.21.15.99:0/2695539907 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd20402ec00 con 0x55a26e912050 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.940+0000 7fd22726fb80 10 client.15471 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.940+0000 7fd22726fb80 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.940+0000 7fd22726fb80 20 client.15471 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.940+0000 7fd22726fb80 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a26e97c800 0x55a26e97ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:47.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.940+0000 7fd22726fb80 4 client.15471 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T00:34:47.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.940+0000 7fd22726fb80 1 -- 172.21.15.99:0/2695539907 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a26e6b0d70 con 0x55a26e97c800 2024-03-30T00:34:47.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.940+0000 7fd210e00700 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a26e97c800 0x55a26e97ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:47.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.941+0000 7fd210e00700 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a26e97c800 0x55a26e97ec20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61410ac4e551564d server_cookie=e50e9c8830569467 in_seq=0 out_seq=0 2024-03-30T00:34:47.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.941+0000 7fd1f2a00700 10 client.15471 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:47.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd1f2a00700 1 -- 172.21.15.99:0/2695539907 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a26e6b0d70 con 0x55a26e97c800 2024-03-30T00:34:47.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd1f2a00700 10 client.15471 handle_command_reply: tid=0 2024-03-30T00:34:47.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd22726fb80 1 client.15471 shutdown 2024-03-30T00:34:47.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd22726fb80 1 -- 172.21.15.99:0/2695539907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e912050 msgr2=0x55a26e97a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd22726fb80 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e912050 0x55a26e97a750 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd2040a3b70 tx=0x7fd204009040 comp rx=0 tx=0).stop 2024-03-30T00:34:47.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd22726fb80 1 -- 172.21.15.99:0/2695539907 shutdown_connections 2024-03-30T00:34:47.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd22726fb80 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a26e97c800 0x55a26e97ec20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd22726fb80 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a26e912050 0x55a26e97a750 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd22726fb80 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a26e910850 0x55a26e9213c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd22726fb80 1 --2- 172.21.15.99:0/2695539907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a26e9111b0 0x55a26e921900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd22726fb80 1 -- 172.21.15.99:0/2695539907 >> 172.21.15.99:0/2695539907 conn(0x55a26e86f720 msgr2=0x55a26e91a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd1f2a00700 0 client.15471 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd22726fb80 1 -- 172.21.15.99:0/2695539907 shutdown_connections 2024-03-30T00:34:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.942+0000 7fd22726fb80 1 -- 172.21.15.99:0/2695539907 wait complete. 2024-03-30T00:34:47.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.943+0000 7fd22726fb80 20 client.15471 trim_cache size 0 max 16384 2024-03-30T00:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "return_code": 0, 2024-03-30T00:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_tag": "27d04c1a-0a19-49c2-8485-26de02b459fd", 2024-03-30T00:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": "asynchronous" 2024-03-30T00:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:34:47.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.943+0000 7fd213800700 1 -- 172.21.15.99:0/3170569084 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd1f40a1120 msgr2=0x7fd1f40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.943+0000 7fd213800700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd1f40a1120 0x7fd1f40a35e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8012a20 tx=0x7fd1f800bc30 comp rx=0 tx=0).stop 2024-03-30T00:34:47.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.943+0000 7fd213800700 1 -- 172.21.15.99:0/3170569084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd20c102910 msgr2=0x7fd20c1a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:47.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.943+0000 7fd213800700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd20c102910 0x7fd20c1a04d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd204007b60 tx=0x7fd204009440 comp rx=0 tx=0).stop 2024-03-30T00:34:47.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.943+0000 7fd213800700 1 -- 172.21.15.99:0/3170569084 shutdown_connections 2024-03-30T00:34:47.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.943+0000 7fd213800700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd20c102910 0x7fd20c1a04d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.943+0000 7fd213800700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd20c100d80 0x7fd20c198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.943+0000 7fd213800700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd1f40a1120 0x7fd1f40a35e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.943+0000 7fd213800700 1 --2- 172.21.15.99:0/3170569084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd20c101f70 0x7fd20c1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:47.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.943+0000 7fd213800700 1 -- 172.21.15.99:0/3170569084 >> 172.21.15.99:0/3170569084 conn(0x7fd20c0fc2f0 msgr2=0x7fd20c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:47.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.944+0000 7fd213800700 1 -- 172.21.15.99:0/3170569084 shutdown_connections 2024-03-30T00:34:47.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:47.944+0000 7fd213800700 1 -- 172.21.15.99:0/3170569084 wait complete. 2024-03-30T00:34:47.968 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T00:34:48.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.209+0000 7ff5e1800700 1 -- 172.21.15.99:0/2581034663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5dc1041e0 msgr2=0x7ff5dc1065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:48.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.209+0000 7ff5e1800700 1 --2- 172.21.15.99:0/2581034663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5dc1041e0 0x7ff5dc1065d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009990 tx=0x7ff5c8009ca0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.209+0000 7ff5e1800700 1 -- 172.21.15.99:0/2581034663 shutdown_connections 2024-03-30T00:34:48.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.209+0000 7ff5e1800700 1 --2- 172.21.15.99:0/2581034663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5dc106b10 0x7ff5dc108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.209+0000 7ff5e1800700 1 --2- 172.21.15.99:0/2581034663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5dc1018b0 0x7ff5dc103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.209+0000 7ff5e1800700 1 --2- 172.21.15.99:0/2581034663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5dc1041e0 0x7ff5dc1065d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.209+0000 7ff5e1800700 1 -- 172.21.15.99:0/2581034663 >> 172.21.15.99:0/2581034663 conn(0x7ff5dc0fb200 msgr2=0x7ff5dc0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:48.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.209+0000 7ff5e1800700 1 -- 172.21.15.99:0/2581034663 shutdown_connections 2024-03-30T00:34:48.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.209+0000 7ff5e1800700 1 -- 172.21.15.99:0/2581034663 wait complete. 2024-03-30T00:34:48.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.210+0000 7ff5e1800700 1 Processor -- start 2024-03-30T00:34:48.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.210+0000 7ff5e1800700 1 -- start start 2024-03-30T00:34:48.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.210+0000 7ff5e1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5dc1018b0 0x7ff5dc198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.210+0000 7ff5e1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5dc1041e0 0x7ff5dc199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.210+0000 7ff5e1800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5dc106b10 0x7ff5dc1a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.210+0000 7ff5e1800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc1a0960 con 0x7ff5dc1041e0 2024-03-30T00:34:48.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.210+0000 7ff5e1800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc1a0aa0 con 0x7ff5dc1018b0 2024-03-30T00:34:48.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.210+0000 7ff5e1800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc1a0c30 con 0x7ff5dc106b10 2024-03-30T00:34:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.211+0000 7ff5da000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5dc1041e0 0x7ff5dc199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.211+0000 7ff5da000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5dc1041e0 0x7ff5dc199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56620/0 (socket says 172.21.15.99:56620) 2024-03-30T00:34:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.211+0000 7ff5da000700 1 -- 172.21.15.99:0/3433895237 learned_addr learned my addr 172.21.15.99:0/3433895237 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.211+0000 7ff5daa00700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5dc1018b0 0x7ff5dc198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.211+0000 7ff5db400700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5dc106b10 0x7ff5dc1a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.211+0000 7ff5da000700 1 -- 172.21.15.99:0/3433895237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5dc106b10 msgr2=0x7ff5dc1a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.211+0000 7ff5da000700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5dc106b10 0x7ff5dc1a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.211+0000 7ff5da000700 1 -- 172.21.15.99:0/3433895237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5dc1018b0 msgr2=0x7ff5dc198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.211+0000 7ff5da000700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5dc1018b0 0x7ff5dc198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.211+0000 7ff5da000700 1 -- 172.21.15.99:0/3433895237 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80095f0 con 0x7ff5dc1041e0 2024-03-30T00:34:48.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.211+0000 7ff5da000700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5dc1041e0 0x7ff5dc199370 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800b2e0 tx=0x7ff5c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:48.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.212+0000 7ff5d3400700 1 -- 172.21.15.99:0/3433895237 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c801d070 con 0x7ff5dc1041e0 2024-03-30T00:34:48.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.212+0000 7ff5d3400700 1 -- 172.21.15.99:0/3433895237 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5c800beb0 con 0x7ff5dc1041e0 2024-03-30T00:34:48.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.212+0000 7ff5d3400700 1 -- 172.21.15.99:0/3433895237 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8021410 con 0x7ff5dc1041e0 2024-03-30T00:34:48.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.212+0000 7ff5e1800700 1 -- 172.21.15.99:0/3433895237 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5dc1a0eb0 con 0x7ff5dc1041e0 2024-03-30T00:34:48.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.212+0000 7ff5e1800700 1 -- 172.21.15.99:0/3433895237 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc1a1220 con 0x7ff5dc1041e0 2024-03-30T00:34:48.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.215+0000 7ff5d3400700 1 -- 172.21.15.99:0/3433895237 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ff5c800f640 con 0x7ff5dc1041e0 2024-03-30T00:34:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.216+0000 7ff5d3400700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff5c00a1120 0x7ff5c00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.216+0000 7ff5f5412b80 1 Processor -- start 2024-03-30T00:34:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5d3400700 1 -- 172.21.15.99:0/3433895237 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff5c80cdd20 con 0x7ff5dc1041e0 2024-03-30T00:34:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5f5412b80 1 -- start start 2024-03-30T00:34:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5daa00700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff5c00a1120 0x7ff5c00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 0x55900df0abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df0b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df0c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900de62990 con 0x55900df0a7d0 2024-03-30T00:34:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900de60350 con 0x55900df0b130 2024-03-30T00:34:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900de5bc40 con 0x55900df0bfd0 2024-03-30T00:34:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5daa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 0x55900df0abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5daa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 0x55900df0abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56626/0 (socket says 172.21.15.99:56626) 2024-03-30T00:34:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5daa00700 1 -- 172.21.15.99:0/265239467 learned_addr learned my addr 172.21.15.99:0/265239467 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5da000700 1 --2- 172.21.15.99:0/265239467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df0b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.217+0000 7ff5db400700 1 --2- 172.21.15.99:0/265239467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df0c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.218+0000 7ff5da000700 1 -- 172.21.15.99:0/265239467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 msgr2=0x55900df0c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:48.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.218+0000 7ff5da000700 1 --2- 172.21.15.99:0/265239467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df0c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.218+0000 7ff5da000700 1 -- 172.21.15.99:0/265239467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 msgr2=0x55900df0abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.218+0000 7ff5da000700 1 --2- 172.21.15.99:0/265239467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 0x55900df0abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.218+0000 7ff5da000700 1 -- 172.21.15.99:0/265239467 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55900dcdd3b0 con 0x55900df0b130 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.218+0000 7ff5daa00700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff5c00a1120 0x7ff5c00a35e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc013650 tx=0x7ff5cc013a10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.218+0000 7ff5da000700 1 --2- 172.21.15.99:0/265239467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df0b9f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80cb7b0 tx=0x7ff5c8004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e495f1fe4e30aa07 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.218+0000 7ff5c7e00700 1 -- 172.21.15.99:0/265239467 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800f460 con 0x55900df0b130 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.218+0000 7ff5c7e00700 1 -- 172.21.15.99:0/265239467 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5c809b890 con 0x55900df0b130 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.218+0000 7ff5c7e00700 1 -- 172.21.15.99:0/265239467 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80a1070 con 0x55900df0b130 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.219+0000 7ff5f5412b80 1 -- 172.21.15.99:0/265239467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 msgr2=0x55900df0b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.219+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/265239467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df0b9f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80cb7b0 tx=0x7ff5c8004320 comp rx=0 tx=0).stop 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.219+0000 7ff5f5412b80 1 -- 172.21.15.99:0/265239467 shutdown_connections 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.219+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/265239467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df0c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.219+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/265239467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 0x55900df0abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.219+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/265239467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df0b9f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.219+0000 7ff5f5412b80 1 -- 172.21.15.99:0/265239467 >> 172.21.15.99:0/265239467 conn(0x55900de696a0 msgr2=0x55900df059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.219+0000 7ff5f5412b80 1 -- 172.21.15.99:0/265239467 shutdown_connections 2024-03-30T00:34:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.219+0000 7ff5f5412b80 1 -- 172.21.15.99:0/265239467 wait complete. 2024-03-30T00:34:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.220+0000 7ff5f5412b80 1 -- start start 2024-03-30T00:34:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.220+0000 7ff5f5412b80 1 Processor -- start 2024-03-30T00:34:48.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.220+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0a7d0 0x55900df14330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.220+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df14870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.220+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0bfd0 0x55900df1b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.220+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900ddf7d80 con 0x55900df0bfd0 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.220+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900ddf7a20 con 0x55900df0b130 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.220+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900dcdf6a0 con 0x55900df0a7d0 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.220+0000 7ff5f5412b80 10 client.0 _ll_register_callbacks cb 0x55900de01470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5db400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0bfd0 0x55900df1b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5db400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0bfd0 0x55900df1b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56632/0 (socket says 172.21.15.99:56632) 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5da000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df14870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5db400700 1 -- 172.21.15.99:0/2455319572 learned_addr learned my addr 172.21.15.99:0/2455319572 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5daa00700 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0a7d0 0x55900df14330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5db400700 1 -- 172.21.15.99:0/2455319572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0a7d0 msgr2=0x55900df14330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5db400700 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0a7d0 0x55900df14330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5db400700 1 -- 172.21.15.99:0/2455319572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 msgr2=0x55900df14870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5db400700 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df14870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5db400700 1 -- 172.21.15.99:0/2455319572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc1a1220 con 0x55900df0bfd0 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5da000700 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df14870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5db400700 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0bfd0 0x55900df1b860 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4007a80 tx=0x7ff5d4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36eac33981c396c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5c4c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5c4c00700 1 -- 172.21.15.99:0/2455319572 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4012b40 con 0x55900df0bfd0 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5c4c00700 1 -- 172.21.15.99:0/2455319572 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff5d4012ca0 con 0x55900df0bfd0 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.221+0000 7ff5f5412b80 1 -- 172.21.15.99:0/2455319572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55900de1c380 con 0x55900df0bfd0 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.222+0000 7ff5c4c00700 1 -- 172.21.15.99:0/2455319572 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5d4012e80 con 0x55900df0bfd0 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.222+0000 7ff5f5412b80 10 client.15483 fetch_fsmap learned FSMap version 24 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.222+0000 7ff5f5412b80 1 -- 172.21.15.99:0/2455319572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55900dc89b60 con 0x55900df0bfd0 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.222+0000 7ff5c4c00700 1 -- 172.21.15.99:0/2455319572 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55900de1c380 con 0x55900df0bfd0 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.222+0000 7ff5c4c00700 1 -- 172.21.15.99:0/2455319572 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff5d4014040 con 0x55900df0bfd0 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.222+0000 7ff5f5412b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:34:48.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.222+0000 7ff5f5412b80 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.222+0000 7ff5f5412b80 20 client.15483 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.222+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55900df20110 0x55900df22530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.222+0000 7ff5f5412b80 4 client.15483 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.222+0000 7ff5f5412b80 1 -- 172.21.15.99:0/2455319572 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55900de54700 con 0x55900df20110 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.223+0000 7ff5daa00700 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55900df20110 0x55900df22530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.223+0000 7ff5daa00700 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55900df20110 0x55900df22530 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=956212005d91da6b server_cookie=20cbeb8fd47d2aeb in_seq=0 out_seq=0 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.223+0000 7ff5c4c00700 10 client.15483 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5c4c00700 1 -- 172.21.15.99:0/2455319572 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55900de54700 con 0x55900df20110 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5c4c00700 10 client.15483 handle_command_reply: tid=0 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5f5412b80 1 client.15483 shutdown 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5f5412b80 1 -- 172.21.15.99:0/2455319572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0bfd0 msgr2=0x55900df1b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0bfd0 0x55900df1b860 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4007a80 tx=0x7ff5d4007d90 comp rx=0 tx=0).stop 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5f5412b80 1 -- 172.21.15.99:0/2455319572 shutdown_connections 2024-03-30T00:34:48.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55900df20110 0x55900df22530 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0bfd0 0x55900df1b860 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0a7d0 0x55900df14330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5c4c00700 0 client.15483 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:48.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/2455319572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df14870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.224+0000 7ff5f5412b80 1 -- 172.21.15.99:0/2455319572 >> 172.21.15.99:0/2455319572 conn(0x55900de696a0 msgr2=0x55900df09170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:48.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.225+0000 7ff5f5412b80 1 -- 172.21.15.99:0/2455319572 shutdown_connections 2024-03-30T00:34:48.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.225+0000 7ff5f5412b80 1 -- 172.21.15.99:0/2455319572 wait complete. 2024-03-30T00:34:48.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.225+0000 7ff5f5412b80 20 client.15483 trim_cache size 0 max 16384 2024-03-30T00:34:48.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5f5412b80 1 Processor -- start 2024-03-30T00:34:48.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5f5412b80 1 -- start start 2024-03-30T00:34:48.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0a7d0 0x55900df0abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0b130 0x55900df1b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df13840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900df13ea0 con 0x55900df0b130 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900df0a370 con 0x55900df0a7d0 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900df121e0 con 0x55900df0bfd0 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5da000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0b130 0x55900df1b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5da000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0b130 0x55900df1b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56640/0 (socket says 172.21.15.99:56640) 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5da000700 1 -- 172.21.15.99:0/586123977 learned_addr learned my addr 172.21.15.99:0/586123977 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.248+0000 7ff5daa00700 1 --2- 172.21.15.99:0/586123977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0a7d0 0x55900df0abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.249+0000 7ff5db400700 1 --2- 172.21.15.99:0/586123977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df13840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.249+0000 7ff5da000700 1 -- 172.21.15.99:0/586123977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 msgr2=0x55900df13840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.249+0000 7ff5da000700 1 --2- 172.21.15.99:0/586123977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df13840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.249+0000 7ff5da000700 1 -- 172.21.15.99:0/586123977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0a7d0 msgr2=0x55900df0abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.249+0000 7ff5da000700 1 --2- 172.21.15.99:0/586123977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0a7d0 0x55900df0abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.249+0000 7ff5da000700 1 -- 172.21.15.99:0/586123977 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55900de21ea0 con 0x55900df0b130 2024-03-30T00:34:48.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.249+0000 7ff5db400700 1 --2- 172.21.15.99:0/586123977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df13840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:34:48.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.249+0000 7ff5da000700 1 --2- 172.21.15.99:0/586123977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0b130 0x55900df1b820 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff5c809b120 tx=0x7ff5c809ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503086fe831c99eb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:48.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.249+0000 7ff5d0c00700 1 -- 172.21.15.99:0/586123977 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800f460 con 0x55900df0b130 2024-03-30T00:34:48.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.249+0000 7ff5d0c00700 1 -- 172.21.15.99:0/586123977 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5c809b190 con 0x55900df0b130 2024-03-30T00:34:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.249+0000 7ff5d0c00700 1 -- 172.21.15.99:0/586123977 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c809a8c0 con 0x55900df0b130 2024-03-30T00:34:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.250+0000 7ff5f5412b80 1 -- 172.21.15.99:0/586123977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0b130 msgr2=0x55900df1b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.250+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/586123977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0b130 0x55900df1b820 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff5c809b120 tx=0x7ff5c809ba00 comp rx=0 tx=0).stop 2024-03-30T00:34:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.250+0000 7ff5f5412b80 1 -- 172.21.15.99:0/586123977 shutdown_connections 2024-03-30T00:34:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.250+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/586123977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df13840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.250+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/586123977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0a7d0 0x55900df0abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.250+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/586123977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0b130 0x55900df1b820 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.250+0000 7ff5f5412b80 1 -- 172.21.15.99:0/586123977 >> 172.21.15.99:0/586123977 conn(0x55900de696a0 msgr2=0x55900defe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.250+0000 7ff5f5412b80 1 -- 172.21.15.99:0/586123977 shutdown_connections 2024-03-30T00:34:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.250+0000 7ff5f5412b80 1 -- 172.21.15.99:0/586123977 wait complete. 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.250+0000 7ff5f5412b80 1 -- start start 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5f5412b80 1 Processor -- start 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 0x55900df197c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df19d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5f5412b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df1ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900df1b160 con 0x55900df0a7d0 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900df1b2d0 con 0x55900df0b130 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5f5412b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55900df1b490 con 0x55900df0bfd0 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5f5412b80 10 client.0 _ll_register_callbacks cb 0x55900de01470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5daa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 0x55900df197c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5daa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 0x55900df197c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56644/0 (socket says 172.21.15.99:56644) 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5daa00700 1 -- 172.21.15.99:0/3670677986 learned_addr learned my addr 172.21.15.99:0/3670677986 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:34:48.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5da000700 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df19d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.251+0000 7ff5db400700 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df1ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5daa00700 1 -- 172.21.15.99:0/3670677986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 msgr2=0x55900df1ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5daa00700 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df1ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5daa00700 1 -- 172.21.15.99:0/3670677986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 msgr2=0x55900df19d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5daa00700 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df19d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5daa00700 1 -- 172.21.15.99:0/3670677986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55900dcdd3b0 con 0x55900df0a7d0 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5daa00700 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 0x55900df197c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55900df22e10 tx=0x7ff5cc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68efc2f70650346 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5c4c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5c4c00700 1 -- 172.21.15.99:0/3670677986 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc018040 con 0x55900df0a7d0 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5c4c00700 1 -- 172.21.15.99:0/3670677986 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7ff5cc02bc90 con 0x55900df0a7d0 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5f5412b80 1 -- 172.21.15.99:0/3670677986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5c809b890 con 0x55900df0a7d0 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5c4c00700 1 -- 172.21.15.99:0/3670677986 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5cc02be50 con 0x55900df0a7d0 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5f5412b80 10 client.15489 fetch_fsmap learned FSMap version 24 2024-03-30T00:34:48.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.252+0000 7ff5f5412b80 1 -- 172.21.15.99:0/3670677986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55900dce1cd0 con 0x55900df0a7d0 2024-03-30T00:34:48.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.253+0000 7ff5c4c00700 1 -- 172.21.15.99:0/3670677986 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5c809b890 con 0x55900df0a7d0 2024-03-30T00:34:48.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.253+0000 7ff5c4c00700 1 -- 172.21.15.99:0/3670677986 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff5cc005020 con 0x55900df0a7d0 2024-03-30T00:34:48.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.253+0000 7ff5f5412b80 10 client.15489 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:34:48.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.253+0000 7ff5f5412b80 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.253+0000 7ff5f5412b80 20 client.15489 populate_metadata read hostname 'smithi099' 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.253+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55900df75000 0x55900df77420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.253+0000 7ff5f5412b80 4 client.15489 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.253+0000 7ff5f5412b80 1 -- 172.21.15.99:0/3670677986 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55900dcaacf0 con 0x55900df75000 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.253+0000 7ff5da000700 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55900df75000 0x55900df77420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5da000700 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55900df75000 0x55900df77420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4271ce407171a940 server_cookie=3c3343fba29c5be5 in_seq=0 out_seq=0 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5c4c00700 10 client.15489 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5c4c00700 1 -- 172.21.15.99:0/3670677986 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55900dcaacf0 con 0x55900df75000 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5c4c00700 10 client.15489 handle_command_reply: tid=0 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5f5412b80 1 client.15489 shutdown 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5f5412b80 1 -- 172.21.15.99:0/3670677986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 msgr2=0x55900df197c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 0x55900df197c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55900df22e10 tx=0x7ff5cc01b000 comp rx=0 tx=0).stop 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5f5412b80 1 -- 172.21.15.99:0/3670677986 shutdown_connections 2024-03-30T00:34:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55900df75000 0x55900df77420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55900df0bfd0 0x55900df1ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55900df0a7d0 0x55900df197c0 secure :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55900df22e10 tx=0x7ff5cc01b000 comp rx=0 tx=0).stop 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5c4c00700 0 client.15489 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5f5412b80 1 --2- 172.21.15.99:0/3670677986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900df0b130 0x55900df19d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5f5412b80 1 -- 172.21.15.99:0/3670677986 >> 172.21.15.99:0/3670677986 conn(0x55900de696a0 msgr2=0x55900df124e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5f5412b80 1 -- 172.21.15.99:0/3670677986 shutdown_connections 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.254+0000 7ff5f5412b80 1 -- 172.21.15.99:0/3670677986 wait complete. 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5f5412b80 20 client.15489 trim_cache size 0 max 16384 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": { 2024-03-30T00:34:48.268 INFO:teuthology.orchestra.run.smithi099.stdout: "27d04c1a-0a19-49c2-8485-26de02b459fd": { 2024-03-30T00:34:48.269 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/", 2024-03-30T00:34:48.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tag": "27d04c1a-0a19-49c2-8485-26de02b459fd", 2024-03-30T00:34:48.269 INFO:teuthology.orchestra.run.smithi099.stdout: "options": "recursive,force" 2024-03-30T00:34:48.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:34:48.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:34:48.269 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:34:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 -- 172.21.15.99:0/3433895237 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff5c00a1120 msgr2=0x7ff5c00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff5c00a1120 0x7ff5c00a35e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc013650 tx=0x7ff5cc013a10 comp rx=0 tx=0).stop 2024-03-30T00:34:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 -- 172.21.15.99:0/3433895237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5dc1041e0 msgr2=0x7ff5dc199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:34:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5dc1041e0 0x7ff5dc199370 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800b2e0 tx=0x7ff5c8004730 comp rx=0 tx=0).stop 2024-03-30T00:34:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 -- 172.21.15.99:0/3433895237 shutdown_connections 2024-03-30T00:34:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5dc106b10 0x7ff5dc1a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff5c00a1120 0x7ff5c00a35e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5dc1018b0 0x7ff5dc198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 --2- 172.21.15.99:0/3433895237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5dc1041e0 0x7ff5dc199370 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:34:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 -- 172.21.15.99:0/3433895237 >> 172.21.15.99:0/3433895237 conn(0x7ff5dc0fb200 msgr2=0x7ff5dc100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:34:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 -- 172.21.15.99:0/3433895237 shutdown_connections 2024-03-30T00:34:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:34:48.255+0000 7ff5e1800700 1 -- 172.21.15.99:0/3433895237 wait complete. 2024-03-30T00:34:48.281 INFO:tasks.cephfs.filesystem:scrub status for tag:27d04c1a-0a19-49c2-8485-26de02b459fd - {'path': '/', 'tag': '27d04c1a-0a19-49c2-8485-26de02b459fd', 'options': 'recursive,force'} 2024-03-30T00:34:48.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:48 smithi161 ceph-mon[42221]: pgmap v304: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:48.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:48 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2392115327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:48.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:48 smithi099 ceph-mon[38079]: pgmap v304: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:48.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:48 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2392115327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:48.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:48 smithi142 ceph-mon[41779]: pgmap v304: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:48.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:48 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2392115327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:34:49.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:49 smithi161 ceph-mon[42221]: scrub queued for path: / 2024-03-30T00:34:49.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:49 smithi161 ceph-mon[42221]: scrub summary: idle+waiting paths [/] 2024-03-30T00:34:49.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:49 smithi161 ceph-mon[42221]: scrub summary: active paths [/] 2024-03-30T00:34:49.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:49 smithi161 ceph-mon[42221]: scrub summary: idle+waiting paths [/] 2024-03-30T00:34:49.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:49 smithi099 ceph-mon[38079]: scrub queued for path: / 2024-03-30T00:34:49.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:49 smithi099 ceph-mon[38079]: scrub summary: idle+waiting paths [/] 2024-03-30T00:34:49.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:49 smithi099 ceph-mon[38079]: scrub summary: active paths [/] 2024-03-30T00:34:49.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:49 smithi099 ceph-mon[38079]: scrub summary: idle+waiting paths [/] 2024-03-30T00:34:49.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:49 smithi142 ceph-mon[41779]: scrub queued for path: / 2024-03-30T00:34:49.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:49 smithi142 ceph-mon[41779]: scrub summary: idle+waiting paths [/] 2024-03-30T00:34:49.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:49 smithi142 ceph-mon[41779]: scrub summary: active paths [/] 2024-03-30T00:34:49.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:49 smithi142 ceph-mon[41779]: scrub summary: idle+waiting paths [/] 2024-03-30T00:34:50.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:50 smithi161 ceph-mon[42221]: pgmap v305: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-30T00:34:50.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:50 smithi099 ceph-mon[38079]: pgmap v305: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-30T00:34:50.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:50 smithi142 ceph-mon[41779]: pgmap v305: 97 pgs: 97 active+clean; 649 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-30T00:34:52.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:52 smithi161 ceph-mon[42221]: pgmap v306: 97 pgs: 97 active+clean; 654 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-30T00:34:52.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:52 smithi099 ceph-mon[38079]: pgmap v306: 97 pgs: 97 active+clean; 654 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-30T00:34:52.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:52 smithi142 ceph-mon[41779]: pgmap v306: 97 pgs: 97 active+clean; 654 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-30T00:34:54.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:54 smithi161 ceph-mon[42221]: pgmap v307: 97 pgs: 97 active+clean; 654 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-30T00:34:54.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:54 smithi099 ceph-mon[38079]: pgmap v307: 97 pgs: 97 active+clean; 654 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-30T00:34:54.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:54 smithi142 ceph-mon[41779]: pgmap v307: 97 pgs: 97 active+clean; 654 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-30T00:34:56.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:56 smithi161 ceph-mon[42221]: pgmap v308: 97 pgs: 97 active+clean; 654 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-30T00:34:56.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:56 smithi099 ceph-mon[38079]: pgmap v308: 97 pgs: 97 active+clean; 654 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-30T00:34:56.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:56 smithi142 ceph-mon[41779]: pgmap v308: 97 pgs: 97 active+clean; 654 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-30T00:34:58.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:58 smithi161 ceph-mon[42221]: pgmap v309: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:58.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:58 smithi099 ceph-mon[38079]: pgmap v309: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:58.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:58 smithi142 ceph-mon[41779]: pgmap v309: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:34:59.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:34:59 smithi161 ceph-mon[42221]: scrub summary: idle 2024-03-30T00:34:59.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:34:59 smithi099 ceph-mon[38079]: scrub summary: idle 2024-03-30T00:34:59.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:34:59 smithi142 ceph-mon[41779]: scrub summary: idle 2024-03-30T00:35:00.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:00 smithi161 ceph-mon[42221]: pgmap v310: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-30T00:35:00.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:00 smithi099 ceph-mon[38079]: pgmap v310: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-30T00:35:00.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:00 smithi142 ceph-mon[41779]: pgmap v310: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-30T00:35:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.927+0000 7fbadf800700 1 -- 172.21.15.99:0/3252250265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8074df0 msgr2=0x7fbad8073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.927+0000 7fbadf800700 1 --2- 172.21.15.99:0/3252250265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8074df0 0x7fbad8073240 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009990 tx=0x7fbacc009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.927+0000 7fbadf800700 1 -- 172.21.15.99:0/3252250265 shutdown_connections 2024-03-30T00:35:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.927+0000 7fbadf800700 1 --2- 172.21.15.99:0/3252250265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad8108bd0 0x7fbad810b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.927+0000 7fbadf800700 1 --2- 172.21.15.99:0/3252250265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8074df0 0x7fbad8073240 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.927+0000 7fbadf800700 1 --2- 172.21.15.99:0/3252250265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbad8073810 0x7fbad8073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.927+0000 7fbadf800700 1 -- 172.21.15.99:0/3252250265 >> 172.21.15.99:0/3252250265 conn(0x7fbad80fc280 msgr2=0x7fbad80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.927+0000 7fbadf800700 1 -- 172.21.15.99:0/3252250265 shutdown_connections 2024-03-30T00:35:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.927+0000 7fbadf800700 1 -- 172.21.15.99:0/3252250265 wait complete. 2024-03-30T00:35:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.928+0000 7fbadf800700 1 Processor -- start 2024-03-30T00:35:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.928+0000 7fbadf800700 1 -- start start 2024-03-30T00:35:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbadf800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad8073810 0x7fbad8198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbadf800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8074df0 0x7fbad8199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbadf800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbad8108bd0 0x7fbad81a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbadf800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad81a08e0 con 0x7fbad8074df0 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbadf800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad81a0a20 con 0x7fbad8108bd0 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbadf800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad81a0bb0 con 0x7fbad8073810 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbad7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8074df0 0x7fbad8199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbad7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8074df0 0x7fbad8199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58114/0 (socket says 172.21.15.99:58114) 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbad7e00700 1 -- 172.21.15.99:0/546581928 learned_addr learned my addr 172.21.15.99:0/546581928 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbadce00700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad8073810 0x7fbad8198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbadd800700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbad8108bd0 0x7fbad81a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbad7e00700 1 -- 172.21.15.99:0/546581928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad8073810 msgr2=0x7fbad8198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbad7e00700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad8073810 0x7fbad8198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbad7e00700 1 -- 172.21.15.99:0/546581928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbad8108bd0 msgr2=0x7fbad81a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbad7e00700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbad8108bd0 0x7fbad81a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.929+0000 7fbad7e00700 1 -- 172.21.15.99:0/546581928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc0095f0 con 0x7fbad8074df0 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.930+0000 7fbad7e00700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8074df0 0x7fbad8199340 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbac400d980 tx=0x7fbac400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.930+0000 7fbad5600700 1 -- 172.21.15.99:0/546581928 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4004020 con 0x7fbad8074df0 2024-03-30T00:35:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.930+0000 7fbad5600700 1 -- 172.21.15.99:0/546581928 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbac400f690 con 0x7fbad8074df0 2024-03-30T00:35:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.930+0000 7fbad5600700 1 -- 172.21.15.99:0/546581928 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac40039e0 con 0x7fbad8074df0 2024-03-30T00:35:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.930+0000 7fbadf800700 1 -- 172.21.15.99:0/546581928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad81a0e90 con 0x7fbad8074df0 2024-03-30T00:35:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.930+0000 7fbadf800700 1 -- 172.21.15.99:0/546581928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad81a1310 con 0x7fbad8074df0 2024-03-30T00:35:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.933+0000 7fbad5600700 1 -- 172.21.15.99:0/546581928 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fbac400f800 con 0x7fbad8074df0 2024-03-30T00:35:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.934+0000 7fbaf32f9b80 1 Processor -- start 2024-03-30T00:35:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.934+0000 7fbad5600700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbac00a1220 0x7fbac00a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbad5600700 1 -- 172.21.15.99:0/546581928 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fbac40cc480 con 0x7fbad8074df0 2024-03-30T00:35:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbaf32f9b80 1 -- start start 2024-03-30T00:35:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbadd800700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbac00a1220 0x7fbac00a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b14310620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b143114d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14311a60 0x563b14311ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b14268930 con 0x563b14310b60 2024-03-30T00:35:00.944 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b142662f0 con 0x563b14311a60 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b14261be0 con 0x563b14310200 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbad7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b143114d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbad7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b143114d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58128/0 (socket says 172.21.15.99:58128) 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.935+0000 7fbad7e00700 1 -- 172.21.15.99:0/4260732650 learned_addr learned my addr 172.21.15.99:0/4260732650 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbadd800700 1 --2- 172.21.15.99:0/4260732650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14311a60 0x563b14311ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbadce00700 1 --2- 172.21.15.99:0/4260732650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b14310620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbad7e00700 1 -- 172.21.15.99:0/4260732650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 msgr2=0x563b14310620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbad7e00700 1 --2- 172.21.15.99:0/4260732650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b14310620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbad7e00700 1 -- 172.21.15.99:0/4260732650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14311a60 msgr2=0x563b14311ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbad7e00700 1 --2- 172.21.15.99:0/4260732650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14311a60 0x563b14311ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbad7e00700 1 -- 172.21.15.99:0/4260732650 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b140e3350 con 0x563b14310b60 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbadd800700 1 --2- 172.21.15.99:0/4260732650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14311a60 0x563b14311ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbadd800700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbac00a1220 0x7fbac00a36e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbad000f540 tx=0x7fbad0009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbad7e00700 1 --2- 172.21.15.99:0/4260732650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b143114d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbac40a17b0 tx=0x7fbac4099470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8a789385bb0726 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbaca000700 1 -- 172.21.15.99:0/4260732650 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4099f60 con 0x563b14310b60 2024-03-30T00:35:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbaca000700 1 -- 172.21.15.99:0/4260732650 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbac4099860 con 0x563b14310b60 2024-03-30T00:35:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.936+0000 7fbaca000700 1 -- 172.21.15.99:0/4260732650 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac40b92e0 con 0x563b14310b60 2024-03-30T00:35:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.937+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4260732650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 msgr2=0x563b143114d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.937+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/4260732650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b143114d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbac40a17b0 tx=0x7fbac4099470 comp rx=0 tx=0).stop 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.937+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4260732650 shutdown_connections 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.937+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/4260732650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14311a60 0x563b14311ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.937+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/4260732650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b14310620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.937+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/4260732650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b143114d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.937+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4260732650 >> 172.21.15.99:0/4260732650 conn(0x563b1426f640 msgr2=0x563b1430b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.937+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4260732650 shutdown_connections 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.937+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4260732650 wait complete. 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.938+0000 7fbaf32f9b80 1 -- start start 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.938+0000 7fbaf32f9b80 1 Processor -- start 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.938+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310200 0x563b1431a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.938+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14310b60 0x563b1431a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.938+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14311a60 0x563b14321770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.938+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b141fdd20 con 0x563b14310200 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.938+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b141fd9c0 con 0x563b14310b60 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.938+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b140e5640 con 0x563b14311a60 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.938+0000 7fbaf32f9b80 10 client.0 _ll_register_callbacks cb 0x563b14207410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.938+0000 7fbadce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310200 0x563b1431a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.938+0000 7fbadce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310200 0x563b1431a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58132/0 (socket says 172.21.15.99:58132) 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.939+0000 7fbadce00700 1 -- 172.21.15.99:0/346307422 learned_addr learned my addr 172.21.15.99:0/346307422 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.939+0000 7fbad7e00700 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14310b60 0x563b1431a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.939+0000 7fbadd800700 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14311a60 0x563b14321770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.939+0000 7fbadce00700 1 -- 172.21.15.99:0/346307422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14311a60 msgr2=0x563b14321770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.939+0000 7fbadce00700 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14311a60 0x563b14321770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.939+0000 7fbadce00700 1 -- 172.21.15.99:0/346307422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14310b60 msgr2=0x563b1431a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.939+0000 7fbadce00700 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14310b60 0x563b1431a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.939+0000 7fbadce00700 1 -- 172.21.15.99:0/346307422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad81a1310 con 0x563b14310200 2024-03-30T00:35:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.939+0000 7fbadd800700 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14311a60 0x563b14321770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.939+0000 7fbadce00700 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310200 0x563b1431a1f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009990 tx=0x7fbacc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581894fe8b5aae07 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.939+0000 7fbabea00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbabea00700 1 -- 172.21.15.99:0/346307422 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc01d030 con 0x563b14310200 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/346307422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b14222320 con 0x563b14310200 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbabea00700 1 -- 172.21.15.99:0/346307422 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fbacc003a90 con 0x563b14310200 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbabea00700 1 -- 172.21.15.99:0/346307422 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbacc005960 con 0x563b14310200 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbaf32f9b80 10 client.15501 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/346307422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563b1408fb00 con 0x563b14310200 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbabea00700 1 -- 172.21.15.99:0/346307422 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbacc033080 con 0x563b14310200 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbabea00700 1 -- 172.21.15.99:0/346307422 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbacc004690 con 0x563b14310200 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbaf32f9b80 10 client.15501 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbaf32f9b80 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbaf32f9b80 20 client.15501 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563b14325fd0 0x563b143283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbaf32f9b80 4 client.15501 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.940+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/346307422 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b1425a6a0 con 0x563b14325fd0 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.941+0000 7fbad7e00700 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563b14325fd0 0x563b143283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.941+0000 7fbad7e00700 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563b14325fd0 0x563b143283f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c61e2216b04399a server_cookie=f21ff864596018e in_seq=0 out_seq=0 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.941+0000 7fbabea00700 10 client.15501 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbabea00700 1 -- 172.21.15.99:0/346307422 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563b1425a6a0 con 0x563b14325fd0 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbabea00700 10 client.15501 handle_command_reply: tid=0 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbaf32f9b80 1 client.15501 shutdown 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/346307422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310200 msgr2=0x563b1431a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310200 0x563b1431a1f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009990 tx=0x7fbacc005270 comp rx=0 tx=0).stop 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/346307422 shutdown_connections 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563b14325fd0 0x563b143283f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14311a60 0x563b14321770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310200 0x563b1431a1f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbabea00700 0 client.15501 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/346307422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14310b60 0x563b1431a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.942+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/346307422 >> 172.21.15.99:0/346307422 conn(0x563b1426f640 msgr2=0x563b1430efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.943+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/346307422 shutdown_connections 2024-03-30T00:35:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.943+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/346307422 wait complete. 2024-03-30T00:35:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.943+0000 7fbaf32f9b80 20 client.15501 trim_cache size 0 max 16384 2024-03-30T00:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbaf32f9b80 1 Processor -- start 2024-03-30T00:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbaf32f9b80 1 -- start start 2024-03-30T00:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b143283c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b14321730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14311a60 0x563b14311e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b143228f0 con 0x563b14310b60 2024-03-30T00:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b143109a0 con 0x563b14311a60 2024-03-30T00:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b14307400 con 0x563b14310200 2024-03-30T00:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbad7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b14321730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbad7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b14321730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58134/0 (socket says 172.21.15.99:58134) 2024-03-30T00:35:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbad7e00700 1 -- 172.21.15.99:0/812593730 learned_addr learned my addr 172.21.15.99:0/812593730 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.958+0000 7fbadce00700 1 --2- 172.21.15.99:0/812593730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b143283c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.959+0000 7fbad7e00700 1 -- 172.21.15.99:0/812593730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 msgr2=0x563b143283c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.959+0000 7fbad7e00700 1 --2- 172.21.15.99:0/812593730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b143283c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.959+0000 7fbad7e00700 1 -- 172.21.15.99:0/812593730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14311a60 msgr2=0x563b14311e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.959+0000 7fbadd800700 1 --2- 172.21.15.99:0/812593730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14311a60 0x563b14311e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.959+0000 7fbad7e00700 1 --2- 172.21.15.99:0/812593730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14311a60 0x563b14311e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.959+0000 7fbad7e00700 1 -- 172.21.15.99:0/812593730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b14227e40 con 0x563b14310b60 2024-03-30T00:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.959+0000 7fbad7e00700 1 --2- 172.21.15.99:0/812593730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b14321730 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbac4010040 tx=0x7fbac4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9e8b6b9d2df5cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.959+0000 7fbacaa00700 1 -- 172.21.15.99:0/812593730 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4099f60 con 0x563b14310b60 2024-03-30T00:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.959+0000 7fbacaa00700 1 -- 172.21.15.99:0/812593730 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbac40b2230 con 0x563b14310b60 2024-03-30T00:35:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.959+0000 7fbacaa00700 1 -- 172.21.15.99:0/812593730 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4099ac0 con 0x563b14310b60 2024-03-30T00:35:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.960+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/812593730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 msgr2=0x563b14321730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.960+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/812593730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b14321730 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbac4010040 tx=0x7fbac4000bc0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.960+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/812593730 shutdown_connections 2024-03-30T00:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.960+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/812593730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14311a60 0x563b14311e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.960+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/812593730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b143283c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.960+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/812593730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14310b60 0x563b14321730 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.960+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/812593730 >> 172.21.15.99:0/812593730 conn(0x563b1426f640 msgr2=0x563b14318010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.960+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/812593730 shutdown_connections 2024-03-30T00:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.960+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/812593730 wait complete. 2024-03-30T00:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.960+0000 7fbaf32f9b80 1 -- start start 2024-03-30T00:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.961+0000 7fbaf32f9b80 1 Processor -- start 2024-03-30T00:35:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.961+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b1431f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.961+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14310b60 0x563b1431f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.961+0000 7fbaf32f9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14311a60 0x563b14320620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.961+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b14320b60 con 0x563b14311a60 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.961+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b14320cd0 con 0x563b14310b60 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.961+0000 7fbaf32f9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b14320e90 con 0x563b14310200 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.961+0000 7fbaf32f9b80 10 client.0 _ll_register_callbacks cb 0x563b14207410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbadd800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14311a60 0x563b14320620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbadd800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14311a60 0x563b14320620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58148/0 (socket says 172.21.15.99:58148) 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbadce00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b1431f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbadd800700 1 -- 172.21.15.99:0/4027916976 learned_addr learned my addr 172.21.15.99:0/4027916976 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbad7e00700 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14310b60 0x563b1431f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbadd800700 1 -- 172.21.15.99:0/4027916976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 msgr2=0x563b1431f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbadd800700 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b1431f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbadd800700 1 -- 172.21.15.99:0/4027916976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14310b60 msgr2=0x563b1431f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbadd800700 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14310b60 0x563b1431f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbadd800700 1 -- 172.21.15.99:0/4027916976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b140e3350 con 0x563b14311a60 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbadce00700 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b1431f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbad7e00700 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14310b60 0x563b1431f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbadd800700 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14311a60 0x563b14320620 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbad0019eb0 tx=0x7fbad0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e307ef7eb524b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.962+0000 7fbabea00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbabea00700 1 -- 172.21.15.99:0/4027916976 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad000e040 con 0x563b14311a60 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbabea00700 1 -- 172.21.15.99:0/4027916976 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8248+0+0 (secure 0 0 0) 0x7fbad000fe40 con 0x563b14311a60 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4027916976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b140b0c90 con 0x563b14311a60 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbabea00700 1 -- 172.21.15.99:0/4027916976 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbad0017dc0 con 0x563b14311a60 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbaf32f9b80 10 client.15507 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4027916976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563b140e7c70 con 0x563b14311a60 2024-03-30T00:35:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbabea00700 1 -- 172.21.15.99:0/4027916976 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563b140b0c90 con 0x563b14311a60 2024-03-30T00:35:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbabea00700 1 -- 172.21.15.99:0/4027916976 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbad0005810 con 0x563b14311a60 2024-03-30T00:35:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbaf32f9b80 10 client.15507 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbaf32f9b80 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbaf32f9b80 20 client.15507 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563b1437aa20 0x563b1437ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbaf32f9b80 4 client.15507 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:35:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.963+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4027916976 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563b140e5c80 con 0x563b1437aa20 2024-03-30T00:35:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.964+0000 7fbadce00700 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563b1437aa20 0x563b1437ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.964+0000 7fbabea00700 10 client.15507 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.964+0000 7fbadce00700 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563b1437aa20 0x563b1437ce40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af64ba90be98a05e server_cookie=89d433feedf4dda2 in_seq=0 out_seq=0 2024-03-30T00:35:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbabea00700 1 -- 172.21.15.99:0/4027916976 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563b140e5c80 con 0x563b1437aa20 2024-03-30T00:35:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbabea00700 10 client.15507 handle_command_reply: tid=0 2024-03-30T00:35:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbaf32f9b80 1 client.15507 shutdown 2024-03-30T00:35:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4027916976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14311a60 msgr2=0x563b14320620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14311a60 0x563b14320620 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbad0019eb0 tx=0x7fbad0004b40 comp rx=0 tx=0).stop 2024-03-30T00:35:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4027916976 shutdown_connections 2024-03-30T00:35:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563b1437aa20 0x563b1437ce40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b14311a60 0x563b14320620 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563b14310200 0x563b1431f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbabea00700 0 client.15507 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbaf32f9b80 1 --2- 172.21.15.99:0/4027916976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b14310b60 0x563b1431f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.977+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4027916976 >> 172.21.15.99:0/4027916976 conn(0x563b1426f640 msgr2=0x563b143193d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4027916976 shutdown_connections 2024-03-30T00:35:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbaf32f9b80 1 -- 172.21.15.99:0/4027916976 wait complete. 2024-03-30T00:35:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbaf32f9b80 20 client.15507 trim_cache size 0 max 16384 2024-03-30T00:35:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:35:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:35:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:35:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:35:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbadf800700 1 -- 172.21.15.99:0/546581928 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbac00a1220 msgr2=0x7fbac00a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbadf800700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbac00a1220 0x7fbac00a36e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbad000f540 tx=0x7fbad0009920 comp rx=0 tx=0).stop 2024-03-30T00:35:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbadf800700 1 -- 172.21.15.99:0/546581928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8074df0 msgr2=0x7fbad8199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbadf800700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8074df0 0x7fbad8199340 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbac400d980 tx=0x7fbac400dc90 comp rx=0 tx=0).stop 2024-03-30T00:35:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbadf800700 1 -- 172.21.15.99:0/546581928 shutdown_connections 2024-03-30T00:35:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbadf800700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbac00a1220 0x7fbac00a36e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbadf800700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbad8108bd0 0x7fbad81a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbadf800700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbad8073810 0x7fbad8198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbadf800700 1 --2- 172.21.15.99:0/546581928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbad8074df0 0x7fbad8199340 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.978+0000 7fbadf800700 1 -- 172.21.15.99:0/546581928 >> 172.21.15.99:0/546581928 conn(0x7fbad80fc280 msgr2=0x7fbad8104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.979+0000 7fbadf800700 1 -- 172.21.15.99:0/546581928 shutdown_connections 2024-03-30T00:35:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:00.979+0000 7fbadf800700 1 -- 172.21.15.99:0/546581928 wait complete. 2024-03-30T00:35:02.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:02 smithi161 ceph-mon[42221]: pgmap v311: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:35:02.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:02 smithi099 ceph-mon[38079]: pgmap v311: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:35:02.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:02 smithi142 ceph-mon[41779]: pgmap v311: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-30T00:35:04.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:04 smithi099 ceph-mon[38079]: pgmap v312: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-03-30T00:35:04.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:04 smithi142 ceph-mon[41779]: pgmap v312: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-03-30T00:35:04.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:04 smithi161 ceph-mon[42221]: pgmap v312: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-03-30T00:35:06.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:06 smithi099 ceph-mon[38079]: pgmap v313: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-30T00:35:06.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:06 smithi142 ceph-mon[41779]: pgmap v313: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-30T00:35:06.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:06 smithi161 ceph-mon[42221]: pgmap v313: 97 pgs: 97 active+clean; 659 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-30T00:35:08.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:08 smithi099 ceph-mon[38079]: pgmap v314: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-30T00:35:08.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:08 smithi142 ceph-mon[41779]: pgmap v314: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-30T00:35:08.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:08 smithi161 ceph-mon[42221]: pgmap v314: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-30T00:35:09.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-30T00:35:09.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-30T00:35:09.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-30T00:35:09.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:09 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-30T00:35:09.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-30T00:35:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-30T00:35:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-30T00:35:09.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:09 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-30T00:35:09.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-30T00:35:09.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-30T00:35:09.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-03-30T00:35:09.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:09 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-30T00:35:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:10 smithi142 ceph-mon[41779]: pgmap v315: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-30T00:35:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-30T00:35:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-30T00:35:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-30T00:35:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:10 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-30T00:35:10.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:10 smithi142 ceph-mon[41779]: osdmap e86: 12 total, 12 up, 12 in 2024-03-30T00:35:10.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:10 smithi161 ceph-mon[42221]: pgmap v315: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-30T00:35:10.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-30T00:35:10.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-30T00:35:10.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-30T00:35:10.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:10 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-30T00:35:10.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:10 smithi161 ceph-mon[42221]: osdmap e86: 12 total, 12 up, 12 in 2024-03-30T00:35:10.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:10 smithi099 ceph-mon[38079]: pgmap v315: 97 pgs: 97 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-30T00:35:10.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-30T00:35:10.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-30T00:35:10.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-03-30T00:35:10.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:10 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-30T00:35:10.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:10 smithi099 ceph-mon[38079]: osdmap e86: 12 total, 12 up, 12 in 2024-03-30T00:35:11.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:11 smithi142 ceph-mon[41779]: osdmap e87: 12 total, 12 up, 12 in 2024-03-30T00:35:11.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:11 smithi161 ceph-mon[42221]: osdmap e87: 12 total, 12 up, 12 in 2024-03-30T00:35:11.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:11 smithi099 ceph-mon[38079]: osdmap e87: 12 total, 12 up, 12 in 2024-03-30T00:35:12.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:12 smithi142 ceph-mon[41779]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-30T00:35:12.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:12 smithi161 ceph-mon[42221]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-30T00:35:12.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:12 smithi099 ceph-mon[38079]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 660 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-03-30T00:35:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.132+0000 7f0cfc000700 1 -- 172.21.15.99:0/1217091199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4100d10 msgr2=0x7f0cf4101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.132+0000 7f0cfc000700 1 --2- 172.21.15.99:0/1217091199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4100d10 0x7f0cf4101130 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80099c0 tx=0x7f0ce8009cd0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.132+0000 7f0cfc000700 1 -- 172.21.15.99:0/1217091199 shutdown_connections 2024-03-30T00:35:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.132+0000 7f0cfc000700 1 --2- 172.21.15.99:0/1217091199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf41028a0 0x7f0cf4109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.132+0000 7f0cfc000700 1 --2- 172.21.15.99:0/1217091199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4100d10 0x7f0cf4101130 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.132+0000 7f0cfc000700 1 --2- 172.21.15.99:0/1217091199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cf4101f00 0x7f0cf4102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.132+0000 7f0cfc000700 1 -- 172.21.15.99:0/1217091199 >> 172.21.15.99:0/1217091199 conn(0x7f0cf40fc280 msgr2=0x7f0cf40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.132+0000 7f0cfc000700 1 -- 172.21.15.99:0/1217091199 shutdown_connections 2024-03-30T00:35:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.133+0000 7f0cfc000700 1 -- 172.21.15.99:0/1217091199 wait complete. 2024-03-30T00:35:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.133+0000 7f0cfc000700 1 Processor -- start 2024-03-30T00:35:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cfc000700 1 -- start start 2024-03-30T00:35:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cfc000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4100d10 0x7f0cf4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cfc000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf4101f00 0x7f0cf4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cfc000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cf41028a0 0x7f0cf41a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cfc000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf41a0930 con 0x7f0cf4100d10 2024-03-30T00:35:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cfc000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf41a0a70 con 0x7f0cf41028a0 2024-03-30T00:35:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cfc000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf41a0c00 con 0x7f0cf4101f00 2024-03-30T00:35:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cf9600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4100d10 0x7f0cf4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cf9600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4100d10 0x7f0cf4198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43378/0 (socket says 172.21.15.99:43378) 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cf9600700 1 -- 172.21.15.99:0/1416460642 learned_addr learned my addr 172.21.15.99:0/1416460642 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf4101f00 0x7f0cf4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.134+0000 7f0cfa000700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cf41028a0 0x7f0cf41a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.135+0000 7f0cf9600700 1 -- 172.21.15.99:0/1416460642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf4101f00 msgr2=0x7f0cf4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.135+0000 7f0cf9600700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf4101f00 0x7f0cf4199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.135+0000 7f0cf9600700 1 -- 172.21.15.99:0/1416460642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cf41028a0 msgr2=0x7f0cf41a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.135+0000 7f0cf9600700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cf41028a0 0x7f0cf41a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.135+0000 7f0cf9600700 1 -- 172.21.15.99:0/1416460642 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce80095f0 con 0x7f0cf4100d10 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.135+0000 7f0cf9600700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4100d10 0x7f0cf4198e00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009f50 tx=0x7f0ce8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.135+0000 7f0cf2000700 1 -- 172.21.15.99:0/1416460642 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce801d070 con 0x7f0cf4100d10 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.135+0000 7f0cf2000700 1 -- 172.21.15.99:0/1416460642 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ce800bda0 con 0x7f0cf4100d10 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.135+0000 7f0cfc000700 1 -- 172.21.15.99:0/1416460642 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf41a0e80 con 0x7f0cf4100d10 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.135+0000 7f0cf2000700 1 -- 172.21.15.99:0/1416460642 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8022dd0 con 0x7f0cf4100d10 2024-03-30T00:35:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.135+0000 7f0cfc000700 1 -- 172.21.15.99:0/1416460642 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf41a11f0 con 0x7f0cf4100d10 2024-03-30T00:35:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.140+0000 7f0cf2000700 1 -- 172.21.15.99:0/1416460642 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f0ce8022570 con 0x7f0cf4100d10 2024-03-30T00:35:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0d0fc02b80 1 Processor -- start 2024-03-30T00:35:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0d0fc02b80 1 -- start start 2024-03-30T00:35:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28200 0x556046d28620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 0x556046d294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 0x556046d29ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046c80930 con 0x556046d28b60 2024-03-30T00:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046c7e2f0 con 0x556046d28200 2024-03-30T00:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046c79be0 con 0x556046d29a60 2024-03-30T00:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0cf2000700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0cdc0a1120 0x7f0cdc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0cf9600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 0x556046d294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0cf2000700 1 -- 172.21.15.99:0/1416460642 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f0ce80cdd10 con 0x7f0cf4100d10 2024-03-30T00:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.141+0000 7f0cf8c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 0x556046d29ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cfa000700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0cdc0a1120 0x7f0cdc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cf8c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 0x556046d29ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:34424/0 (socket says 172.21.15.99:34424) 2024-03-30T00:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cf8c00700 1 -- 172.21.15.99:0/3739729380 learned_addr learned my addr 172.21.15.99:0/3739729380 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cfa000700 1 --2- 172.21.15.99:0/3739729380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28200 0x556046d28620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cf8c00700 1 -- 172.21.15.99:0/3739729380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28200 msgr2=0x556046d28620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/3739729380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28200 0x556046d28620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cf8c00700 1 -- 172.21.15.99:0/3739729380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 msgr2=0x556046d294d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/3739729380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 0x556046d294d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cf8c00700 1 -- 172.21.15.99:0/3739729380 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556046afb350 con 0x556046d29a60 2024-03-30T00:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cf9600700 1 --2- 172.21.15.99:0/3739729380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 0x556046d294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cfa000700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0cdc0a1120 0x7f0cdc0a35e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009550 tx=0x7f0cec009070 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.142+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/3739729380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 0x556046d29ef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0007d70 tx=0x7f0ce00106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42bf6bebbf8fffa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.143+0000 7f0ce6600700 1 -- 172.21.15.99:0/3739729380 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0010e10 con 0x556046d29a60 2024-03-30T00:35:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.143+0000 7f0ce6600700 1 -- 172.21.15.99:0/3739729380 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ce0015070 con 0x556046d29a60 2024-03-30T00:35:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.143+0000 7f0ce6600700 1 -- 172.21.15.99:0/3739729380 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0003dc0 con 0x556046d29a60 2024-03-30T00:35:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.143+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3739729380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 msgr2=0x556046d29ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.143+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/3739729380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 0x556046d29ef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0007d70 tx=0x7f0ce00106b0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.143+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3739729380 shutdown_connections 2024-03-30T00:35:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.143+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/3739729380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 0x556046d29ef0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.143+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/3739729380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28200 0x556046d28620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.143+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/3739729380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 0x556046d294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.143+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3739729380 >> 172.21.15.99:0/3739729380 conn(0x556046c87640 msgr2=0x556046d23050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.144+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3739729380 shutdown_connections 2024-03-30T00:35:14.153 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.144+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3739729380 wait complete. 2024-03-30T00:35:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.144+0000 7f0d0fc02b80 1 -- start start 2024-03-30T00:35:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.144+0000 7f0d0fc02b80 1 Processor -- start 2024-03-30T00:35:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d28200 0x556046d32170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28b60 0x556046d326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d29a60 0x556046d396f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046c15d20 con 0x556046d29a60 2024-03-30T00:35:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046c159c0 con 0x556046d28b60 2024-03-30T00:35:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046afd640 con 0x556046d28200 2024-03-30T00:35:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0d0fc02b80 10 client.0 _ll_register_callbacks cb 0x556046c1f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cfa000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d29a60 0x556046d396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cfa000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d29a60 0x556046d396f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43396/0 (socket says 172.21.15.99:43396) 2024-03-30T00:35:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cfa000700 1 -- 172.21.15.99:0/3882424080 learned_addr learned my addr 172.21.15.99:0/3882424080 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cf9600700 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d28200 0x556046d32170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28b60 0x556046d326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cfa000700 1 -- 172.21.15.99:0/3882424080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d28200 msgr2=0x556046d32170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cfa000700 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d28200 0x556046d32170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cfa000700 1 -- 172.21.15.99:0/3882424080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28b60 msgr2=0x556046d326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cfa000700 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28b60 0x556046d326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cfa000700 1 -- 172.21.15.99:0/3882424080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf41a11f0 con 0x556046d29a60 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cf9600700 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d28200 0x556046d32170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.145+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28b60 0x556046d326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.146+0000 7f0cfa000700 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d29a60 0x556046d396f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0cec019a60 tx=0x7f0cec004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979b111f2c9878a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.146+0000 7f0cdaa00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.146+0000 7f0cdaa00700 1 -- 172.21.15.99:0/3882424080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec01e070 con 0x556046d29a60 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.146+0000 7f0cdaa00700 1 -- 172.21.15.99:0/3882424080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f0cec004bb0 con 0x556046d29a60 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.146+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3882424080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556046c3a320 con 0x556046d29a60 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.146+0000 7f0cdaa00700 1 -- 172.21.15.99:0/3882424080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cec004d70 con 0x556046d29a60 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.146+0000 7f0d0fc02b80 10 client.15519 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.146+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3882424080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f0ce00079a0 con 0x556046d29a60 2024-03-30T00:35:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.147+0000 7f0cdaa00700 1 -- 172.21.15.99:0/3882424080 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0cec032080 con 0x556046d29a60 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.147+0000 7f0cdaa00700 1 -- 172.21.15.99:0/3882424080 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0cec024680 con 0x556046d29a60 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.147+0000 7f0d0fc02b80 10 client.15519 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.147+0000 7f0d0fc02b80 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.147+0000 7f0d0fc02b80 20 client.15519 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.147+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556046d3e080 0x556046d404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.147+0000 7f0d0fc02b80 4 client.15519 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.147+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3882424080 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556046c726a0 con 0x556046d3e080 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.147+0000 7f0cf9600700 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556046d3e080 0x556046d404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.148+0000 7f0cf9600700 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556046d3e080 0x556046d404a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e196f2a5f3d08ddf server_cookie=2dc032a11377b7c4 in_seq=0 out_seq=0 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.148+0000 7f0cdaa00700 10 client.15519 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0cdaa00700 1 -- 172.21.15.99:0/3882424080 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556046c726a0 con 0x556046d3e080 2024-03-30T00:35:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0cdaa00700 10 client.15519 handle_command_reply: tid=0 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 1 client.15519 shutdown 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3882424080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d29a60 msgr2=0x556046d396f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d29a60 0x556046d396f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0cec019a60 tx=0x7f0cec004b40 comp rx=0 tx=0).stop 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3882424080 shutdown_connections 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556046d3e080 0x556046d404a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d29a60 0x556046d396f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d28200 0x556046d32170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/3882424080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28b60 0x556046d326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0cdaa00700 0 client.15519 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3882424080 >> 172.21.15.99:0/3882424080 conn(0x556046c87640 msgr2=0x556046d26fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3882424080 shutdown_connections 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/3882424080 wait complete. 2024-03-30T00:35:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.149+0000 7f0d0fc02b80 20 client.15519 trim_cache size 0 max 16384 2024-03-30T00:35:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.164+0000 7f0d0fc02b80 1 Processor -- start 2024-03-30T00:35:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.164+0000 7f0d0fc02b80 1 -- start start 2024-03-30T00:35:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d28200 0x556046d29e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 0x556046d39610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d29a60 0x556046d1d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046d1f400 con 0x556046d28b60 2024-03-30T00:35:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046c726a0 con 0x556046d29a60 2024-03-30T00:35:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046d3b480 con 0x556046d28200 2024-03-30T00:35:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0cf8c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 0x556046d39610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0cf8c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 0x556046d39610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43410/0 (socket says 172.21.15.99:43410) 2024-03-30T00:35:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0cf8c00700 1 -- 172.21.15.99:0/1287071757 learned_addr learned my addr 172.21.15.99:0/1287071757 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0cf9600700 1 --2- 172.21.15.99:0/1287071757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d28200 0x556046d29e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0cf8c00700 1 -- 172.21.15.99:0/1287071757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d28200 msgr2=0x556046d29e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/1287071757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d28200 0x556046d29e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0cf8c00700 1 -- 172.21.15.99:0/1287071757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d29a60 msgr2=0x556046d1d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/1287071757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d29a60 0x556046d1d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.165+0000 7f0cf8c00700 1 -- 172.21.15.99:0/1287071757 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556046aa7b00 con 0x556046d28b60 2024-03-30T00:35:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.166+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/1287071757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 0x556046d39610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005fd0 tx=0x7f0ce0005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a30331ffc44c02 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.166+0000 7f0ce7000700 1 -- 172.21.15.99:0/1287071757 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0003900 con 0x556046d28b60 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.166+0000 7f0ce7000700 1 -- 172.21.15.99:0/1287071757 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ce0020040 con 0x556046d28b60 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.166+0000 7f0ce7000700 1 -- 172.21.15.99:0/1287071757 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00055f0 con 0x556046d28b60 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.166+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/1287071757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 msgr2=0x556046d39610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.166+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/1287071757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 0x556046d39610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0005fd0 tx=0x7f0ce0005920 comp rx=0 tx=0).stop 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.167+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/1287071757 shutdown_connections 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.167+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/1287071757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d29a60 0x556046d1d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.167+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/1287071757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d28200 0x556046d29e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.167+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/1287071757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28b60 0x556046d39610 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.167+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/1287071757 >> 172.21.15.99:0/1287071757 conn(0x556046c87640 msgr2=0x556046d30050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.167+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/1287071757 shutdown_connections 2024-03-30T00:35:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.167+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/1287071757 wait complete. 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.167+0000 7f0d0fc02b80 1 -- start start 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.167+0000 7f0d0fc02b80 1 Processor -- start 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28200 0x556046d377d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28b60 0x556046d37d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0d0fc02b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 0x556046d38b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046d39130 con 0x556046d28200 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046d392a0 con 0x556046d28b60 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0d0fc02b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556046d39460 con 0x556046d29a60 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0d0fc02b80 10 client.0 _ll_register_callbacks cb 0x556046c1f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cf9600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28200 0x556046d377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cf9600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28200 0x556046d377d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43412/0 (socket says 172.21.15.99:43412) 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cf9600700 1 -- 172.21.15.99:0/4038358447 learned_addr learned my addr 172.21.15.99:0/4038358447 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28b60 0x556046d37d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cfa000700 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 0x556046d38b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cf9600700 1 -- 172.21.15.99:0/4038358447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 msgr2=0x556046d38b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cf9600700 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 0x556046d38b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cf9600700 1 -- 172.21.15.99:0/4038358447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28b60 msgr2=0x556046d37d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cf9600700 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28b60 0x556046d37d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cf9600700 1 -- 172.21.15.99:0/4038358447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556046afb350 con 0x556046d28200 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cfa000700 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 0x556046d38b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cf9600700 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28200 0x556046d377d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ce809ae10 tx=0x7f0ce800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8068dd5c47985fda server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.168+0000 7f0cdaa00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0cdaa00700 1 -- 172.21.15.99:0/4038358447 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8079020 con 0x556046d28200 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0cdaa00700 1 -- 172.21.15.99:0/4038358447 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f0ce809ef40 con 0x556046d28200 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/4038358447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ce0015070 con 0x556046d28200 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0cdaa00700 1 -- 172.21.15.99:0/4038358447 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce80a3500 con 0x556046d28200 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0d0fc02b80 10 client.15525 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/4038358447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556046c3fe40 con 0x556046d28200 2024-03-30T00:35:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0cdaa00700 1 -- 172.21.15.99:0/4038358447 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ce0015070 con 0x556046d28200 2024-03-30T00:35:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0cdaa00700 1 -- 172.21.15.99:0/4038358447 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0ce8075b90 con 0x556046d28200 2024-03-30T00:35:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0d0fc02b80 10 client.15525 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0d0fc02b80 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0d0fc02b80 20 client.15525 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556046d92e50 0x556046d95270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0d0fc02b80 4 client.15525 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:35:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.169+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/4038358447 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556046ac8c90 con 0x556046d92e50 2024-03-30T00:35:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.170+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556046d92e50 0x556046d95270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.170+0000 7f0cdaa00700 10 client.15525 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.170+0000 7f0cf8c00700 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556046d92e50 0x556046d95270 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc51ab3c7a3ee778 server_cookie=7e4f8fd8399d27e4 in_seq=0 out_seq=0 2024-03-30T00:35:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0cdaa00700 1 -- 172.21.15.99:0/4038358447 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556046ac8c90 con 0x556046d92e50 2024-03-30T00:35:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0cdaa00700 10 client.15525 handle_command_reply: tid=0 2024-03-30T00:35:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 1 client.15525 shutdown 2024-03-30T00:35:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/4038358447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28200 msgr2=0x556046d377d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28200 0x556046d377d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ce809ae10 tx=0x7f0ce800bf90 comp rx=0 tx=0).stop 2024-03-30T00:35:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/4038358447 shutdown_connections 2024-03-30T00:35:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556046d92e50 0x556046d95270 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556046d29a60 0x556046d38b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556046d28200 0x556046d377d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0cdaa00700 0 client.15525 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 1 --2- 172.21.15.99:0/4038358447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556046d28b60 0x556046d37d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/4038358447 >> 172.21.15.99:0/4038358447 conn(0x556046c87640 msgr2=0x556046d2fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/4038358447 shutdown_connections 2024-03-30T00:35:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 1 -- 172.21.15.99:0/4038358447 wait complete. 2024-03-30T00:35:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.175+0000 7f0d0fc02b80 20 client.15525 trim_cache size 0 max 16384 2024-03-30T00:35:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:35:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:35:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:35:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:35:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 -- 172.21.15.99:0/1416460642 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0cdc0a1120 msgr2=0x7f0cdc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0cdc0a1120 0x7f0cdc0a35e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009550 tx=0x7f0cec009070 comp rx=0 tx=0).stop 2024-03-30T00:35:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 -- 172.21.15.99:0/1416460642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4100d10 msgr2=0x7f0cf4198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4100d10 0x7f0cf4198e00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009f50 tx=0x7f0ce8005b00 comp rx=0 tx=0).stop 2024-03-30T00:35:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 -- 172.21.15.99:0/1416460642 shutdown_connections 2024-03-30T00:35:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0cdc0a1120 0x7f0cdc0a35e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cf41028a0 0x7f0cf41a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4100d10 0x7f0cf4198e00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 --2- 172.21.15.99:0/1416460642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf4101f00 0x7f0cf4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 -- 172.21.15.99:0/1416460642 >> 172.21.15.99:0/1416460642 conn(0x7f0cf40fc280 msgr2=0x7f0cf41077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 -- 172.21.15.99:0/1416460642 shutdown_connections 2024-03-30T00:35:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:14.176+0000 7f0cfc000700 1 -- 172.21.15.99:0/1416460642 wait complete. 2024-03-30T00:35:14.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:14 smithi142 ceph-mon[41779]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 895 B/s wr, 8 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:14.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:14 smithi161 ceph-mon[42221]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 895 B/s wr, 8 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:14.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:14 smithi099 ceph-mon[38079]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 895 B/s wr, 8 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:16.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:16 smithi142 ceph-mon[41779]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:16.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:16 smithi161 ceph-mon[42221]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:16.988 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:16 smithi099 ceph-mon[38079]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 660 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:18.282 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T00:35:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.479+0000 7f7c79a00700 1 -- 172.21.15.99:0/2385460109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c74073810 msgr2=0x7f7c74073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.479+0000 7f7c79a00700 1 --2- 172.21.15.99:0/2385460109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c74073810 0x7f7c74073c90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009990 tx=0x7f7c60009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.479+0000 7f7c79a00700 1 -- 172.21.15.99:0/2385460109 shutdown_connections 2024-03-30T00:35:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.479+0000 7f7c79a00700 1 --2- 172.21.15.99:0/2385460109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c74108c40 0x7f7c7410b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.479+0000 7f7c79a00700 1 --2- 172.21.15.99:0/2385460109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c74074df0 0x7f7c74073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.479+0000 7f7c79a00700 1 --2- 172.21.15.99:0/2385460109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c74073810 0x7f7c74073c90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.479+0000 7f7c79a00700 1 -- 172.21.15.99:0/2385460109 >> 172.21.15.99:0/2385460109 conn(0x7f7c740fc2f0 msgr2=0x7f7c740fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:18.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.479+0000 7f7c79a00700 1 -- 172.21.15.99:0/2385460109 shutdown_connections 2024-03-30T00:35:18.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.479+0000 7f7c79a00700 1 -- 172.21.15.99:0/2385460109 wait complete. 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.480+0000 7f7c79a00700 1 Processor -- start 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.480+0000 7f7c79a00700 1 -- start start 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.480+0000 7f7c79a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c74073810 0x7f7c74198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.480+0000 7f7c79a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c74074df0 0x7f7c741993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.480+0000 7f7c79a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c74108c40 0x7f7c741a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.480+0000 7f7c79a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741a09d0 con 0x7f7c74073810 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.480+0000 7f7c79a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741a0b10 con 0x7f7c74108c40 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c79a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741a0ca0 con 0x7f7c74074df0 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c72a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c74073810 0x7f7c74198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c72a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c74073810 0x7f7c74198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34190/0 (socket says 172.21.15.99:34190) 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c72a00700 1 -- 172.21.15.99:0/1478093371 learned_addr learned my addr 172.21.15.99:0/1478093371 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c73400700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c74108c40 0x7f7c741a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c72000700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c74074df0 0x7f7c741993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c72a00700 1 -- 172.21.15.99:0/1478093371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c74074df0 msgr2=0x7f7c741993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c72a00700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c74074df0 0x7f7c741993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c72a00700 1 -- 172.21.15.99:0/1478093371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c74108c40 msgr2=0x7f7c741a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c72a00700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c74108c40 0x7f7c741a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c72a00700 1 -- 172.21.15.99:0/1478093371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c600095f0 con 0x7f7c74073810 2024-03-30T00:35:18.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.481+0000 7f7c72a00700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c74073810 0x7f7c74198ea0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400e9a0 tx=0x7f7c6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:18.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.482+0000 7f7c6b400700 1 -- 172.21.15.99:0/1478093371 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64009790 con 0x7f7c74073810 2024-03-30T00:35:18.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.482+0000 7f7c6b400700 1 -- 172.21.15.99:0/1478093371 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c64010430 con 0x7f7c74073810 2024-03-30T00:35:18.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.482+0000 7f7c79a00700 1 -- 172.21.15.99:0/1478093371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c741a0f80 con 0x7f7c74073810 2024-03-30T00:35:18.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.482+0000 7f7c6b400700 1 -- 172.21.15.99:0/1478093371 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6400cea0 con 0x7f7c74073810 2024-03-30T00:35:18.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.482+0000 7f7c79a00700 1 -- 172.21.15.99:0/1478093371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c741a13a0 con 0x7f7c74073810 2024-03-30T00:35:18.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.485+0000 7f7c6b400700 1 -- 172.21.15.99:0/1478093371 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7c640105a0 con 0x7f7c74073810 2024-03-30T00:35:18.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.486+0000 7f7c8d4b0b80 1 Processor -- start 2024-03-30T00:35:18.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.486+0000 7f7c8d4b0b80 1 -- start start 2024-03-30T00:35:18.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.486+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f87d0 0x5650399f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x5650399f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c6b400700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7c580a1120 0x7f7c580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9fd0 0x5650399fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x565039950990 con 0x5650399f9fd0 2024-03-30T00:35:18.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503994e350 con 0x5650399f9130 2024-03-30T00:35:18.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x565039949c40 con 0x5650399f87d0 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c6b400700 1 -- 172.21.15.99:0/1478093371 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f7c64014070 con 0x7f7c74073810 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c72a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x5650399f99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c72a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x5650399f99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:35560/0 (socket says 172.21.15.99:35560) 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c72000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9fd0 0x5650399fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c72a00700 1 -- 172.21.15.99:0/1518303324 learned_addr learned my addr 172.21.15.99:0/1518303324 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c73400700 1 --2- 172.21.15.99:0/1518303324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f87d0 0x5650399f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c73400700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7c580a1120 0x7f7c580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c72a00700 1 -- 172.21.15.99:0/1518303324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f87d0 msgr2=0x5650399f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c72a00700 1 --2- 172.21.15.99:0/1518303324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f87d0 0x5650399f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c72a00700 1 -- 172.21.15.99:0/1518303324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9fd0 msgr2=0x5650399fa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c72a00700 1 --2- 172.21.15.99:0/1518303324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9fd0 0x5650399fa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.487+0000 7f7c72a00700 1 -- 172.21.15.99:0/1518303324 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650397cb3b0 con 0x5650399f9130 2024-03-30T00:35:18.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.488+0000 7f7c73400700 1 --2- 172.21.15.99:0/1518303324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f87d0 0x5650399f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:35:18.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.488+0000 7f7c72a00700 1 --2- 172.21.15.99:0/1518303324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x5650399f99f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c64099e80 tx=0x7f7c64099b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b91d28c96ed54d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:18.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.488+0000 7f7c5fe00700 1 -- 172.21.15.99:0/1518303324 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64099ef0 con 0x5650399f9130 2024-03-30T00:35:18.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.488+0000 7f7c5fe00700 1 -- 172.21.15.99:0/1518303324 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c6409dd20 con 0x5650399f9130 2024-03-30T00:35:18.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.488+0000 7f7c5fe00700 1 -- 172.21.15.99:0/1518303324 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c640993e0 con 0x5650399f9130 2024-03-30T00:35:18.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.489+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/1518303324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 msgr2=0x5650399f99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.489+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/1518303324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x5650399f99f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c64099e80 tx=0x7f7c64099b60 comp rx=0 tx=0).stop 2024-03-30T00:35:18.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.489+0000 7f7c73400700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7c580a1120 0x7f7c580a35e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009550 tx=0x7f7c6c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:18.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.489+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/1518303324 shutdown_connections 2024-03-30T00:35:18.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.489+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/1518303324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9fd0 0x5650399fa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.489+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/1518303324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f87d0 0x5650399f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.489+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/1518303324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x5650399f99f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.489+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/1518303324 >> 172.21.15.99:0/1518303324 conn(0x5650399576a0 msgr2=0x5650399f39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.489+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/1518303324 shutdown_connections 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.489+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/1518303324 wait complete. 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.489+0000 7f7c8d4b0b80 1 -- start start 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.490+0000 7f7c8d4b0b80 1 Processor -- start 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.490+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 0x565039a025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.490+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x565039a02b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.490+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f9fd0 0x565039a09b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.490+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650398e5d80 con 0x5650399f87d0 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.490+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650398e5a20 con 0x5650399f9130 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.490+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650397cd6a0 con 0x5650399f9fd0 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.490+0000 7f7c8d4b0b80 10 client.0 _ll_register_callbacks cb 0x5650398ef470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c72000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x565039a02b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c73400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f9fd0 0x565039a09b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c72a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 0x565039a025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c72000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x565039a02b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:35576/0 (socket says 172.21.15.99:35576) 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c72000700 1 -- 172.21.15.99:0/3255872878 learned_addr learned my addr 172.21.15.99:0/3255872878 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c73400700 1 -- 172.21.15.99:0/3255872878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 msgr2=0x565039a02b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c73400700 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x565039a02b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c73400700 1 -- 172.21.15.99:0/3255872878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 msgr2=0x565039a025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c73400700 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 0x565039a025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c73400700 1 -- 172.21.15.99:0/3255872878 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c741a13a0 con 0x5650399f9fd0 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c72a00700 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 0x565039a025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c73400700 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f9fd0 0x565039a09b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009ec0 tx=0x7f7c6c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1807edb180ece41c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.491+0000 7f7c5cc00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c5cc00700 1 -- 172.21.15.99:0/3255872878 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00e040 con 0x5650399f9fd0 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3255872878 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56503990a380 con 0x5650399f9fd0 2024-03-30T00:35:18.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c5cc00700 1 -- 172.21.15.99:0/3255872878 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f7c6c007e40 con 0x5650399f9fd0 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c5cc00700 1 -- 172.21.15.99:0/3255872878 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c6c015cc0 con 0x5650399f9fd0 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c8d4b0b80 10 client.25261 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3255872878 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x565039777b60 con 0x5650399f9fd0 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c5cc00700 1 -- 172.21.15.99:0/3255872878 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56503990a380 con 0x5650399f9fd0 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c5cc00700 1 -- 172.21.15.99:0/3255872878 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7c6c004590 con 0x5650399f9fd0 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c8d4b0b80 10 client.25261 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c8d4b0b80 10 client.25261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c8d4b0b80 20 client.25261 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565039a0e380 0x565039a107a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c8d4b0b80 4 client.25261 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.492+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3255872878 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565039942700 con 0x565039a0e380 2024-03-30T00:35:18.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.493+0000 7f7c72a00700 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565039a0e380 0x565039a107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.493+0000 7f7c72a00700 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565039a0e380 0x565039a107a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb23ecae9a8316cd server_cookie=db1600d196cb23ec in_seq=0 out_seq=0 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.493+0000 7f7c5cc00700 10 client.25261 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.494+0000 7f7c5cc00700 1 -- 172.21.15.99:0/3255872878 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x565039942700 con 0x565039a0e380 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.494+0000 7f7c5cc00700 10 client.25261 handle_command_reply: tid=0 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.494+0000 7f7c8d4b0b80 1 client.25261 shutdown 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.494+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3255872878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f9fd0 msgr2=0x565039a09b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.494+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f9fd0 0x565039a09b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009ec0 tx=0x7f7c6c004b40 comp rx=0 tx=0).stop 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.494+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3255872878 shutdown_connections 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.494+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565039a0e380 0x565039a107a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.494+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f9fd0 0x565039a09b50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.494+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 0x565039a025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.495+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/3255872878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x565039a02b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.495+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3255872878 >> 172.21.15.99:0/3255872878 conn(0x5650399576a0 msgr2=0x5650399f7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:18.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.495+0000 7f7c5cc00700 0 client.25261 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:18.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.495+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3255872878 shutdown_connections 2024-03-30T00:35:18.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.495+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3255872878 wait complete. 2024-03-30T00:35:18.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.495+0000 7f7c8d4b0b80 20 client.25261 trim_cache size 0 max 16384 2024-03-30T00:35:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.513+0000 7f7c8d4b0b80 1 Processor -- start 2024-03-30T00:35:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.513+0000 7f7c8d4b0b80 1 -- start start 2024-03-30T00:35:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.513+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f87d0 0x565039a10770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.513+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9130 0x565039a09b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.513+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9fd0 0x5650399f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.513+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x565039a03640 con 0x5650399f9130 2024-03-30T00:35:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.513+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565039a02170 con 0x5650399f9fd0 2024-03-30T00:35:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.513+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x565039a01600 con 0x5650399f87d0 2024-03-30T00:35:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.514+0000 7f7c72000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9130 0x565039a09b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.514+0000 7f7c72000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9130 0x565039a09b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34210/0 (socket says 172.21.15.99:34210) 2024-03-30T00:35:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.514+0000 7f7c72000700 1 -- 172.21.15.99:0/3737575341 learned_addr learned my addr 172.21.15.99:0/3737575341 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.514+0000 7f7c72a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f87d0 0x565039a10770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.514+0000 7f7c73400700 1 --2- 172.21.15.99:0/3737575341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9fd0 0x5650399f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.514+0000 7f7c72000700 1 -- 172.21.15.99:0/3737575341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f87d0 msgr2=0x565039a10770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.514+0000 7f7c72000700 1 --2- 172.21.15.99:0/3737575341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f87d0 0x565039a10770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.514+0000 7f7c72000700 1 -- 172.21.15.99:0/3737575341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9fd0 msgr2=0x5650399f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.514+0000 7f7c72000700 1 --2- 172.21.15.99:0/3737575341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9fd0 0x5650399f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.514+0000 7f7c72000700 1 -- 172.21.15.99:0/3737575341 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56503990fea0 con 0x5650399f9130 2024-03-30T00:35:18.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.515+0000 7f7c72000700 1 --2- 172.21.15.99:0/3737575341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9130 0x565039a09b10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7c600098e0 tx=0x7f7c6000b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c6211cb039796c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:18.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.515+0000 7f7c68c00700 1 -- 172.21.15.99:0/3737575341 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6001d030 con 0x5650399f9130 2024-03-30T00:35:18.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.515+0000 7f7c68c00700 1 -- 172.21.15.99:0/3737575341 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c6001e040 con 0x5650399f9130 2024-03-30T00:35:18.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.515+0000 7f7c68c00700 1 -- 172.21.15.99:0/3737575341 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000b710 con 0x5650399f9130 2024-03-30T00:35:18.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.516+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3737575341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9130 msgr2=0x565039a09b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.516+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/3737575341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9130 0x565039a09b10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7c600098e0 tx=0x7f7c6000b610 comp rx=0 tx=0).stop 2024-03-30T00:35:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.516+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3737575341 shutdown_connections 2024-03-30T00:35:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.516+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/3737575341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9fd0 0x5650399f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.516+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/3737575341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f87d0 0x565039a10770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.516+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/3737575341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f9130 0x565039a09b10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.516+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3737575341 >> 172.21.15.99:0/3737575341 conn(0x5650399576a0 msgr2=0x5650399ec5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.516+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3737575341 shutdown_connections 2024-03-30T00:35:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.516+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/3737575341 wait complete. 2024-03-30T00:35:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.517+0000 7f7c8d4b0b80 1 -- start start 2024-03-30T00:35:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.517+0000 7f7c8d4b0b80 1 Processor -- start 2024-03-30T00:35:18.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.517+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 0x565039a08380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.517+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x565039a088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.517+0000 7f7c8d4b0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f9fd0 0x565039a08e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.517+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x565039a09f00 con 0x5650399f87d0 2024-03-30T00:35:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.517+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565039a621d0 con 0x5650399f9130 2024-03-30T00:35:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.517+0000 7f7c8d4b0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x565039a62390 con 0x5650399f9fd0 2024-03-30T00:35:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.517+0000 7f7c8d4b0b80 10 client.0 _ll_register_callbacks cb 0x5650398ef470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c72a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 0x565039a08380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c72000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x565039a088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c73400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f9fd0 0x565039a08e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c72a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 0x565039a08380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34214/0 (socket says 172.21.15.99:34214) 2024-03-30T00:35:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c72a00700 1 -- 172.21.15.99:0/958023857 learned_addr learned my addr 172.21.15.99:0/958023857 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c72a00700 1 -- 172.21.15.99:0/958023857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f9fd0 msgr2=0x565039a08e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c72a00700 1 --2- 172.21.15.99:0/958023857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f9fd0 0x565039a08e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c72a00700 1 -- 172.21.15.99:0/958023857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 msgr2=0x565039a088c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c72a00700 1 --2- 172.21.15.99:0/958023857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x565039a088c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c72a00700 1 -- 172.21.15.99:0/958023857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7410bcf0 con 0x5650399f87d0 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c72a00700 1 --2- 172.21.15.99:0/958023857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 0x565039a08380 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c64096960 tx=0x7f7c640109a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bfde812e52ecdf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.518+0000 7f7c5cc00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c5cc00700 1 -- 172.21.15.99:0/958023857 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c640ca7b0 con 0x5650399f87d0 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c5cc00700 1 -- 172.21.15.99:0/958023857 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f7c64099ef0 con 0x5650399f87d0 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/958023857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c6409dd20 con 0x5650399f87d0 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c5cc00700 1 -- 172.21.15.99:0/958023857 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c64099850 con 0x5650399f87d0 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c8d4b0b80 10 client.15543 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:18.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/958023857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5650397cfcd0 con 0x5650399f87d0 2024-03-30T00:35:18.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c5cc00700 1 -- 172.21.15.99:0/958023857 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c64063020 con 0x5650399f87d0 2024-03-30T00:35:18.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c5cc00700 1 -- 172.21.15.99:0/958023857 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7c640a2000 con 0x5650399f87d0 2024-03-30T00:35:18.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c8d4b0b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:18.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c8d4b0b80 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:18.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c8d4b0b80 20 client.15543 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:18.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/958023857 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565039a63c40 0x565039a66060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c8d4b0b80 4 client.15543 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:35:18.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.519+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/958023857 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565039798cf0 con 0x565039a63c40 2024-03-30T00:35:18.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.520+0000 7f7c72000700 1 --2- 172.21.15.99:0/958023857 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565039a63c40 0x565039a66060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c72000700 1 --2- 172.21.15.99:0/958023857 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565039a63c40 0x565039a66060 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6617908391f704a server_cookie=f5a5e71de73cf6c8 in_seq=0 out_seq=0 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c5cc00700 10 client.15543 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c5cc00700 1 -- 172.21.15.99:0/958023857 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x565039798cf0 con 0x565039a63c40 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c5cc00700 10 client.15543 handle_command_reply: tid=0 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 1 client.15543 shutdown 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/958023857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 msgr2=0x565039a08380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/958023857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 0x565039a08380 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c64096960 tx=0x7f7c640109a0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/958023857 shutdown_connections 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/958023857 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565039a63c40 0x565039a66060 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/958023857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5650399f9fd0 0x565039a08e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/958023857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5650399f87d0 0x565039a08380 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 1 --2- 172.21.15.99:0/958023857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650399f9130 0x565039a088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/958023857 >> 172.21.15.99:0/958023857 conn(0x5650399576a0 msgr2=0x565039a002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c5cc00700 0 client.15543 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/958023857 shutdown_connections 2024-03-30T00:35:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 1 -- 172.21.15.99:0/958023857 wait complete. 2024-03-30T00:35:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.521+0000 7f7c8d4b0b80 20 client.15543 trim_cache size 0 max 16384 2024-03-30T00:35:18.532 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:35:18.532 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "no active scrubs running", 2024-03-30T00:35:18.532 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": {} 2024-03-30T00:35:18.532 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:35:18.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 -- 172.21.15.99:0/1478093371 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7c580a1120 msgr2=0x7f7c580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7c580a1120 0x7f7c580a35e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009550 tx=0x7f7c6c00b040 comp rx=0 tx=0).stop 2024-03-30T00:35:18.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 -- 172.21.15.99:0/1478093371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c74073810 msgr2=0x7f7c74198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c74073810 0x7f7c74198ea0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400e9a0 tx=0x7f7c6400ecb0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 -- 172.21.15.99:0/1478093371 shutdown_connections 2024-03-30T00:35:18.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7c580a1120 0x7f7c580a35e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c74108c40 0x7f7c741a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c74073810 0x7f7c74198ea0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 --2- 172.21.15.99:0/1478093371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c74074df0 0x7f7c741993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 -- 172.21.15.99:0/1478093371 >> 172.21.15.99:0/1478093371 conn(0x7f7c740fc2f0 msgr2=0x7f7c74104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:18.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 -- 172.21.15.99:0/1478093371 shutdown_connections 2024-03-30T00:35:18.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.522+0000 7f7c79a00700 1 -- 172.21.15.99:0/1478093371 wait complete. 2024-03-30T00:35:18.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:35:18.547 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:35:18.607 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:18 smithi099 ceph-mon[38079]: pgmap v321: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 7 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:18.607 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:18 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:35:18.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.745+0000 7fefeec00700 1 -- 172.21.15.99:0/1771213995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe8074df0 msgr2=0x7fefe8073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.745+0000 7fefeec00700 1 --2- 172.21.15.99:0/1771213995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe8074df0 0x7fefe8073240 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009990 tx=0x7fefd8009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.745+0000 7fefeec00700 1 -- 172.21.15.99:0/1771213995 shutdown_connections 2024-03-30T00:35:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.745+0000 7fefeec00700 1 --2- 172.21.15.99:0/1771213995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefe8108b40 0x7fefe810afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.745+0000 7fefeec00700 1 --2- 172.21.15.99:0/1771213995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe8074df0 0x7fefe8073240 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.745+0000 7fefeec00700 1 --2- 172.21.15.99:0/1771213995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe8073810 0x7fefe8073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.745+0000 7fefeec00700 1 -- 172.21.15.99:0/1771213995 >> 172.21.15.99:0/1771213995 conn(0x7fefe80fc1f0 msgr2=0x7fefe80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.745+0000 7fefeec00700 1 -- 172.21.15.99:0/1771213995 shutdown_connections 2024-03-30T00:35:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.746+0000 7fefeec00700 1 -- 172.21.15.99:0/1771213995 wait complete. 2024-03-30T00:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.746+0000 7fefeec00700 1 Processor -- start 2024-03-30T00:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefeec00700 1 -- start start 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefeec00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe8073810 0x7fefe8198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefeec00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe8074df0 0x7fefe8199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefeec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefe8108b40 0x7fefe81a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefeec00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe81a07e0 con 0x7fefe8074df0 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefeec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe81a0950 con 0x7fefe8108b40 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefeec00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe81a0b10 con 0x7fefe8073810 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefe7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe8074df0 0x7fefe8199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefe7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe8074df0 0x7fefe8199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34234/0 (socket says 172.21.15.99:34234) 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefe7400700 1 -- 172.21.15.99:0/2365476884 learned_addr learned my addr 172.21.15.99:0/2365476884 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefe7e00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe8073810 0x7fefe8198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.747+0000 7fefecc00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefe8108b40 0x7fefe81a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefe7400700 1 -- 172.21.15.99:0/2365476884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe8073810 msgr2=0x7fefe8198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefe7400700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe8073810 0x7fefe8198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefe7400700 1 -- 172.21.15.99:0/2365476884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefe8108b40 msgr2=0x7fefe81a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefe7400700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefe8108b40 0x7fefe81a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefe7400700 1 -- 172.21.15.99:0/2365476884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd80095f0 con 0x7fefe8074df0 2024-03-30T00:35:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefe7e00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe8073810 0x7fefe8198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefe7400700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe8074df0 0x7fefe8199250 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fefd400d980 tx=0x7fefd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefe4c00700 1 -- 172.21.15.99:0/2365476884 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4004020 con 0x7fefe8074df0 2024-03-30T00:35:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefeec00700 1 -- 172.21.15.99:0/2365476884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe81a0df0 con 0x7fefe8074df0 2024-03-30T00:35:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefe4c00700 1 -- 172.21.15.99:0/2365476884 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fefd400f690 con 0x7fefe8074df0 2024-03-30T00:35:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefe4c00700 1 -- 172.21.15.99:0/2365476884 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd40039e0 con 0x7fefe8074df0 2024-03-30T00:35:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.748+0000 7fefeec00700 1 -- 172.21.15.99:0/2365476884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe81a1270 con 0x7fefe8074df0 2024-03-30T00:35:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.750+0000 7fefeec00700 1 -- 172.21.15.99:0/2365476884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe8107900 con 0x7fefe8074df0 2024-03-30T00:35:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.751+0000 7fefe4c00700 1 -- 172.21.15.99:0/2365476884 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fefd400f800 con 0x7fefe8074df0 2024-03-30T00:35:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.753+0000 7fefe4c00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fefcc0a1120 0x7fefcc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.753+0000 7fefe4c00700 1 -- 172.21.15.99:0/2365476884 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fefd40cc540 con 0x7fefe8074df0 2024-03-30T00:35:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.753+0000 7fefe7e00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fefcc0a1120 0x7fefcc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.754+0000 7fefe7e00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fefcc0a1120 0x7fefcc0a35e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fefd8000c00 tx=0x7fefd8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:18.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:18.761+0000 7fefe4c00700 1 -- 172.21.15.99:0/2365476884 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fefd4095bf0 con 0x7fefe8074df0 2024-03-30T00:35:18.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:18 smithi142 ceph-mon[41779]: pgmap v321: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 7 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:18.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:18 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:35:18.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:18 smithi161 ceph-mon[42221]: pgmap v321: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 7 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:18.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:18 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:35:19.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.042+0000 7fefeec00700 1 -- 172.21.15.99:0/2365476884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefe8067a10 con 0x7fefe8074df0 2024-03-30T00:35:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.043+0000 7fefe4c00700 1 -- 172.21.15.99:0/2365476884 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10860 (secure 0 0 0) 0x7fefd4099210 con 0x7fefe8074df0 2024-03-30T00:35:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:35:19.050 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:34:39.366251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:35:19.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 -- 172.21.15.99:0/2365476884 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fefcc0a1120 msgr2=0x7fefcc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fefcc0a1120 0x7fefcc0a35e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fefd8000c00 tx=0x7fefd8005fb0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 -- 172.21.15.99:0/2365476884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe8074df0 msgr2=0x7fefe8199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe8074df0 0x7fefe8199250 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fefd400d980 tx=0x7fefd400dc90 comp rx=0 tx=0).stop 2024-03-30T00:35:19.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 -- 172.21.15.99:0/2365476884 shutdown_connections 2024-03-30T00:35:19.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fefcc0a1120 0x7fefcc0a35e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefe8108b40 0x7fefe81a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fefe8073810 0x7fefe8198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 --2- 172.21.15.99:0/2365476884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefe8074df0 0x7fefe8199250 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 -- 172.21.15.99:0/2365476884 >> 172.21.15.99:0/2365476884 conn(0x7fefe80fc1f0 msgr2=0x7fefe8104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 -- 172.21.15.99:0/2365476884 shutdown_connections 2024-03-30T00:35:19.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.046+0000 7fefeec00700 1 -- 172.21.15.99:0/2365476884 wait complete. 2024-03-30T00:35:19.054 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2024-03-30T00:35:19.074 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T00:35:19.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.273+0000 7f831f600700 1 -- 172.21.15.99:0/4138512792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8318100d80 msgr2=0x7f83181011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.273+0000 7f831f600700 1 --2- 172.21.15.99:0/4138512792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8318100d80 0x7f83181011a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f830c0099c0 tx=0x7f830c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.273+0000 7f831f600700 1 -- 172.21.15.99:0/4138512792 shutdown_connections 2024-03-30T00:35:19.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.273+0000 7f831f600700 1 --2- 172.21.15.99:0/4138512792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8318102910 0x7f83181091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.273+0000 7f831f600700 1 --2- 172.21.15.99:0/4138512792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8318100d80 0x7f83181011a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.273+0000 7f831f600700 1 --2- 172.21.15.99:0/4138512792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8318101f70 0x7f83181023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.273+0000 7f831f600700 1 -- 172.21.15.99:0/4138512792 >> 172.21.15.99:0/4138512792 conn(0x7f83180fc2f0 msgr2=0x7f83180fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.273+0000 7f831f600700 1 -- 172.21.15.99:0/4138512792 shutdown_connections 2024-03-30T00:35:19.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.274+0000 7f831f600700 1 -- 172.21.15.99:0/4138512792 wait complete. 2024-03-30T00:35:19.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.275+0000 7f831f600700 1 Processor -- start 2024-03-30T00:35:19.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.275+0000 7f831f600700 1 -- start start 2024-03-30T00:35:19.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.275+0000 7f831f600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8318100d80 0x7f8318196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f831f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8318101f70 0x7f83181971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f831f600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8318102910 0x7f831819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f831f600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831819e780 con 0x7f8318102910 2024-03-30T00:35:19.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f831f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831819e8f0 con 0x7f8318101f70 2024-03-30T00:35:19.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f831f600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831819eab0 con 0x7f8318100d80 2024-03-30T00:35:19.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f831d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8318102910 0x7f831819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f8317e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8318101f70 0x7f83181971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f831cc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8318100d80 0x7f8318196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f831d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8318102910 0x7f831819e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34258/0 (socket says 172.21.15.99:34258) 2024-03-30T00:35:19.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f831d600700 1 -- 172.21.15.99:0/2878939267 learned_addr learned my addr 172.21.15.99:0/2878939267 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f8317e00700 1 -- 172.21.15.99:0/2878939267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8318100d80 msgr2=0x7f8318196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f8317e00700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8318100d80 0x7f8318196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f8317e00700 1 -- 172.21.15.99:0/2878939267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8318102910 msgr2=0x7f831819e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f8317e00700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8318102910 0x7f831819e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f8317e00700 1 -- 172.21.15.99:0/2878939267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c0095f0 con 0x7f8318101f70 2024-03-30T00:35:19.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.276+0000 7f831d600700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8318102910 0x7f831819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:19.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.277+0000 7f8317e00700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8318101f70 0x7f83181971d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f830400e9f0 tx=0x7f830400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.277+0000 7f8315600700 1 -- 172.21.15.99:0/2878939267 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830400c9c0 con 0x7f8318101f70 2024-03-30T00:35:19.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.277+0000 7f831f600700 1 -- 172.21.15.99:0/2878939267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831819ed90 con 0x7f8318101f70 2024-03-30T00:35:19.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.277+0000 7f8315600700 1 -- 172.21.15.99:0/2878939267 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8304010430 con 0x7f8318101f70 2024-03-30T00:35:19.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.277+0000 7f8315600700 1 -- 172.21.15.99:0/2878939267 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304018680 con 0x7f8318101f70 2024-03-30T00:35:19.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.277+0000 7f831f600700 1 -- 172.21.15.99:0/2878939267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831819f200 con 0x7f8318101f70 2024-03-30T00:35:19.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.282+0000 7f8315600700 1 -- 172.21.15.99:0/2878939267 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f830400cb20 con 0x7f8318101f70 2024-03-30T00:35:19.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.282+0000 7f83330c2b80 1 Processor -- start 2024-03-30T00:35:19.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f83330c2b80 1 -- start start 2024-03-30T00:35:19.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7397f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 0x55c7397f3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f83330c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c739749990 con 0x55c7397f2fd0 2024-03-30T00:35:19.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f83330c2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c739747350 con 0x55c7397f2130 2024-03-30T00:35:19.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f83330c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c739742c40 con 0x55c7397f17d0 2024-03-30T00:35:19.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f831cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f831cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397f29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:35636/0 (socket says 172.21.15.99:35636) 2024-03-30T00:35:19.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f831d600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7397f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f831cc00700 1 -- 172.21.15.99:0/1169777421 learned_addr learned my addr 172.21.15.99:0/1169777421 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.283+0000 7f8317e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 0x55c7397f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.284+0000 7f831d600700 1 -- 172.21.15.99:0/1169777421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 msgr2=0x55c7397f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.284+0000 7f831d600700 1 --2- 172.21.15.99:0/1169777421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.284+0000 7f831d600700 1 -- 172.21.15.99:0/1169777421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 msgr2=0x55c7397f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.284+0000 7f831d600700 1 --2- 172.21.15.99:0/1169777421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 0x55c7397f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.284+0000 7f831d600700 1 -- 172.21.15.99:0/1169777421 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7395c43b0 con 0x55c7397f17d0 2024-03-30T00:35:19.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.284+0000 7f8317e00700 1 --2- 172.21.15.99:0/1169777421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 0x55c7397f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:19.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.284+0000 7f8315600700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f83000a1120 0x7f83000a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.284+0000 7f8315600700 1 -- 172.21.15.99:0/2878939267 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f8304014070 con 0x7f8318101f70 2024-03-30T00:35:19.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.284+0000 7f831d600700 1 --2- 172.21.15.99:0/1169777421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7397f1bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8310009430 tx=0x7f8310009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9a88f2b6bff70b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.285+0000 7f831cc00700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f83000a1120 0x7f83000a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.285+0000 7f8309c00700 1 -- 172.21.15.99:0/1169777421 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310004db0 con 0x55c7397f17d0 2024-03-30T00:35:19.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.285+0000 7f8309c00700 1 -- 172.21.15.99:0/1169777421 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f831000f040 con 0x55c7397f17d0 2024-03-30T00:35:19.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.285+0000 7f8309c00700 1 -- 172.21.15.99:0/1169777421 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310005350 con 0x55c7397f17d0 2024-03-30T00:35:19.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.286+0000 7f83330c2b80 1 -- 172.21.15.99:0/1169777421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 msgr2=0x55c7397f1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.286+0000 7f831cc00700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f83000a1120 0x7f83000a35e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f830c00b580 tx=0x7f830c003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.286+0000 7f83330c2b80 1 --2- 172.21.15.99:0/1169777421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7397f1bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8310009430 tx=0x7f8310009920 comp rx=0 tx=0).stop 2024-03-30T00:35:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.286+0000 7f83330c2b80 1 -- 172.21.15.99:0/1169777421 shutdown_connections 2024-03-30T00:35:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.286+0000 7f83330c2b80 1 --2- 172.21.15.99:0/1169777421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 0x55c7397f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.286+0000 7f83330c2b80 1 --2- 172.21.15.99:0/1169777421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7397f1bf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.286+0000 7f83330c2b80 1 --2- 172.21.15.99:0/1169777421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.286+0000 7f83330c2b80 1 -- 172.21.15.99:0/1169777421 >> 172.21.15.99:0/1169777421 conn(0x55c7397506a0 msgr2=0x55c7397ec9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.286+0000 7f83330c2b80 1 -- 172.21.15.99:0/1169777421 shutdown_connections 2024-03-30T00:35:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.286+0000 7f83330c2b80 1 -- 172.21.15.99:0/1169777421 wait complete. 2024-03-30T00:35:19.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.287+0000 7f83330c2b80 1 -- start start 2024-03-30T00:35:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.287+0000 7f83330c2b80 1 Processor -- start 2024-03-30T00:35:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.287+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7397ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397fff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 0x55c739806fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f83330c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7396ded80 con 0x55c7397f2fd0 2024-03-30T00:35:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f83330c2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7396dea20 con 0x55c7397f2130 2024-03-30T00:35:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f83330c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7395c66a0 con 0x55c7397f17d0 2024-03-30T00:35:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f83330c2b80 10 client.0 _ll_register_callbacks cb 0x55c7396e8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f831d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 0x55c739806fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f8317e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f831d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 0x55c739806fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34280/0 (socket says 172.21.15.99:34280) 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f831d600700 1 -- 172.21.15.99:0/1722159002 learned_addr learned my addr 172.21.15.99:0/1722159002 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f831cc00700 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7397ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f831d600700 1 -- 172.21.15.99:0/1722159002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 msgr2=0x55c7397ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f831d600700 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7397ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f831d600700 1 -- 172.21.15.99:0/1722159002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 msgr2=0x55c7397fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f831d600700 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f831d600700 1 -- 172.21.15.99:0/1722159002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831819f200 con 0x55c7397f2fd0 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.288+0000 7f8317e00700 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.289+0000 7f831d600700 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 0x55c739806fc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f831000b010 tx=0x7f831000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acdb2e122008dd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.289+0000 7f82fe000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.289+0000 7f82fe000700 1 -- 172.21.15.99:0/1722159002 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310022070 con 0x55c7397f2fd0 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.289+0000 7f83330c2b80 1 -- 172.21.15.99:0/1722159002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c739703380 con 0x55c7397f2fd0 2024-03-30T00:35:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.289+0000 7f82fe000700 1 -- 172.21.15.99:0/1722159002 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f831000e940 con 0x55c7397f2fd0 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.289+0000 7f82fe000700 1 -- 172.21.15.99:0/1722159002 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f831000eb40 con 0x55c7397f2fd0 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.289+0000 7f83330c2b80 10 client.15561 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.289+0000 7f83330c2b80 1 -- 172.21.15.99:0/1722159002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f8310009110 con 0x55c7397f2fd0 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.289+0000 7f82fe000700 1 -- 172.21.15.99:0/1722159002 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8310036080 con 0x55c7397f2fd0 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.290+0000 7f82fe000700 1 -- 172.21.15.99:0/1722159002 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8310028540 con 0x55c7397f2fd0 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.290+0000 7f83330c2b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.290+0000 7f83330c2b80 10 client.15561 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.290+0000 7f83330c2b80 20 client.15561 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.290+0000 7f83330c2b80 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55c73980b7b0 0x55c73980dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.290+0000 7f83330c2b80 4 client.15561 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.290+0000 7f83330c2b80 1 -- 172.21.15.99:0/1722159002 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c73973b700 con 0x55c73980b7b0 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.290+0000 7f831cc00700 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55c73980b7b0 0x55c73980dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.291+0000 7f82fe000700 10 client.15561 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:35:19.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.291+0000 7f831cc00700 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55c73980b7b0 0x55c73980dbd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a9c61d297455812 server_cookie=d10bf7f10d596595 in_seq=0 out_seq=0 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.291+0000 7f82fe000700 1 -- 172.21.15.99:0/1722159002 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c73973b700 con 0x55c73980b7b0 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.291+0000 7f82fe000700 10 client.15561 handle_command_reply: tid=0 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 1 client.15561 shutdown 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 1 -- 172.21.15.99:0/1722159002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 msgr2=0x55c739806fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 0x55c739806fc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f831000b010 tx=0x7f831000e720 comp rx=0 tx=0).stop 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 1 -- 172.21.15.99:0/1722159002 shutdown_connections 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55c73980b7b0 0x55c73980dbd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2fd0 0x55c739806fc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7397ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f82fe000700 0 client.15561 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 1 --2- 172.21.15.99:0/1722159002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 1 -- 172.21.15.99:0/1722159002 >> 172.21.15.99:0/1722159002 conn(0x55c7397506a0 msgr2=0x55c7397f0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 1 -- 172.21.15.99:0/1722159002 shutdown_connections 2024-03-30T00:35:19.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 1 -- 172.21.15.99:0/1722159002 wait complete. 2024-03-30T00:35:19.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.292+0000 7f83330c2b80 20 client.15561 trim_cache size 0 max 16384 2024-03-30T00:35:19.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.309+0000 7f83330c2b80 1 Processor -- start 2024-03-30T00:35:19.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.309+0000 7f83330c2b80 1 -- start start 2024-03-30T00:35:19.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.309+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f17d0 0x55c7397f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.309+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.309+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f2fd0 0x55c739806f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.309+0000 7f83330c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7397f91d0 con 0x55c7397f17d0 2024-03-30T00:35:19.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.309+0000 7f83330c2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73980b430 con 0x55c7397f2130 2024-03-30T00:35:19.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.309+0000 7f83330c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7397e8480 con 0x55c7397f2fd0 2024-03-30T00:35:19.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.310+0000 7f831cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f17d0 0x55c7397f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.310+0000 7f8317e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.310+0000 7f8317e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397fab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:35658/0 (socket says 172.21.15.99:35658) 2024-03-30T00:35:19.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.310+0000 7f831d600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f2fd0 0x55c739806f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.310+0000 7f8317e00700 1 -- 172.21.15.99:0/3767583288 learned_addr learned my addr 172.21.15.99:0/3767583288 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.310+0000 7f8317e00700 1 -- 172.21.15.99:0/3767583288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f2fd0 msgr2=0x55c739806f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.310+0000 7f8317e00700 1 --2- 172.21.15.99:0/3767583288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f2fd0 0x55c739806f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.310+0000 7f8317e00700 1 -- 172.21.15.99:0/3767583288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f17d0 msgr2=0x55c7397f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.310+0000 7f8317e00700 1 --2- 172.21.15.99:0/3767583288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f17d0 0x55c7397f1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.310+0000 7f8317e00700 1 -- 172.21.15.99:0/3767583288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c739570b60 con 0x55c7397f2130 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.311+0000 7f8317e00700 1 --2- 172.21.15.99:0/3767583288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397fab40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8304096330 tx=0x7f830409a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa54fab5d8df6d86 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.311+0000 7f830a600700 1 -- 172.21.15.99:0/3767583288 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304096170 con 0x55c7397f2130 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.311+0000 7f830a600700 1 -- 172.21.15.99:0/3767583288 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8304099c00 con 0x55c7397f2130 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.311+0000 7f830a600700 1 -- 172.21.15.99:0/3767583288 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830409a530 con 0x55c7397f2130 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.311+0000 7f83330c2b80 1 -- 172.21.15.99:0/3767583288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 msgr2=0x55c7397fab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.311+0000 7f83330c2b80 1 --2- 172.21.15.99:0/3767583288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397fab40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8304096330 tx=0x7f830409a370 comp rx=0 tx=0).stop 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.312+0000 7f83330c2b80 1 -- 172.21.15.99:0/3767583288 shutdown_connections 2024-03-30T00:35:19.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.312+0000 7f83330c2b80 1 --2- 172.21.15.99:0/3767583288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f2fd0 0x55c739806f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.312+0000 7f83330c2b80 1 --2- 172.21.15.99:0/3767583288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f17d0 0x55c7397f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.312+0000 7f83330c2b80 1 --2- 172.21.15.99:0/3767583288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2130 0x55c7397fab40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.312+0000 7f83330c2b80 1 -- 172.21.15.99:0/3767583288 >> 172.21.15.99:0/3767583288 conn(0x55c7397506a0 msgr2=0x55c7397e55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.312+0000 7f83330c2b80 1 -- 172.21.15.99:0/3767583288 shutdown_connections 2024-03-30T00:35:19.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.312+0000 7f83330c2b80 1 -- 172.21.15.99:0/3767583288 wait complete. 2024-03-30T00:35:19.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.312+0000 7f83330c2b80 1 -- start start 2024-03-30T00:35:19.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.312+0000 7f83330c2b80 1 Processor -- start 2024-03-30T00:35:19.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7398051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2130 0x55c7398056e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f83330c2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2fd0 0x55c7398065b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f83330c2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c739806af0 con 0x55c7397f2130 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f83330c2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c739806c60 con 0x55c7397f2fd0 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f83330c2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c739806e20 con 0x55c7397f17d0 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f83330c2b80 10 client.0 _ll_register_callbacks cb 0x55c7396e8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f8317e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2130 0x55c7398056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f8317e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2130 0x55c7398056e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34296/0 (socket says 172.21.15.99:34296) 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f8317e00700 1 -- 172.21.15.99:0/2059537112 learned_addr learned my addr 172.21.15.99:0/2059537112 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f831cc00700 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7398051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f831d600700 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2fd0 0x55c7398065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f8317e00700 1 -- 172.21.15.99:0/2059537112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 msgr2=0x55c7398051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f8317e00700 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7398051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f8317e00700 1 -- 172.21.15.99:0/2059537112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2fd0 msgr2=0x55c7398065b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f8317e00700 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2fd0 0x55c7398065b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f8317e00700 1 -- 172.21.15.99:0/2059537112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831819ed90 con 0x55c7397f2130 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f8317e00700 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2130 0x55c7398056e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f830400cfd0 tx=0x7f83040d2370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91c8c2abdfda290 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.313+0000 7f82fe000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f82fe000700 1 -- 172.21.15.99:0/2059537112 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304099790 con 0x55c7397f2130 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f82fe000700 1 -- 172.21.15.99:0/2059537112 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f830409ae40 con 0x55c7397f2130 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f83330c2b80 1 -- 172.21.15.99:0/2059537112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f831000f040 con 0x55c7397f2130 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f82fe000700 1 -- 172.21.15.99:0/2059537112 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8304010880 con 0x55c7397f2130 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f83330c2b80 10 client.15567 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f83330c2b80 1 -- 172.21.15.99:0/2059537112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c739708ea0 con 0x55c7397f2130 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f82fe000700 1 -- 172.21.15.99:0/2059537112 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f83040b6020 con 0x55c7397f2130 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f82fe000700 1 -- 172.21.15.99:0/2059537112 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8304098d90 con 0x55c7397f2130 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f83330c2b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:19.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f83330c2b80 10 client.15567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f83330c2b80 20 client.15567 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f83330c2b80 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55c7398604e0 0x55c739862900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.314+0000 7f83330c2b80 4 client.15567 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.315+0000 7f83330c2b80 1 -- 172.21.15.99:0/2059537112 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c739591cf0 con 0x55c7398604e0 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.315+0000 7f831cc00700 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55c7398604e0 0x55c739862900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.315+0000 7f82fe000700 10 client.15567 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.315+0000 7f831cc00700 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55c7398604e0 0x55c739862900 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d2e4ecb21b64f81 server_cookie=1c8d6ae59e94f76d in_seq=0 out_seq=0 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f82fe000700 1 -- 172.21.15.99:0/2059537112 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c739591cf0 con 0x55c7398604e0 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f82fe000700 10 client.15567 handle_command_reply: tid=0 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 1 client.15567 shutdown 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 1 -- 172.21.15.99:0/2059537112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2130 msgr2=0x55c7398056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2130 0x55c7398056e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f830400cfd0 tx=0x7f83040d2370 comp rx=0 tx=0).stop 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 1 -- 172.21.15.99:0/2059537112 shutdown_connections 2024-03-30T00:35:19.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55c7398604e0 0x55c739862900 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7397f2fd0 0x55c7398065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c7397f17d0 0x55c7398051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 1 --2- 172.21.15.99:0/2059537112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7397f2130 0x55c7398056e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 1 -- 172.21.15.99:0/2059537112 >> 172.21.15.99:0/2059537112 conn(0x55c7397506a0 msgr2=0x55c7397f61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f82fe000700 0 client.15567 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:35:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 1 -- 172.21.15.99:0/2059537112 shutdown_connections 2024-03-30T00:35:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 1 -- 172.21.15.99:0/2059537112 wait complete. 2024-03-30T00:35:19.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.316+0000 7f83330c2b80 20 client.15567 trim_cache size 0 max 16384 2024-03-30T00:35:19.327 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:35:19.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 -- 172.21.15.99:0/2878939267 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f83000a1120 msgr2=0x7f83000a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f83000a1120 0x7f83000a35e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f830c00b580 tx=0x7f830c003680 comp rx=0 tx=0).stop 2024-03-30T00:35:19.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 -- 172.21.15.99:0/2878939267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8318101f70 msgr2=0x7f83181971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8318101f70 0x7f83181971d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f830400e9f0 tx=0x7f830400ed00 comp rx=0 tx=0).stop 2024-03-30T00:35:19.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 -- 172.21.15.99:0/2878939267 shutdown_connections 2024-03-30T00:35:19.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8318102910 0x7f831819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8318100d80 0x7f8318196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f83000a1120 0x7f83000a35e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 --2- 172.21.15.99:0/2878939267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8318101f70 0x7f83181971d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 -- 172.21.15.99:0/2878939267 >> 172.21.15.99:0/2878939267 conn(0x7f83180fc2f0 msgr2=0x7f8318105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 -- 172.21.15.99:0/2878939267 shutdown_connections 2024-03-30T00:35:19.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.317+0000 7f831f600700 1 -- 172.21.15.99:0/2878939267 wait complete. 2024-03-30T00:35:19.342 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T00:35:19.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.541+0000 7fc171800700 1 -- 172.21.15.99:0/3139789052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc16c100d80 msgr2=0x7fc16c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.541+0000 7fc171800700 1 --2- 172.21.15.99:0/3139789052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc16c100d80 0x7fc16c1011a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0099c0 tx=0x7fc15c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.541+0000 7fc171800700 1 -- 172.21.15.99:0/3139789052 shutdown_connections 2024-03-30T00:35:19.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.541+0000 7fc171800700 1 --2- 172.21.15.99:0/3139789052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16c102910 0x7fc16c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.541+0000 7fc171800700 1 --2- 172.21.15.99:0/3139789052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc16c100d80 0x7fc16c1011a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.541+0000 7fc171800700 1 --2- 172.21.15.99:0/3139789052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc16c101f70 0x7fc16c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.541+0000 7fc171800700 1 -- 172.21.15.99:0/3139789052 >> 172.21.15.99:0/3139789052 conn(0x7fc16c0fc2f0 msgr2=0x7fc16c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.541+0000 7fc171800700 1 -- 172.21.15.99:0/3139789052 shutdown_connections 2024-03-30T00:35:19.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.541+0000 7fc171800700 1 -- 172.21.15.99:0/3139789052 wait complete. 2024-03-30T00:35:19.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.542+0000 7fc171800700 1 Processor -- start 2024-03-30T00:35:19.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.542+0000 7fc171800700 1 -- start start 2024-03-30T00:35:19.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc171800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16c100d80 0x7fc16c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc171800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc16c101f70 0x7fc16c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc171800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc16c102910 0x7fc16c1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc171800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c1a0a00 con 0x7fc16c101f70 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc171800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c1a0b40 con 0x7fc16c102910 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc171800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c1a0cd0 con 0x7fc16c100d80 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16aa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16c100d80 0x7fc16c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc16c101f70 0x7fc16c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16aa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16c100d80 0x7fc16c198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55598/0 (socket says 172.21.15.99:55598) 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc16c101f70 0x7fc16c199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34314/0 (socket says 172.21.15.99:34314) 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16aa00700 1 -- 172.21.15.99:0/3612976808 learned_addr learned my addr 172.21.15.99:0/3612976808 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16b400700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc16c102910 0x7fc16c1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16a000700 1 -- 172.21.15.99:0/3612976808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16c100d80 msgr2=0x7fc16c198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16a000700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16c100d80 0x7fc16c198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16a000700 1 -- 172.21.15.99:0/3612976808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc16c102910 msgr2=0x7fc16c1a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16a000700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc16c102910 0x7fc16c1a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16a000700 1 -- 172.21.15.99:0/3612976808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c0095f0 con 0x7fc16c101f70 2024-03-30T00:35:19.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.543+0000 7fc16aa00700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16c100d80 0x7fc16c198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:19.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.544+0000 7fc16a000700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc16c101f70 0x7fc16c199410 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc15800e9f0 tx=0x7fc15800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.544+0000 7fc163400700 1 -- 172.21.15.99:0/3612976808 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15800c9c0 con 0x7fc16c101f70 2024-03-30T00:35:19.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.544+0000 7fc163400700 1 -- 172.21.15.99:0/3612976808 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc158010430 con 0x7fc16c101f70 2024-03-30T00:35:19.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.544+0000 7fc163400700 1 -- 172.21.15.99:0/3612976808 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158018680 con 0x7fc16c101f70 2024-03-30T00:35:19.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.544+0000 7fc171800700 1 -- 172.21.15.99:0/3612976808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c1a0fb0 con 0x7fc16c101f70 2024-03-30T00:35:19.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.544+0000 7fc171800700 1 -- 172.21.15.99:0/3612976808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c1a13e0 con 0x7fc16c101f70 2024-03-30T00:35:19.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.549+0000 7fc18527eb80 1 Processor -- start 2024-03-30T00:35:19.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.549+0000 7fc163400700 1 -- 172.21.15.99:0/3612976808 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc1580105a0 con 0x7fc16c101f70 2024-03-30T00:35:19.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.549+0000 7fc18527eb80 1 -- start start 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f6cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6d130 0x55bec0f6d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6dfd0 0x55bec0f6e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc18527eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0ec4990 con 0x55bec0f6dfd0 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc18527eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0ec2350 con 0x55bec0f6d130 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc18527eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0ebdc40 con 0x55bec0f6c7d0 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc16b400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f6cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc16b400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f6cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55608/0 (socket says 172.21.15.99:55608) 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc16b400700 1 -- 172.21.15.99:0/2982532724 learned_addr learned my addr 172.21.15.99:0/2982532724 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc16aa00700 1 --2- 172.21.15.99:0/2982532724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6d130 0x55bec0f6d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc16a000700 1 --2- 172.21.15.99:0/2982532724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6dfd0 0x55bec0f6e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc16b400700 1 -- 172.21.15.99:0/2982532724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6d130 msgr2=0x55bec0f6d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc16b400700 1 --2- 172.21.15.99:0/2982532724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6d130 0x55bec0f6d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc16b400700 1 -- 172.21.15.99:0/2982532724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6dfd0 msgr2=0x55bec0f6e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc16b400700 1 --2- 172.21.15.99:0/2982532724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6dfd0 0x55bec0f6e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc16b400700 1 -- 172.21.15.99:0/2982532724 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bec0d3f3b0 con 0x55bec0f6c7d0 2024-03-30T00:35:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.550+0000 7fc16a000700 1 --2- 172.21.15.99:0/2982532724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6dfd0 0x55bec0f6e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:35:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.551+0000 7fc16b400700 1 --2- 172.21.15.99:0/2982532724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f6cbf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc164011680 tx=0x7fc164009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7294a2452e01f4b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.551+0000 7fc157e00700 1 -- 172.21.15.99:0/2982532724 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16400e040 con 0x55bec0f6c7d0 2024-03-30T00:35:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.551+0000 7fc163400700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1500a1120 0x7fc1500a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.551+0000 7fc157e00700 1 -- 172.21.15.99:0/2982532724 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc16400f040 con 0x55bec0f6c7d0 2024-03-30T00:35:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.551+0000 7fc157e00700 1 -- 172.21.15.99:0/2982532724 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164004730 con 0x55bec0f6c7d0 2024-03-30T00:35:19.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.551+0000 7fc163400700 1 -- 172.21.15.99:0/3612976808 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fc158014070 con 0x7fc16c101f70 2024-03-30T00:35:19.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.551+0000 7fc16aa00700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1500a1120 0x7fc1500a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.552+0000 7fc18527eb80 1 -- 172.21.15.99:0/2982532724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 msgr2=0x55bec0f6cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.552+0000 7fc18527eb80 1 --2- 172.21.15.99:0/2982532724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f6cbf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc164011680 tx=0x7fc164009920 comp rx=0 tx=0).stop 2024-03-30T00:35:19.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.552+0000 7fc18527eb80 1 -- 172.21.15.99:0/2982532724 shutdown_connections 2024-03-30T00:35:19.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.552+0000 7fc18527eb80 1 --2- 172.21.15.99:0/2982532724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6dfd0 0x55bec0f6e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.552+0000 7fc18527eb80 1 --2- 172.21.15.99:0/2982532724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f6cbf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.552+0000 7fc18527eb80 1 --2- 172.21.15.99:0/2982532724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6d130 0x55bec0f6d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.552+0000 7fc18527eb80 1 -- 172.21.15.99:0/2982532724 >> 172.21.15.99:0/2982532724 conn(0x55bec0ecb6a0 msgr2=0x55bec0f679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.552+0000 7fc16aa00700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1500a1120 0x7fc1500a35e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc15c006010 tx=0x7fc15c005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.552+0000 7fc18527eb80 1 -- 172.21.15.99:0/2982532724 shutdown_connections 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.552+0000 7fc18527eb80 1 -- 172.21.15.99:0/2982532724 wait complete. 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.553+0000 7fc18527eb80 1 -- start start 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.553+0000 7fc18527eb80 1 Processor -- start 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f76630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f76b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f7dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc18527eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0e59d80 con 0x55bec0f6d130 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc18527eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0e59a20 con 0x55bec0f6dfd0 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc18527eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0d416a0 con 0x55bec0f6c7d0 2024-03-30T00:35:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc18527eb80 10 client.0 _ll_register_callbacks cb 0x55bec0e63470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:19.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc16aa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f76630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc16aa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f76630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55610/0 (socket says 172.21.15.99:55610) 2024-03-30T00:35:19.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc16aa00700 1 -- 172.21.15.99:0/2881103107 learned_addr learned my addr 172.21.15.99:0/2881103107 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc16a000700 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f76b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.554+0000 7fc16b400700 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f7dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.555+0000 7fc16aa00700 1 -- 172.21.15.99:0/2881103107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 msgr2=0x55bec0f7dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.555+0000 7fc16aa00700 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f7dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.555+0000 7fc16aa00700 1 -- 172.21.15.99:0/2881103107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 msgr2=0x55bec0f76b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.555+0000 7fc16aa00700 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f76b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.555+0000 7fc16aa00700 1 -- 172.21.15.99:0/2881103107 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc164011330 con 0x55bec0f6c7d0 2024-03-30T00:35:19.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.555+0000 7fc16aa00700 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f76630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00bf10 tx=0x7fc15c01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd303a18ddaf6c12 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.555+0000 7fc154c00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:35:19.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.555+0000 7fc154c00700 1 -- 172.21.15.99:0/2881103107 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c02c070 con 0x55bec0f6c7d0 2024-03-30T00:35:19.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.555+0000 7fc154c00700 1 -- 172.21.15.99:0/2881103107 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fc15c003d00 con 0x55bec0f6c7d0 2024-03-30T00:35:19.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.555+0000 7fc18527eb80 1 -- 172.21.15.99:0/2881103107 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bec0e7e380 con 0x55bec0f6c7d0 2024-03-30T00:35:19.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc154c00700 1 -- 172.21.15.99:0/2881103107 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc15c00f460 con 0x55bec0f6c7d0 2024-03-30T00:35:19.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc18527eb80 10 client.25303 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:19.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc18527eb80 1 -- 172.21.15.99:0/2881103107 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bec0cebb60 con 0x55bec0f6c7d0 2024-03-30T00:35:19.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc154c00700 1 -- 172.21.15.99:0/2881103107 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bec0e7e380 con 0x55bec0f6c7d0 2024-03-30T00:35:19.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc154c00700 1 -- 172.21.15.99:0/2881103107 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc15c025540 con 0x55bec0f6c7d0 2024-03-30T00:35:19.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc18527eb80 10 client.25303 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:19.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc18527eb80 10 client.25303 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-30T00:35:19.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc18527eb80 20 client.25303 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:19.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc18527eb80 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55bec0f82360 0x55bec0f84780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc18527eb80 4 client.25303 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:19.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc18527eb80 1 -- 172.21.15.99:0/2881103107 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bec0eb6700 con 0x55bec0f82360 2024-03-30T00:35:19.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.556+0000 7fc16a000700 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55bec0f82360 0x55bec0f84780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.557+0000 7fc16a000700 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55bec0f82360 0x55bec0f84780 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f87c1cc7d69e7ad5 server_cookie=a054f174501ed92e in_seq=0 out_seq=0 2024-03-30T00:35:19.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.557+0000 7fc154c00700 10 client.25303 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:35:19.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc154c00700 1 -- 172.21.15.99:0/2881103107 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bec0eb6700 con 0x55bec0f82360 2024-03-30T00:35:19.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc154c00700 10 client.25303 handle_command_reply: tid=0 2024-03-30T00:35:19.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 1 client.25303 shutdown 2024-03-30T00:35:19.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 1 -- 172.21.15.99:0/2881103107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 msgr2=0x55bec0f76630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f76630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00bf10 tx=0x7fc15c01a040 comp rx=0 tx=0).stop 2024-03-30T00:35:19.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 1 -- 172.21.15.99:0/2881103107 shutdown_connections 2024-03-30T00:35:19.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55bec0f82360 0x55bec0f84780 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f7dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f76630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 1 --2- 172.21.15.99:0/2881103107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f76b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc154c00700 0 client.25303 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:35:19.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 1 -- 172.21.15.99:0/2881103107 >> 172.21.15.99:0/2881103107 conn(0x55bec0ecb6a0 msgr2=0x55bec0f6b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 1 -- 172.21.15.99:0/2881103107 shutdown_connections 2024-03-30T00:35:19.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 1 -- 172.21.15.99:0/2881103107 wait complete. 2024-03-30T00:35:19.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.558+0000 7fc18527eb80 20 client.25303 trim_cache size 0 max 16384 2024-03-30T00:35:19.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc18527eb80 1 Processor -- start 2024-03-30T00:35:19.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc18527eb80 1 -- start start 2024-03-30T00:35:19.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f7e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f7db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f6e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc18527eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0f7ed60 con 0x55bec0f6d130 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc18527eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0f6cf70 con 0x55bec0f6dfd0 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc18527eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0f77820 con 0x55bec0f6c7d0 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc16a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f7db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc16aa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f7e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc16a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f7db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34352/0 (socket says 172.21.15.99:34352) 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc16a000700 1 -- 172.21.15.99:0/279576059 learned_addr learned my addr 172.21.15.99:0/279576059 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.572+0000 7fc16b400700 1 --2- 172.21.15.99:0/279576059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f6e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.573+0000 7fc16aa00700 1 -- 172.21.15.99:0/279576059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 msgr2=0x55bec0f6e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.573+0000 7fc16aa00700 1 --2- 172.21.15.99:0/279576059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f6e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.573+0000 7fc16aa00700 1 -- 172.21.15.99:0/279576059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 msgr2=0x55bec0f7db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.573+0000 7fc16aa00700 1 --2- 172.21.15.99:0/279576059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f7db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.573+0000 7fc16aa00700 1 -- 172.21.15.99:0/279576059 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bec0e83ea0 con 0x55bec0f6c7d0 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.573+0000 7fc16aa00700 1 --2- 172.21.15.99:0/279576059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f7e260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009990 tx=0x7fc15c025af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9527c8ee795798d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.573+0000 7fc160c00700 1 -- 172.21.15.99:0/279576059 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c02c020 con 0x55bec0f6c7d0 2024-03-30T00:35:19.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.573+0000 7fc160c00700 1 -- 172.21.15.99:0/279576059 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc15c025bc0 con 0x55bec0f6c7d0 2024-03-30T00:35:19.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.573+0000 7fc160c00700 1 -- 172.21.15.99:0/279576059 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c00fd60 con 0x55bec0f6c7d0 2024-03-30T00:35:19.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.574+0000 7fc18527eb80 1 -- 172.21.15.99:0/279576059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 msgr2=0x55bec0f7e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.574+0000 7fc18527eb80 1 --2- 172.21.15.99:0/279576059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f7e260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009990 tx=0x7fc15c025af0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.574+0000 7fc18527eb80 1 -- 172.21.15.99:0/279576059 shutdown_connections 2024-03-30T00:35:19.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.574+0000 7fc18527eb80 1 --2- 172.21.15.99:0/279576059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f6e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.574+0000 7fc18527eb80 1 --2- 172.21.15.99:0/279576059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f7e260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.574+0000 7fc18527eb80 1 --2- 172.21.15.99:0/279576059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f7db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.574+0000 7fc18527eb80 1 -- 172.21.15.99:0/279576059 >> 172.21.15.99:0/279576059 conn(0x55bec0ecb6a0 msgr2=0x55bec0f605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.574+0000 7fc18527eb80 1 -- 172.21.15.99:0/279576059 shutdown_connections 2024-03-30T00:35:19.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.574+0000 7fc18527eb80 1 -- 172.21.15.99:0/279576059 wait complete. 2024-03-30T00:35:19.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.575+0000 7fc18527eb80 1 -- start start 2024-03-30T00:35:19.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.575+0000 7fc18527eb80 1 Processor -- start 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.575+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f7bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.575+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f7c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.575+0000 7fc18527eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f7d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.575+0000 7fc18527eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0f7d630 con 0x55bec0f6d130 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.575+0000 7fc18527eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0f7d7a0 con 0x55bec0f6dfd0 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.575+0000 7fc18527eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bec0f7d910 con 0x55bec0f6c7d0 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.575+0000 7fc18527eb80 10 client.0 _ll_register_callbacks cb 0x55bec0e63470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f7c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16aa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f7bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16aa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f7bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55624/0 (socket says 172.21.15.99:55624) 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16aa00700 1 -- 172.21.15.99:0/3295264612 learned_addr learned my addr 172.21.15.99:0/3295264612 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f7c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34358/0 (socket says 172.21.15.99:34358) 2024-03-30T00:35:19.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16b400700 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f7d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16aa00700 1 -- 172.21.15.99:0/3295264612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 msgr2=0x55bec0f7d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16aa00700 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f7d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16aa00700 1 -- 172.21.15.99:0/3295264612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 msgr2=0x55bec0f7c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16aa00700 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f7c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16aa00700 1 -- 172.21.15.99:0/3295264612 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bec0cebb60 con 0x55bec0f6c7d0 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc16aa00700 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f7bbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc15c003ac0 tx=0x7fc15c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5eb0e213e30b683 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.576+0000 7fc154c00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.577+0000 7fc154c00700 1 -- 172.21.15.99:0/3295264612 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c02c030 con 0x55bec0f6c7d0 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.577+0000 7fc154c00700 1 -- 172.21.15.99:0/3295264612 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fc15c045460 con 0x55bec0f6c7d0 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.577+0000 7fc18527eb80 1 -- 172.21.15.99:0/3295264612 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc16400f040 con 0x55bec0f6c7d0 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.577+0000 7fc154c00700 1 -- 172.21.15.99:0/3295264612 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc15c00fe70 con 0x55bec0f6c7d0 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.577+0000 7fc18527eb80 10 client.25309 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.577+0000 7fc18527eb80 1 -- 172.21.15.99:0/3295264612 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bec0d43cd0 con 0x55bec0f6c7d0 2024-03-30T00:35:19.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.577+0000 7fc154c00700 1 -- 172.21.15.99:0/3295264612 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc15c053020 con 0x55bec0f6c7d0 2024-03-30T00:35:19.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.577+0000 7fc154c00700 1 -- 172.21.15.99:0/3295264612 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc15c04f760 con 0x55bec0f6c7d0 2024-03-30T00:35:19.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.577+0000 7fc18527eb80 10 client.25309 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:19.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.577+0000 7fc18527eb80 10 client.25309 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-30T00:35:19.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.577+0000 7fc18527eb80 20 client.25309 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:19.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.578+0000 7fc18527eb80 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55bec0fd6ee0 0x55bec0fd92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.578+0000 7fc18527eb80 4 client.25309 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:35:19.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.578+0000 7fc18527eb80 1 -- 172.21.15.99:0/3295264612 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bec0d0ccf0 con 0x55bec0fd6ee0 2024-03-30T00:35:19.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.578+0000 7fc16a000700 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55bec0fd6ee0 0x55bec0fd92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.578+0000 7fc16a000700 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55bec0fd6ee0 0x55bec0fd92d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9aa13efd35da57ef server_cookie=902b15c1b898addb in_seq=0 out_seq=0 2024-03-30T00:35:19.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.578+0000 7fc154c00700 10 client.25309 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:35:19.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.578+0000 7fc154c00700 1 -- 172.21.15.99:0/3295264612 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bec0d0ccf0 con 0x55bec0fd6ee0 2024-03-30T00:35:19.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.578+0000 7fc154c00700 10 client.25309 handle_command_reply: tid=0 2024-03-30T00:35:19.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.578+0000 7fc18527eb80 1 client.25309 shutdown 2024-03-30T00:35:19.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc18527eb80 1 -- 172.21.15.99:0/3295264612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 msgr2=0x55bec0f7bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc18527eb80 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f7bbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc15c003ac0 tx=0x7fc15c00f690 comp rx=0 tx=0).stop 2024-03-30T00:35:19.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc18527eb80 1 -- 172.21.15.99:0/3295264612 shutdown_connections 2024-03-30T00:35:19.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc18527eb80 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55bec0fd6ee0 0x55bec0fd92d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc18527eb80 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bec0f6dfd0 0x55bec0f7d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc18527eb80 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bec0f6c7d0 0x55bec0f7bbf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc154c00700 0 client.25309 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:35:19.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc18527eb80 1 --2- 172.21.15.99:0/3295264612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bec0f6d130 0x55bec0f7c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc18527eb80 1 -- 172.21.15.99:0/3295264612 >> 172.21.15.99:0/3295264612 conn(0x55bec0ecb6a0 msgr2=0x55bec0f711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc18527eb80 1 -- 172.21.15.99:0/3295264612 shutdown_connections 2024-03-30T00:35:19.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc18527eb80 1 -- 172.21.15.99:0/3295264612 wait complete. 2024-03-30T00:35:19.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.579+0000 7fc18527eb80 20 client.25309 trim_cache size 0 max 16384 2024-03-30T00:35:19.591 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:35:19.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 -- 172.21.15.99:0/3612976808 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1500a1120 msgr2=0x7fc1500a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1500a1120 0x7fc1500a35e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc15c006010 tx=0x7fc15c005040 comp rx=0 tx=0).stop 2024-03-30T00:35:19.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 -- 172.21.15.99:0/3612976808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc16c101f70 msgr2=0x7fc16c199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc16c101f70 0x7fc16c199410 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc15800e9f0 tx=0x7fc15800ed00 comp rx=0 tx=0).stop 2024-03-30T00:35:19.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 -- 172.21.15.99:0/3612976808 shutdown_connections 2024-03-30T00:35:19.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1500a1120 0x7fc1500a35e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc16c102910 0x7fc16c1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc16c100d80 0x7fc16c198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 --2- 172.21.15.99:0/3612976808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc16c101f70 0x7fc16c199410 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 -- 172.21.15.99:0/3612976808 >> 172.21.15.99:0/3612976808 conn(0x7fc16c0fc2f0 msgr2=0x7fc16c107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 -- 172.21.15.99:0/3612976808 shutdown_connections 2024-03-30T00:35:19.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.580+0000 7fc171800700 1 -- 172.21.15.99:0/3612976808 wait complete. 2024-03-30T00:35:19.602 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:19 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2365476884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:35:19.605 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T00:35:19.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.807+0000 7f4f3ce00700 1 -- 172.21.15.99:0/3763377194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38100d80 msgr2=0x7f4f381011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.807+0000 7f4f3ce00700 1 --2- 172.21.15.99:0/3763377194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38100d80 0x7f4f381011a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4f280099c0 tx=0x7f4f28009cd0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.807+0000 7f4f3ce00700 1 -- 172.21.15.99:0/3763377194 shutdown_connections 2024-03-30T00:35:19.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.807+0000 7f4f3ce00700 1 --2- 172.21.15.99:0/3763377194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f38102910 0x7f4f381091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.807+0000 7f4f3ce00700 1 --2- 172.21.15.99:0/3763377194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38100d80 0x7f4f381011a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.807+0000 7f4f3ce00700 1 --2- 172.21.15.99:0/3763377194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f38101f70 0x7f4f381023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.807+0000 7f4f3ce00700 1 -- 172.21.15.99:0/3763377194 >> 172.21.15.99:0/3763377194 conn(0x7f4f380fc2f0 msgr2=0x7f4f380fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.807+0000 7f4f3ce00700 1 -- 172.21.15.99:0/3763377194 shutdown_connections 2024-03-30T00:35:19.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.807+0000 7f4f3ce00700 1 -- 172.21.15.99:0/3763377194 wait complete. 2024-03-30T00:35:19.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.808+0000 7f4f3ce00700 1 Processor -- start 2024-03-30T00:35:19.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f3ce00700 1 -- start start 2024-03-30T00:35:19.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f3ce00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f38100d80 0x7f4f38196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f3ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38101f70 0x7f4f38197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f3ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f38102910 0x7f4f3819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f3ce00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3819e780 con 0x7f4f38101f70 2024-03-30T00:35:19.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f3ce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3819e8f0 con 0x7f4f38102910 2024-03-30T00:35:19.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f3ce00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3819eab0 con 0x7f4f38100d80 2024-03-30T00:35:19.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f35400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38101f70 0x7f4f38197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f35400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38101f70 0x7f4f38197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34366/0 (socket says 172.21.15.99:34366) 2024-03-30T00:35:19.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f35400700 1 -- 172.21.15.99:0/4084839302 learned_addr learned my addr 172.21.15.99:0/4084839302 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f36800700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f38102910 0x7f4f3819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.809+0000 7f4f35e00700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f38100d80 0x7f4f38196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f35400700 1 -- 172.21.15.99:0/4084839302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f38100d80 msgr2=0x7f4f38196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f35400700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f38100d80 0x7f4f38196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f35400700 1 -- 172.21.15.99:0/4084839302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f38102910 msgr2=0x7f4f3819e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f35400700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f38102910 0x7f4f3819e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f35400700 1 -- 172.21.15.99:0/4084839302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f280095f0 con 0x7f4f38101f70 2024-03-30T00:35:19.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f36800700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f38102910 0x7f4f3819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:19.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f35400700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38101f70 0x7f4f38197220 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400e9a0 tx=0x7f4f2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f2ea00700 1 -- 172.21.15.99:0/4084839302 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24009790 con 0x7f4f38101f70 2024-03-30T00:35:19.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f2ea00700 1 -- 172.21.15.99:0/4084839302 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f2400cbd0 con 0x7f4f38101f70 2024-03-30T00:35:19.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f2ea00700 1 -- 172.21.15.99:0/4084839302 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24018680 con 0x7f4f38101f70 2024-03-30T00:35:19.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f3ce00700 1 -- 172.21.15.99:0/4084839302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3819ed90 con 0x7f4f38101f70 2024-03-30T00:35:19.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.810+0000 7f4f3ce00700 1 -- 172.21.15.99:0/4084839302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3819f1c0 con 0x7f4f38101f70 2024-03-30T00:35:19.821 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:19 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2365476884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:35:19.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.813+0000 7f4f2ea00700 1 -- 172.21.15.99:0/4084839302 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f4f2400cd40 con 0x7f4f38101f70 2024-03-30T00:35:19.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.814+0000 7f4f2ea00700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4f1c0a11a0 0x7f4f1c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.815+0000 7f4f2ea00700 1 -- 172.21.15.99:0/4084839302 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f4f24014070 con 0x7f4f38101f70 2024-03-30T00:35:19.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.815+0000 7f4f35e00700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4f1c0a11a0 0x7f4f1c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.815+0000 7f4f35e00700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4f1c0a11a0 0x7f4f1c0a3660 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009990 tx=0x7f4f2801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.815+0000 7f4f509dbb80 1 Processor -- start 2024-03-30T00:35:19.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f509dbb80 1 -- start start 2024-03-30T00:35:19.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 0x55564fa6dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa6e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6efd0 0x55564fa6f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564f9c5990 con 0x55564fa6e130 2024-03-30T00:35:19.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564f9c3350 con 0x55564fa6efd0 2024-03-30T00:35:19.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564f9bec40 con 0x55564fa6d7d0 2024-03-30T00:35:19.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f35400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa6e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f35400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa6e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34370/0 (socket says 172.21.15.99:34370) 2024-03-30T00:35:19.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f35400700 1 -- 172.21.15.99:0/2144924409 learned_addr learned my addr 172.21.15.99:0/2144924409 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f35e00700 1 --2- 172.21.15.99:0/2144924409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 0x55564fa6dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.816+0000 7f4f36800700 1 --2- 172.21.15.99:0/2144924409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6efd0 0x55564fa6f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.817+0000 7f4f35400700 1 -- 172.21.15.99:0/2144924409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 msgr2=0x55564fa6dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.817+0000 7f4f35400700 1 --2- 172.21.15.99:0/2144924409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 0x55564fa6dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.817+0000 7f4f35400700 1 -- 172.21.15.99:0/2144924409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6efd0 msgr2=0x55564fa6f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.817+0000 7f4f35400700 1 --2- 172.21.15.99:0/2144924409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6efd0 0x55564fa6f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.817+0000 7f4f35400700 1 -- 172.21.15.99:0/2144924409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55564f8403b0 con 0x55564fa6e130 2024-03-30T00:35:19.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.817+0000 7f4f36800700 1 --2- 172.21.15.99:0/2144924409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6efd0 0x55564fa6f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:19.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.817+0000 7f4f35400700 1 --2- 172.21.15.99:0/2144924409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa6e9f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4f24099ec0 tx=0x7f4f24099b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829c2fbead969a15 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.817+0000 7f4f23400700 1 -- 172.21.15.99:0/2144924409 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2409a630 con 0x55564fa6e130 2024-03-30T00:35:19.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.817+0000 7f4f23400700 1 -- 172.21.15.99:0/2144924409 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f24099f30 con 0x55564fa6e130 2024-03-30T00:35:19.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.817+0000 7f4f23400700 1 -- 172.21.15.99:0/2144924409 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24099680 con 0x55564fa6e130 2024-03-30T00:35:19.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.818+0000 7f4f509dbb80 1 -- 172.21.15.99:0/2144924409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 msgr2=0x55564fa6e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.818+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/2144924409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa6e9f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4f24099ec0 tx=0x7f4f24099b40 comp rx=0 tx=0).stop 2024-03-30T00:35:19.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.818+0000 7f4f509dbb80 1 -- 172.21.15.99:0/2144924409 shutdown_connections 2024-03-30T00:35:19.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.818+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/2144924409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6efd0 0x55564fa6f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.818+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/2144924409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 0x55564fa6dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.818+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/2144924409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa6e9f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.818+0000 7f4f509dbb80 1 -- 172.21.15.99:0/2144924409 >> 172.21.15.99:0/2144924409 conn(0x55564f9cc6a0 msgr2=0x55564fa689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.818+0000 7f4f509dbb80 1 -- 172.21.15.99:0/2144924409 shutdown_connections 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.818+0000 7f4f509dbb80 1 -- 172.21.15.99:0/2144924409 wait complete. 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.819+0000 7f4f509dbb80 1 -- start start 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.819+0000 7f4f509dbb80 1 Processor -- start 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.819+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 0x55564fa7b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.819+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa7bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.819+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6efd0 0x55564fa82f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.819+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564f95ad80 con 0x55564fa6e130 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.819+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564f95aa20 con 0x55564fa6efd0 2024-03-30T00:35:19.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.819+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564f8426a0 con 0x55564fa6d7d0 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.819+0000 7f4f509dbb80 10 client.0 _ll_register_callbacks cb 0x55564f964470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.819+0000 7f4f35e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 0x55564fa7b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f35e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 0x55564fa7b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55658/0 (socket says 172.21.15.99:55658) 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f35e00700 1 -- 172.21.15.99:0/3148912643 learned_addr learned my addr 172.21.15.99:0/3148912643 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f35400700 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa7bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f36800700 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6efd0 0x55564fa82f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f35e00700 1 -- 172.21.15.99:0/3148912643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6efd0 msgr2=0x55564fa82f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f35e00700 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6efd0 0x55564fa82f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f35e00700 1 -- 172.21.15.99:0/3148912643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 msgr2=0x55564fa7bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f35e00700 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa7bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f35e00700 1 -- 172.21.15.99:0/3148912643 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3819f1c0 con 0x55564fa6d7d0 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f35400700 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa7bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f35e00700 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 0x55564fa7b9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f28005f50 tx=0x7f4f28009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691fcf202a30f353 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.820+0000 7f4f17e00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:35:19.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.821+0000 7f4f17e00700 1 -- 172.21.15.99:0/3148912643 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2802c040 con 0x55564fa6d7d0 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.821+0000 7f4f17e00700 1 -- 172.21.15.99:0/3148912643 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f4f2800f460 con 0x55564fa6d7d0 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.821+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3148912643 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55564f97f380 con 0x55564fa6d7d0 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.821+0000 7f4f17e00700 1 -- 172.21.15.99:0/3148912643 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f2800bcf0 con 0x55564fa6d7d0 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.821+0000 7f4f509dbb80 10 client.25321 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.821+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3148912643 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55564f7ecb60 con 0x55564fa6d7d0 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.821+0000 7f4f17e00700 1 -- 172.21.15.99:0/3148912643 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f2803e020 con 0x55564fa6d7d0 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.821+0000 7f4f17e00700 1 -- 172.21.15.99:0/3148912643 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4f28026630 con 0x55564fa6d7d0 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.821+0000 7f4f509dbb80 10 client.25321 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.821+0000 7f4f509dbb80 10 client.25321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.822+0000 7f4f509dbb80 20 client.25321 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.822+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55564fa87760 0x55564fa89b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.822+0000 7f4f509dbb80 4 client.25321 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.822+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3148912643 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55564f9b7700 con 0x55564fa87760 2024-03-30T00:35:19.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.822+0000 7f4f35400700 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55564fa87760 0x55564fa89b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.823+0000 7f4f17e00700 10 client.25321 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.823+0000 7f4f35400700 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55564fa87760 0x55564fa89b80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1914e73ef2e0eb server_cookie=f4ea335423fa9fa3 in_seq=0 out_seq=0 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f17e00700 1 -- 172.21.15.99:0/3148912643 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55564f9b7700 con 0x55564fa87760 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f17e00700 10 client.25321 handle_command_reply: tid=0 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 1 client.25321 shutdown 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3148912643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 msgr2=0x55564fa7b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 0x55564fa7b9d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f28005f50 tx=0x7f4f28009cd0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3148912643 shutdown_connections 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55564fa87760 0x55564fa89b80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6efd0 0x55564fa82f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f17e00700 0 client.25321 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6d7d0 0x55564fa7b9d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/3148912643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa7bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3148912643 >> 172.21.15.99:0/3148912643 conn(0x55564f9cc6a0 msgr2=0x55564fa6c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3148912643 shutdown_connections 2024-03-30T00:35:19.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3148912643 wait complete. 2024-03-30T00:35:19.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.824+0000 7f4f509dbb80 20 client.25321 trim_cache size 0 max 16384 2024-03-30T00:35:19.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.841+0000 7f4f509dbb80 1 Processor -- start 2024-03-30T00:35:19.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.841+0000 7f4f509dbb80 1 -- start start 2024-03-30T00:35:19.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.842+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 0x55564fa89b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.842+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6e130 0x55564fa6dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.842+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6efd0 0x55564fa82f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.842+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fa87460 con 0x55564fa6efd0 2024-03-30T00:35:19.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.842+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fa83d30 con 0x55564fa6d7d0 2024-03-30T00:35:19.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.842+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fa83ef0 con 0x55564fa6e130 2024-03-30T00:35:19.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.842+0000 7f4f36800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6efd0 0x55564fa82f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.842+0000 7f4f36800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6efd0 0x55564fa82f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34382/0 (socket says 172.21.15.99:34382) 2024-03-30T00:35:19.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.842+0000 7f4f36800700 1 -- 172.21.15.99:0/3959849919 learned_addr learned my addr 172.21.15.99:0/3959849919 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.842+0000 7f4f35400700 1 --2- 172.21.15.99:0/3959849919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6e130 0x55564fa6dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.842+0000 7f4f35e00700 1 --2- 172.21.15.99:0/3959849919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 0x55564fa89b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.843+0000 7f4f36800700 1 -- 172.21.15.99:0/3959849919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6e130 msgr2=0x55564fa6dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.843+0000 7f4f36800700 1 --2- 172.21.15.99:0/3959849919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6e130 0x55564fa6dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.843+0000 7f4f36800700 1 -- 172.21.15.99:0/3959849919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 msgr2=0x55564fa89b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.843+0000 7f4f36800700 1 --2- 172.21.15.99:0/3959849919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 0x55564fa89b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.843+0000 7f4f36800700 1 -- 172.21.15.99:0/3959849919 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55564f984ea0 con 0x55564fa6efd0 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.843+0000 7f4f35400700 1 --2- 172.21.15.99:0/3959849919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6e130 0x55564fa6dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.843+0000 7f4f36800700 1 --2- 172.21.15.99:0/3959849919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6efd0 0x55564fa82f10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009dd0 tx=0x7f4f300077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e47002f75d486c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.843+0000 7f4f23e00700 1 -- 172.21.15.99:0/3959849919 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30004bb0 con 0x55564fa6efd0 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.843+0000 7f4f23e00700 1 -- 172.21.15.99:0/3959849919 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f3000f040 con 0x55564fa6efd0 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.843+0000 7f4f23e00700 1 -- 172.21.15.99:0/3959849919 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30017e50 con 0x55564fa6efd0 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.844+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3959849919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6efd0 msgr2=0x55564fa82f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.844+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/3959849919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6efd0 0x55564fa82f10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009dd0 tx=0x7f4f300077e0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.844+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3959849919 shutdown_connections 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.844+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/3959849919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6efd0 0x55564fa82f10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.844+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/3959849919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 0x55564fa89b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.844+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/3959849919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6e130 0x55564fa6dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.844+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3959849919 >> 172.21.15.99:0/3959849919 conn(0x55564f9cc6a0 msgr2=0x55564fa8e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.844+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3959849919 shutdown_connections 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.844+0000 7f4f509dbb80 1 -- 172.21.15.99:0/3959849919 wait complete. 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.844+0000 7f4f509dbb80 1 -- start start 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f509dbb80 1 Processor -- start 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 0x55564fa80ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f509dbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6efd0 0x55564fa82150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fa82690 con 0x55564fa6e130 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fa82800 con 0x55564fa6d7d0 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f509dbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fa829c0 con 0x55564fa6efd0 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f509dbb80 10 client.0 _ll_register_callbacks cb 0x55564f964470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f35400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f35e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 0x55564fa80ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f35e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 0x55564fa80ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:35768/0 (socket says 172.21.15.99:35768) 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f35e00700 1 -- 172.21.15.99:0/1396496073 learned_addr learned my addr 172.21.15.99:0/1396496073 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.845+0000 7f4f36800700 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6efd0 0x55564fa82150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.846+0000 7f4f35e00700 1 -- 172.21.15.99:0/1396496073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6efd0 msgr2=0x55564fa82150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.846+0000 7f4f35e00700 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6efd0 0x55564fa82150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.846+0000 7f4f35e00700 1 -- 172.21.15.99:0/1396496073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 msgr2=0x55564fa811e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.846+0000 7f4f35e00700 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa811e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.846+0000 7f4f35e00700 1 -- 172.21.15.99:0/1396496073 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55564f8403b0 con 0x55564fa6d7d0 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.846+0000 7f4f35e00700 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 0x55564fa80ca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f280098e0 tx=0x7f4f2800baf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568323e49d47ace3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.846+0000 7f4f17e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.846+0000 7f4f17e00700 1 -- 172.21.15.99:0/1396496073 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2802c040 con 0x55564fa6d7d0 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.846+0000 7f4f509dbb80 1 -- 172.21.15.99:0/1396496073 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f24099f30 con 0x55564fa6d7d0 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.846+0000 7f4f17e00700 1 -- 172.21.15.99:0/1396496073 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f4f2803f080 con 0x55564fa6d7d0 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.847+0000 7f4f17e00700 1 -- 172.21.15.99:0/1396496073 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f2800bcf0 con 0x55564fa6d7d0 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.847+0000 7f4f509dbb80 10 client.25316 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:19.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.847+0000 7f4f509dbb80 1 -- 172.21.15.99:0/1396496073 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f4f30009a30 con 0x55564fa6d7d0 2024-03-30T00:35:19.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.847+0000 7f4f17e00700 1 -- 172.21.15.99:0/1396496073 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f24099f30 con 0x55564fa6d7d0 2024-03-30T00:35:19.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.847+0000 7f4f17e00700 1 -- 172.21.15.99:0/1396496073 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4f28030c80 con 0x55564fa6d7d0 2024-03-30T00:35:19.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.847+0000 7f4f509dbb80 10 client.25316 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:19.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.847+0000 7f4f509dbb80 10 client.25316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:19.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.847+0000 7f4f509dbb80 20 client.25316 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:19.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.847+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55564fadc050 0x55564fade470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:19.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.847+0000 7f4f509dbb80 4 client.25316 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:35:19.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.847+0000 7f4f509dbb80 1 -- 172.21.15.99:0/1396496073 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55564f80dcf0 con 0x55564fadc050 2024-03-30T00:35:19.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.848+0000 7f4f35400700 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55564fadc050 0x55564fade470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:19.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.848+0000 7f4f35400700 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55564fadc050 0x55564fade470 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2da5ac9f793a36ba server_cookie=330f40f010a44d0c in_seq=0 out_seq=0 2024-03-30T00:35:19.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.848+0000 7f4f17e00700 10 client.25316 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:19.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.848+0000 7f4f17e00700 1 -- 172.21.15.99:0/1396496073 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55564f80dcf0 con 0x55564fadc050 2024-03-30T00:35:19.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.848+0000 7f4f17e00700 10 client.25316 handle_command_reply: tid=0 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.848+0000 7f4f509dbb80 1 client.25316 shutdown 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f509dbb80 1 -- 172.21.15.99:0/1396496073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 msgr2=0x55564fa80ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 0x55564fa80ca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f280098e0 tx=0x7f4f2800baf0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f509dbb80 1 -- 172.21.15.99:0/1396496073 shutdown_connections 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55564fadc050 0x55564fade470 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55564fa6efd0 0x55564fa82150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55564fa6d7d0 0x55564fa80ca0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f509dbb80 1 --2- 172.21.15.99:0/1396496073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55564fa6e130 0x55564fa811e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f17e00700 0 client.25316 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f509dbb80 1 -- 172.21.15.99:0/1396496073 >> 172.21.15.99:0/1396496073 conn(0x55564f9cc6a0 msgr2=0x55564fa615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f509dbb80 1 -- 172.21.15.99:0/1396496073 shutdown_connections 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f509dbb80 1 -- 172.21.15.99:0/1396496073 wait complete. 2024-03-30T00:35:19.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.849+0000 7f4f509dbb80 20 client.25316 trim_cache size 0 max 16384 2024-03-30T00:35:19.861 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:35:19.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 -- 172.21.15.99:0/4084839302 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4f1c0a11a0 msgr2=0x7f4f1c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4f1c0a11a0 0x7f4f1c0a3660 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009990 tx=0x7f4f2801a040 comp rx=0 tx=0).stop 2024-03-30T00:35:19.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 -- 172.21.15.99:0/4084839302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38101f70 msgr2=0x7f4f38197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:19.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38101f70 0x7f4f38197220 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400e9a0 tx=0x7f4f2400ecb0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 -- 172.21.15.99:0/4084839302 shutdown_connections 2024-03-30T00:35:19.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4f1c0a11a0 0x7f4f1c0a3660 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f38102910 0x7f4f3819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f38100d80 0x7f4f38196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 --2- 172.21.15.99:0/4084839302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f38101f70 0x7f4f38197220 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:19.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 -- 172.21.15.99:0/4084839302 >> 172.21.15.99:0/4084839302 conn(0x7f4f380fc2f0 msgr2=0x7f4f38105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:19.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 -- 172.21.15.99:0/4084839302 shutdown_connections 2024-03-30T00:35:19.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:19.850+0000 7f4f3ce00700 1 -- 172.21.15.99:0/4084839302 wait complete. 2024-03-30T00:35:19.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:19 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2365476884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:35:20.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:20 smithi142 ceph-mon[41779]: pgmap v322: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 321 B/s wr, 6 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:20.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:20 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.879 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T00:35:20.880 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T00:35:20.937 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:20 smithi099 ceph-mon[38079]: pgmap v322: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 321 B/s wr, 6 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:20.937 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.937 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.938 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.938 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.938 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.938 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:20 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:20 smithi161 ceph-mon[42221]: pgmap v322: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 321 B/s wr, 6 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-03-30T00:35:20.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:20.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:20 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:21.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.072+0000 7fa2d3e00700 1 -- 172.21.15.99:0/1061928600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc1017e0 msgr2=0x7fa2cc103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.072+0000 7fa2d3e00700 1 --2- 172.21.15.99:0/1061928600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc1017e0 0x7fa2cc103bd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009990 tx=0x7fa2c0009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.072+0000 7fa2d3e00700 1 -- 172.21.15.99:0/1061928600 shutdown_connections 2024-03-30T00:35:21.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.072+0000 7fa2d3e00700 1 --2- 172.21.15.99:0/1061928600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc106a40 0x7fa2cc108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.072+0000 7fa2d3e00700 1 --2- 172.21.15.99:0/1061928600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc1017e0 0x7fa2cc103bd0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.072+0000 7fa2d3e00700 1 --2- 172.21.15.99:0/1061928600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc104110 0x7fa2cc106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.072+0000 7fa2d3e00700 1 -- 172.21.15.99:0/1061928600 >> 172.21.15.99:0/1061928600 conn(0x7fa2cc0fb1d0 msgr2=0x7fa2cc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.072+0000 7fa2d3e00700 1 -- 172.21.15.99:0/1061928600 shutdown_connections 2024-03-30T00:35:21.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.072+0000 7fa2d3e00700 1 -- 172.21.15.99:0/1061928600 wait complete. 2024-03-30T00:35:21.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.073+0000 7fa2d3e00700 1 Processor -- start 2024-03-30T00:35:21.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.073+0000 7fa2d3e00700 1 -- start start 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.073+0000 7fa2d3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc1017e0 0x7fa2cc196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.073+0000 7fa2d3e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc104110 0x7fa2cc197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.073+0000 7fa2d3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc106a40 0x7fa2cc19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.073+0000 7fa2d3e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc19e680 con 0x7fa2cc106a40 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.073+0000 7fa2d3e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc19e7c0 con 0x7fa2cc1017e0 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.073+0000 7fa2d3e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc19e950 con 0x7fa2cc104110 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2d1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc106a40 0x7fa2cc19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2d1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc106a40 0x7fa2cc19e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34410/0 (socket says 172.21.15.99:34410) 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2d1e00700 1 -- 172.21.15.99:0/3044033654 learned_addr learned my addr 172.21.15.99:0/3044033654 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2d1400700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc1017e0 0x7fa2cc196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc104110 0x7fa2cc197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2d1e00700 1 -- 172.21.15.99:0/3044033654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc104110 msgr2=0x7fa2cc197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2d1e00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc104110 0x7fa2cc197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2d1e00700 1 -- 172.21.15.99:0/3044033654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc1017e0 msgr2=0x7fa2cc196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2d1e00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc1017e0 0x7fa2cc196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2d1e00700 1 -- 172.21.15.99:0/3044033654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c00095f0 con 0x7fa2cc106a40 2024-03-30T00:35:21.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2d1e00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc106a40 0x7fa2cc19e0b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400ef30 tx=0x7fa2c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.074+0000 7fa2ca000700 1 -- 172.21.15.99:0/3044033654 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4018070 con 0x7fa2cc106a40 2024-03-30T00:35:21.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.075+0000 7fa2d3e00700 1 -- 172.21.15.99:0/3044033654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc19ebd0 con 0x7fa2cc106a40 2024-03-30T00:35:21.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.075+0000 7fa2ca000700 1 -- 172.21.15.99:0/3044033654 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2c400f040 con 0x7fa2cc106a40 2024-03-30T00:35:21.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.075+0000 7fa2ca000700 1 -- 172.21.15.99:0/3044033654 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c400cc70 con 0x7fa2cc106a40 2024-03-30T00:35:21.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.075+0000 7fa2d3e00700 1 -- 172.21.15.99:0/3044033654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc19f090 con 0x7fa2cc106a40 2024-03-30T00:35:21.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.078+0000 7fa2ca000700 1 -- 172.21.15.99:0/3044033654 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa2c4007500 con 0x7fa2cc106a40 2024-03-30T00:35:21.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.079+0000 7fa2e79d2b80 1 Processor -- start 2024-03-30T00:35:21.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.079+0000 7fa2e79d2b80 1 -- start start 2024-03-30T00:35:21.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.079+0000 7fa2ca000700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a1120 0x7fa2b40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.079+0000 7fa2ca000700 1 -- 172.21.15.99:0/3044033654 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fa2c40cc4e0 con 0x7fa2cc106a40 2024-03-30T00:35:21.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.079+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f4850 0x557a421f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.079+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f51b0 0x557a421f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.079+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f6050 0x557a421f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.079+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a1120 0x7fa2b40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.079+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4214ca10 con 0x557a421f6050 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.079+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4214a3d0 con 0x557a421f51b0 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.079+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a42145cc0 con 0x557a421f4850 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f6050 0x557a421f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f6050 0x557a421f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34420/0 (socket says 172.21.15.99:34420) 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d1e00700 1 -- 172.21.15.99:0/1714529584 learned_addr learned my addr 172.21.15.99:0/1714529584 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/1714529584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f51b0 0x557a421f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d1400700 1 --2- 172.21.15.99:0/1714529584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f4850 0x557a421f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d1e00700 1 -- 172.21.15.99:0/1714529584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f4850 msgr2=0x557a421f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d1e00700 1 --2- 172.21.15.99:0/1714529584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f4850 0x557a421f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d1e00700 1 -- 172.21.15.99:0/1714529584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f51b0 msgr2=0x557a421f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d1e00700 1 --2- 172.21.15.99:0/1714529584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f51b0 0x557a421f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d1e00700 1 -- 172.21.15.99:0/1714529584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a41fc7430 con 0x557a421f6050 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/1714529584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f51b0 0x557a421f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:21.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a1120 0x7fa2b40a35e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80046e0 tx=0x7fa2b8009860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2d1e00700 1 --2- 172.21.15.99:0/1714529584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f6050 0x557a421f64d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4095de0 tx=0x7fa2c40994f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7f6325d68fccf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.080+0000 7fa2be600700 1 -- 172.21.15.99:0/1714529584 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4084040 con 0x557a421f6050 2024-03-30T00:35:21.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.081+0000 7fa2be600700 1 -- 172.21.15.99:0/1714529584 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2c4099fe0 con 0x557a421f6050 2024-03-30T00:35:21.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.081+0000 7fa2be600700 1 -- 172.21.15.99:0/1714529584 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c400ce50 con 0x557a421f6050 2024-03-30T00:35:21.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.081+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1714529584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f6050 msgr2=0x557a421f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.081+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/1714529584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f6050 0x557a421f64d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4095de0 tx=0x7fa2c40994f0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.081+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1714529584 shutdown_connections 2024-03-30T00:35:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.081+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/1714529584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f6050 0x557a421f64d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.081+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/1714529584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f4850 0x557a421f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.081+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/1714529584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f51b0 0x557a421f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.081+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1714529584 >> 172.21.15.99:0/1714529584 conn(0x557a42153720 msgr2=0x557a421efa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.081+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1714529584 shutdown_connections 2024-03-30T00:35:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.081+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1714529584 wait complete. 2024-03-30T00:35:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.082+0000 7fa2e79d2b80 1 -- start start 2024-03-30T00:35:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.082+0000 7fa2e79d2b80 1 Processor -- start 2024-03-30T00:35:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.082+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42202c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.082+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f51b0 0x557a42203150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.082+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f6050 0x557a4220a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a420e1e00 con 0x557a421f4850 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a420e1aa0 con 0x557a421f6050 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a41fc9720 con 0x557a421f51b0 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2e79d2b80 10 client.0 _ll_register_callbacks cb 0x557a420eb4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2d1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42202c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2d1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42202c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34430/0 (socket says 172.21.15.99:34430) 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2d1400700 1 -- 172.21.15.99:0/2619985128 learned_addr learned my addr 172.21.15.99:0/2619985128 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2d1e00700 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f6050 0x557a4220a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f51b0 0x557a42203150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2d1400700 1 -- 172.21.15.99:0/2619985128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f51b0 msgr2=0x557a42203150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2d1400700 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f51b0 0x557a42203150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2d1400700 1 -- 172.21.15.99:0/2619985128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f6050 msgr2=0x557a4220a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2d1400700 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f6050 0x557a4220a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2d1400700 1 -- 172.21.15.99:0/2619985128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc19f090 con 0x557a421f4850 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2d1400700 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42202c10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0000c00 tx=0x7fa2c0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d75ce644cad839c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.083+0000 7fa2b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.084+0000 7fa2b2a00700 1 -- 172.21.15.99:0/2619985128 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c001d070 con 0x557a421f4850 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.084+0000 7fa2b2a00700 1 -- 172.21.15.99:0/2619985128 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fa2c00056c0 con 0x557a421f4850 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.084+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/2619985128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a42106400 con 0x557a421f4850 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.084+0000 7fa2b2a00700 1 -- 172.21.15.99:0/2619985128 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2c0003a90 con 0x557a421f4850 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.084+0000 7fa2e79d2b80 10 client.15612 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.084+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/2619985128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557a41f73be0 con 0x557a421f4850 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.084+0000 7fa2b2a00700 1 -- 172.21.15.99:0/2619985128 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557a42106400 con 0x557a421f4850 2024-03-30T00:35:21.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.084+0000 7fa2b2a00700 1 -- 172.21.15.99:0/2619985128 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa2c0004570 con 0x557a421f4850 2024-03-30T00:35:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.084+0000 7fa2e79d2b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.084+0000 7fa2e79d2b80 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.084+0000 7fa2e79d2b80 20 client.15612 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.085+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x557a4220e800 0x557a42210c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.085+0000 7fa2e79d2b80 4 client.15612 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.085+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/2619985128 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a4213e780 con 0x557a4220e800 2024-03-30T00:35:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.085+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x557a4220e800 0x557a42210c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.085+0000 7fa2b2a00700 10 client.15612 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.085+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x557a4220e800 0x557a42210c20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=263435a2c28a52f5 server_cookie=ba6badba7d0973d2 in_seq=0 out_seq=0 2024-03-30T00:35:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2b2a00700 1 -- 172.21.15.99:0/2619985128 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a4213e780 con 0x557a4220e800 2024-03-30T00:35:21.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2b2a00700 10 client.15612 handle_command_reply: tid=0 2024-03-30T00:35:21.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2e79d2b80 1 client.15612 shutdown 2024-03-30T00:35:21.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/2619985128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 msgr2=0x557a42202c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42202c10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0000c00 tx=0x7fa2c0005270 comp rx=0 tx=0).stop 2024-03-30T00:35:21.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/2619985128 shutdown_connections 2024-03-30T00:35:21.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x557a4220e800 0x557a42210c20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f6050 0x557a4220a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42202c10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2b2a00700 0 client.15612 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:21.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/2619985128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f51b0 0x557a42203150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.086+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/2619985128 >> 172.21.15.99:0/2619985128 conn(0x557a42153720 msgr2=0x557a421f31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.087+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/2619985128 shutdown_connections 2024-03-30T00:35:21.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.087+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/2619985128 wait complete. 2024-03-30T00:35:21.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.087+0000 7fa2e79d2b80 20 client.15612 trim_cache size 0 max 16384 2024-03-30T00:35:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.104+0000 7fa2e79d2b80 1 Processor -- start 2024-03-30T00:35:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.104+0000 7fa2e79d2b80 1 -- start start 2024-03-30T00:35:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a421f6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f51b0 0x557a4220a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f6050 0x557a421e8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a421f5900 con 0x557a421f4850 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a4220b1d0 con 0x557a421f51b0 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a421f06e0 con 0x557a421f6050 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2d1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a421f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2d1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a421f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34442/0 (socket says 172.21.15.99:34442) 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2d1400700 1 -- 172.21.15.99:0/529971572 learned_addr learned my addr 172.21.15.99:0/529971572 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/529971572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f51b0 0x557a4220a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2d1e00700 1 --2- 172.21.15.99:0/529971572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f6050 0x557a421e8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2d1400700 1 -- 172.21.15.99:0/529971572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f6050 msgr2=0x557a421e8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2d1400700 1 --2- 172.21.15.99:0/529971572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f6050 0x557a421e8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2d1400700 1 -- 172.21.15.99:0/529971572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f51b0 msgr2=0x557a4220a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2d1400700 1 --2- 172.21.15.99:0/529971572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f51b0 0x557a4220a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2d1400700 1 -- 172.21.15.99:0/529971572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a421fc980 con 0x557a421f4850 2024-03-30T00:35:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.105+0000 7fa2d1e00700 1 --2- 172.21.15.99:0/529971572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f6050 0x557a421e8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:35:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.106+0000 7fa2d1400700 1 --2- 172.21.15.99:0/529971572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a421f6450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0005ac0 tx=0x7fa2c000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb8050a2bc8534c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.106+0000 7fa2bf000700 1 -- 172.21.15.99:0/529971572 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c001d040 con 0x557a421f4850 2024-03-30T00:35:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.106+0000 7fa2bf000700 1 -- 172.21.15.99:0/529971572 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2c001e040 con 0x557a421f4850 2024-03-30T00:35:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.106+0000 7fa2bf000700 1 -- 172.21.15.99:0/529971572 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c000b970 con 0x557a421f4850 2024-03-30T00:35:21.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.106+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/529971572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 msgr2=0x557a421f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.106+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/529971572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a421f6450 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0005ac0 tx=0x7fa2c000b410 comp rx=0 tx=0).stop 2024-03-30T00:35:21.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.107+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/529971572 shutdown_connections 2024-03-30T00:35:21.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.107+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/529971572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f6050 0x557a421e8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.107+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/529971572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a421f6450 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.107+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/529971572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f51b0 0x557a4220a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.107+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/529971572 >> 172.21.15.99:0/529971572 conn(0x557a42153720 msgr2=0x557a421fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.107+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/529971572 shutdown_connections 2024-03-30T00:35:21.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.107+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/529971572 wait complete. 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.107+0000 7fa2e79d2b80 1 -- start start 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.107+0000 7fa2e79d2b80 1 Processor -- start 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42208a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f51b0 0x557a42208f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2e79d2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f6050 0x557a42209480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a422627a0 con 0x557a421f4850 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a42262910 con 0x557a421f6050 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2e79d2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a42262ad0 con 0x557a421f51b0 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2e79d2b80 10 client.0 _ll_register_callbacks cb 0x557a420eb4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42208a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42208a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34450/0 (socket says 172.21.15.99:34450) 2024-03-30T00:35:21.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d1400700 1 -- 172.21.15.99:0/1320316650 learned_addr learned my addr 172.21.15.99:0/1320316650 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d1e00700 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f6050 0x557a42209480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f51b0 0x557a42208f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d1400700 1 -- 172.21.15.99:0/1320316650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f51b0 msgr2=0x557a42208f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d1400700 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f51b0 0x557a42208f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d1400700 1 -- 172.21.15.99:0/1320316650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f6050 msgr2=0x557a42209480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d1400700 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f6050 0x557a42209480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d1400700 1 -- 172.21.15.99:0/1320316650 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a41f73be0 con 0x557a421f4850 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f51b0 0x557a42208f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.108+0000 7fa2d1400700 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42208a00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0005c20 tx=0x7fa2c0004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a77b4be4828f3d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2b2a00700 1 -- 172.21.15.99:0/1320316650 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c001d070 con 0x557a421f4850 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2b2a00700 1 -- 172.21.15.99:0/1320316650 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fa2c0000e40 con 0x557a421f4850 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1320316650 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2c4099fe0 con 0x557a421f4850 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2b2a00700 1 -- 172.21.15.99:0/1320316650 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2c0003c60 con 0x557a421f4850 2024-03-30T00:35:21.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2e79d2b80 10 client.15618 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1320316650 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557a4210bf20 con 0x557a421f4850 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2b2a00700 1 -- 172.21.15.99:0/1320316650 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2c4099fe0 con 0x557a421f4850 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2b2a00700 1 -- 172.21.15.99:0/1320316650 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa2c00246e0 con 0x557a421f4850 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2e79d2b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2e79d2b80 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2e79d2b80 20 client.15618 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x557a42264000 0x557a42266420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2e79d2b80 4 client.15618 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.109+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1320316650 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557a41f94d70 con 0x557a42264000 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.110+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x557a42264000 0x557a42266420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.110+0000 7fa2b2a00700 10 client.15618 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:21.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.110+0000 7fa2d0a00700 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x557a42264000 0x557a42266420 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe0e8228dcece89 server_cookie=8e0c087537d0ae11 in_seq=0 out_seq=0 2024-03-30T00:35:21.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.111+0000 7fa2b2a00700 1 -- 172.21.15.99:0/1320316650 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557a41f94d70 con 0x557a42264000 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.111+0000 7fa2b2a00700 10 client.15618 handle_command_reply: tid=0 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.111+0000 7fa2e79d2b80 1 client.15618 shutdown 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.111+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1320316650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 msgr2=0x557a42208a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.111+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42208a00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0005c20 tx=0x7fa2c0004e80 comp rx=0 tx=0).stop 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1320316650 shutdown_connections 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x557a42264000 0x557a42266420 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a421f6050 0x557a42209480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557a421f4850 0x557a42208a00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2e79d2b80 1 --2- 172.21.15.99:0/1320316650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557a421f51b0 0x557a42208f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1320316650 >> 172.21.15.99:0/1320316650 conn(0x557a42153720 msgr2=0x557a421e8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2b2a00700 0 client.15618 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1320316650 shutdown_connections 2024-03-30T00:35:21.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2e79d2b80 1 -- 172.21.15.99:0/1320316650 wait complete. 2024-03-30T00:35:21.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2e79d2b80 20 client.15618 trim_cache size 0 max 16384 2024-03-30T00:35:21.123 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:35:21.123 INFO:teuthology.orchestra.run.smithi099.stdout: "return_code": 0, 2024-03-30T00:35:21.123 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_tag": "1700173d-da29-48fc-ac7f-8c0f5aea4859", 2024-03-30T00:35:21.123 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": "asynchronous" 2024-03-30T00:35:21.123 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:35:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2d3e00700 1 -- 172.21.15.99:0/3044033654 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a1120 msgr2=0x7fa2b40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2d3e00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a1120 0x7fa2b40a35e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80046e0 tx=0x7fa2b8009860 comp rx=0 tx=0).stop 2024-03-30T00:35:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2d3e00700 1 -- 172.21.15.99:0/3044033654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc106a40 msgr2=0x7fa2cc19e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.112+0000 7fa2d3e00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc106a40 0x7fa2cc19e0b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400ef30 tx=0x7fa2c400c3a0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.113+0000 7fa2d3e00700 1 -- 172.21.15.99:0/3044033654 shutdown_connections 2024-03-30T00:35:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.113+0000 7fa2d3e00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2cc106a40 0x7fa2cc19e0b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.113+0000 7fa2d3e00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa2b40a1120 0x7fa2b40a35e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.113+0000 7fa2d3e00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc1017e0 0x7fa2cc196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.113+0000 7fa2d3e00700 1 --2- 172.21.15.99:0/3044033654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc104110 0x7fa2cc197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.113+0000 7fa2d3e00700 1 -- 172.21.15.99:0/3044033654 >> 172.21.15.99:0/3044033654 conn(0x7fa2cc0fb1d0 msgr2=0x7fa2cc103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.113+0000 7fa2d3e00700 1 -- 172.21.15.99:0/3044033654 shutdown_connections 2024-03-30T00:35:21.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.113+0000 7fa2d3e00700 1 -- 172.21.15.99:0/3044033654 wait complete. 2024-03-30T00:35:21.137 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T00:35:21.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.324+0000 7f75ef000700 1 -- 172.21.15.99:0/834438772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e8074df0 msgr2=0x7f75e8073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.324+0000 7f75ef000700 1 --2- 172.21.15.99:0/834438772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e8074df0 0x7f75e8073240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009990 tx=0x7f75d8009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.324+0000 7f75ef000700 1 -- 172.21.15.99:0/834438772 shutdown_connections 2024-03-30T00:35:21.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.324+0000 7f75ef000700 1 --2- 172.21.15.99:0/834438772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e8108c40 0x7f75e810b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.324+0000 7f75ef000700 1 --2- 172.21.15.99:0/834438772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e8074df0 0x7f75e8073240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.324+0000 7f75ef000700 1 --2- 172.21.15.99:0/834438772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75e8073810 0x7f75e8073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.324+0000 7f75ef000700 1 -- 172.21.15.99:0/834438772 >> 172.21.15.99:0/834438772 conn(0x7f75e80fc2f0 msgr2=0x7f75e80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.324+0000 7f75ef000700 1 -- 172.21.15.99:0/834438772 shutdown_connections 2024-03-30T00:35:21.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.324+0000 7f75ef000700 1 -- 172.21.15.99:0/834438772 wait complete. 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.325+0000 7f75ef000700 1 Processor -- start 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.325+0000 7f75ef000700 1 -- start start 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.325+0000 7f75ef000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e8073810 0x7f75e8198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.325+0000 7f75ef000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e8074df0 0x7f75e81993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.325+0000 7f75ef000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75e8108c40 0x7f75e81a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.325+0000 7f75ef000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e81a09a0 con 0x7f75e8108c40 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.325+0000 7f75ef000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e81a0ae0 con 0x7f75e8073810 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.325+0000 7f75ef000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e81a0c70 con 0x7f75e8074df0 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75ed000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75e8108c40 0x7f75e81a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75e7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e8073810 0x7f75e8198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75ed000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75e8108c40 0x7f75e81a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34466/0 (socket says 172.21.15.99:34466) 2024-03-30T00:35:21.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75ed000700 1 -- 172.21.15.99:0/791648318 learned_addr learned my addr 172.21.15.99:0/791648318 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75e7400700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e8074df0 0x7f75e81993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75e7e00700 1 -- 172.21.15.99:0/791648318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e8074df0 msgr2=0x7f75e81993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75e7e00700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e8074df0 0x7f75e81993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75e7e00700 1 -- 172.21.15.99:0/791648318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75e8108c40 msgr2=0x7f75e81a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75e7e00700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75e8108c40 0x7f75e81a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75e7e00700 1 -- 172.21.15.99:0/791648318 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80095f0 con 0x7f75e8073810 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75ed000700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75e8108c40 0x7f75e81a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75e7e00700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e8073810 0x7f75e8198e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f75d8003f60 tx=0x7f75d8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.326+0000 7f75e4c00700 1 -- 172.21.15.99:0/791648318 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d801d070 con 0x7f75e8073810 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.327+0000 7f75e4c00700 1 -- 172.21.15.99:0/791648318 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75d800bdb0 con 0x7f75e8073810 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.327+0000 7f75ef000700 1 -- 172.21.15.99:0/791648318 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e81a0ef0 con 0x7f75e8073810 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.327+0000 7f75e4c00700 1 -- 172.21.15.99:0/791648318 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8022d20 con 0x7f75e8073810 2024-03-30T00:35:21.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.327+0000 7f75ef000700 1 -- 172.21.15.99:0/791648318 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e81a12b0 con 0x7f75e8073810 2024-03-30T00:35:21.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.330+0000 7f75e4c00700 1 -- 172.21.15.99:0/791648318 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f75d8022470 con 0x7f75e8073810 2024-03-30T00:35:21.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.331+0000 7f7602ab8b80 1 Processor -- start 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f7602ab8b80 1 -- start start 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f75e4c00700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75cc0a11a0 0x7f75cc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbd130 0x55cda0fbd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbdfd0 0x55cda0fbe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f75e4c00700 1 -- 172.21.15.99:0/791648318 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f75d80cde80 con 0x7f75e8073810 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0f14990 con 0x55cda0fbc7d0 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0f12350 con 0x55cda0fbdfd0 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0f0dc40 con 0x55cda0fbd130 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f75ed000700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75cc0a11a0 0x7f75cc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f75e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f75ed000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbdfd0 0x55cda0fbe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f75ed000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbdfd0 0x55cda0fbe450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:35880/0 (socket says 172.21.15.99:35880) 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f75ed000700 1 -- 172.21.15.99:0/2476901859 learned_addr learned my addr 172.21.15.99:0/2476901859 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f75e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34474/0 (socket says 172.21.15.99:34474) 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.332+0000 7f75e7400700 1 --2- 172.21.15.99:0/2476901859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbd130 0x55cda0fbd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.333+0000 7f75e7e00700 1 -- 172.21.15.99:0/2476901859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbd130 msgr2=0x55cda0fbd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.333+0000 7f75e7e00700 1 --2- 172.21.15.99:0/2476901859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbd130 0x55cda0fbd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.333+0000 7f75e7e00700 1 -- 172.21.15.99:0/2476901859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbdfd0 msgr2=0x55cda0fbe450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.333+0000 7f75e7e00700 1 --2- 172.21.15.99:0/2476901859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbdfd0 0x55cda0fbe450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.333+0000 7f75e7e00700 1 -- 172.21.15.99:0/2476901859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cda0d8f3b0 con 0x55cda0fbc7d0 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.333+0000 7f75ed000700 1 --2- 172.21.15.99:0/2476901859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbdfd0 0x55cda0fbe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.333+0000 7f75ed000700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75cc0a11a0 0x7f75cc0a3660 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f75e0007db0 tx=0x7f75e00094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.333+0000 7f75e7e00700 1 --2- 172.21.15.99:0/2476901859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f75d8000c00 tx=0x7f75d8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fcce82d6c27f37 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.333+0000 7f75dd600700 1 -- 172.21.15.99:0/2476901859 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8022ea0 con 0x55cda0fbc7d0 2024-03-30T00:35:21.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.333+0000 7f75dd600700 1 -- 172.21.15.99:0/2476901859 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75d8095e90 con 0x55cda0fbc7d0 2024-03-30T00:35:21.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.333+0000 7f75dd600700 1 -- 172.21.15.99:0/2476901859 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d809a9d0 con 0x55cda0fbc7d0 2024-03-30T00:35:21.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.334+0000 7f7602ab8b80 1 -- 172.21.15.99:0/2476901859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 msgr2=0x55cda0fbcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.334+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/2476901859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f75d8000c00 tx=0x7f75d8009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.334+0000 7f7602ab8b80 1 -- 172.21.15.99:0/2476901859 shutdown_connections 2024-03-30T00:35:21.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.334+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/2476901859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbdfd0 0x55cda0fbe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.334+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/2476901859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.334+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/2476901859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbd130 0x55cda0fbd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.334+0000 7f7602ab8b80 1 -- 172.21.15.99:0/2476901859 >> 172.21.15.99:0/2476901859 conn(0x55cda0f1b6a0 msgr2=0x55cda0fb79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.334+0000 7f7602ab8b80 1 -- 172.21.15.99:0/2476901859 shutdown_connections 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.334+0000 7f7602ab8b80 1 -- 172.21.15.99:0/2476901859 wait complete. 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.335+0000 7f7602ab8b80 1 -- start start 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.335+0000 7f7602ab8b80 1 Processor -- start 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.335+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbc7d0 0x55cda0fc62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.335+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbd130 0x55cda0fc6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.335+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.335+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0ea9d80 con 0x55cda0fbdfd0 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.335+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0ea9a20 con 0x55cda0fbd130 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.335+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0d916a0 con 0x55cda0fbc7d0 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.335+0000 7f7602ab8b80 10 client.0 _ll_register_callbacks cb 0x55cda0eb3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.335+0000 7f75ed000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75ed000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34490/0 (socket says 172.21.15.99:34490) 2024-03-30T00:35:21.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75ed000700 1 -- 172.21.15.99:0/3639902800 learned_addr learned my addr 172.21.15.99:0/3639902800 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75e7400700 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbd130 0x55cda0fc6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75e7e00700 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbc7d0 0x55cda0fc62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75ed000700 1 -- 172.21.15.99:0/3639902800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbc7d0 msgr2=0x55cda0fc62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75ed000700 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbc7d0 0x55cda0fc62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75ed000700 1 -- 172.21.15.99:0/3639902800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbd130 msgr2=0x55cda0fc6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75ed000700 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbd130 0x55cda0fc6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75ed000700 1 -- 172.21.15.99:0/3639902800 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e000ed40 con 0x55cda0fbdfd0 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75ed000700 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcd860 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f75e001b9b0 tx=0x7f75e0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f363b9ba6302be4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75d2000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75d2000700 1 -- 172.21.15.99:0/3639902800 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e001a840 con 0x55cda0fbdfd0 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f75d2000700 1 -- 172.21.15.99:0/3639902800 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f75e0021040 con 0x55cda0fbdfd0 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.336+0000 7f7602ab8b80 1 -- 172.21.15.99:0/3639902800 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cda0ece380 con 0x55cda0fbdfd0 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.337+0000 7f75d2000700 1 -- 172.21.15.99:0/3639902800 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75e0007760 con 0x55cda0fbdfd0 2024-03-30T00:35:21.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.337+0000 7f7602ab8b80 10 client.15630 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.337+0000 7f7602ab8b80 1 -- 172.21.15.99:0/3639902800 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f75e81a12b0 con 0x55cda0fbdfd0 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.337+0000 7f75d2000700 1 -- 172.21.15.99:0/3639902800 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cda0ece380 con 0x55cda0fbdfd0 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.337+0000 7f75d2000700 1 -- 172.21.15.99:0/3639902800 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f75e0025830 con 0x55cda0fbdfd0 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.337+0000 7f7602ab8b80 10 client.15630 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.337+0000 7f7602ab8b80 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.337+0000 7f7602ab8b80 20 client.15630 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.337+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cda0fd2190 0x55cda0fd45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.337+0000 7f7602ab8b80 4 client.15630 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.337+0000 7f7602ab8b80 1 -- 172.21.15.99:0/3639902800 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cda0f06700 con 0x55cda0fd2190 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.338+0000 7f75e7e00700 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cda0fd2190 0x55cda0fd45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.338+0000 7f75e7e00700 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cda0fd2190 0x55cda0fd45b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24604420c6b0fb81 server_cookie=a75c7f7fb5fdd237 in_seq=0 out_seq=0 2024-03-30T00:35:21.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.338+0000 7f75d2000700 10 client.15630 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:21.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f75d2000700 1 -- 172.21.15.99:0/3639902800 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cda0f06700 con 0x55cda0fd2190 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f75d2000700 10 client.15630 handle_command_reply: tid=0 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f7602ab8b80 1 client.15630 shutdown 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f7602ab8b80 1 -- 172.21.15.99:0/3639902800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbdfd0 msgr2=0x55cda0fcd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcd860 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f75e001b9b0 tx=0x7f75e0004b40 comp rx=0 tx=0).stop 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f7602ab8b80 1 -- 172.21.15.99:0/3639902800 shutdown_connections 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cda0fd2190 0x55cda0fd45b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcd860 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbc7d0 0x55cda0fc62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/3639902800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbd130 0x55cda0fc6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f75d2000700 0 client.15630 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f7602ab8b80 1 -- 172.21.15.99:0/3639902800 >> 172.21.15.99:0/3639902800 conn(0x55cda0f1b6a0 msgr2=0x55cda0fbb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f7602ab8b80 1 -- 172.21.15.99:0/3639902800 shutdown_connections 2024-03-30T00:35:21.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.339+0000 7f7602ab8b80 1 -- 172.21.15.99:0/3639902800 wait complete. 2024-03-30T00:35:21.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.340+0000 7f7602ab8b80 20 client.15630 trim_cache size 0 max 16384 2024-03-30T00:35:21.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.357+0000 7f7602ab8b80 1 Processor -- start 2024-03-30T00:35:21.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.357+0000 7f7602ab8b80 1 -- start start 2024-03-30T00:35:21.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.357+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.357+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbd130 0x55cda0fcd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.357+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.357+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0fc5e50 con 0x55cda0fbd130 2024-03-30T00:35:21.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.357+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0fd4940 con 0x55cda0fbc7d0 2024-03-30T00:35:21.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.357+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0fb3480 con 0x55cda0fbdfd0 2024-03-30T00:35:21.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75e7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75e7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:35892/0 (socket says 172.21.15.99:35892) 2024-03-30T00:35:21.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75e7e00700 1 -- 172.21.15.99:0/1853103237 learned_addr learned my addr 172.21.15.99:0/1853103237 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:21.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75e7400700 1 --2- 172.21.15.99:0/1853103237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbd130 0x55cda0fcd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75ed000700 1 --2- 172.21.15.99:0/1853103237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75e7e00700 1 -- 172.21.15.99:0/1853103237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 msgr2=0x55cda0fcdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75e7e00700 1 --2- 172.21.15.99:0/1853103237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75e7e00700 1 -- 172.21.15.99:0/1853103237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbd130 msgr2=0x55cda0fcd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75e7e00700 1 --2- 172.21.15.99:0/1853103237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbd130 0x55cda0fcd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75e7e00700 1 -- 172.21.15.99:0/1853103237 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cda0d3bb60 con 0x55cda0fbc7d0 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75ed000700 1 --2- 172.21.15.99:0/1853103237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75e7400700 1 --2- 172.21.15.99:0/1853103237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbd130 0x55cda0fcd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75e7e00700 1 --2- 172.21.15.99:0/1853103237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f75d809b8e0 tx=0x7f75d80a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ff9c2c02cdc469a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.358+0000 7f75de000700 1 -- 172.21.15.99:0/1853103237 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d80b9040 con 0x55cda0fbc7d0 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.359+0000 7f75de000700 1 -- 172.21.15.99:0/1853103237 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75d809f130 con 0x55cda0fbc7d0 2024-03-30T00:35:21.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.359+0000 7f75de000700 1 -- 172.21.15.99:0/1853103237 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d809a9d0 con 0x55cda0fbc7d0 2024-03-30T00:35:21.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.359+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1853103237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbc7d0 msgr2=0x55cda0fbcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.359+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/1853103237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f75d809b8e0 tx=0x7f75d80a1040 comp rx=0 tx=0).stop 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.359+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1853103237 shutdown_connections 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.359+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/1853103237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.359+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/1853103237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbc7d0 0x55cda0fbcbb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.359+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/1853103237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbd130 0x55cda0fcd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.359+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1853103237 >> 172.21.15.99:0/1853103237 conn(0x55cda0f1b6a0 msgr2=0x55cda0fb05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.359+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1853103237 shutdown_connections 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.359+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1853103237 wait complete. 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.360+0000 7f7602ab8b80 1 -- start start 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.360+0000 7f7602ab8b80 1 Processor -- start 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fcb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbd130 0x55cda0fcb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f7602ab8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0fccdc0 con 0x55cda0fbc7d0 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0fccf30 con 0x55cda0fbd130 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f7602ab8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cda0fcd0f0 con 0x55cda0fbdfd0 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f7602ab8b80 10 client.0 _ll_register_callbacks cb 0x55cda0eb3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f75e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fcb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f75ed000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f75e7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbd130 0x55cda0fcb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f75e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fcb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34506/0 (socket says 172.21.15.99:34506) 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f75e7e00700 1 -- 172.21.15.99:0/1928420562 learned_addr learned my addr 172.21.15.99:0/1928420562 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f75e7e00700 1 -- 172.21.15.99:0/1928420562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 msgr2=0x55cda0fcc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f75e7e00700 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f75e7e00700 1 -- 172.21.15.99:0/1928420562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbd130 msgr2=0x55cda0fcb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f75e7e00700 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbd130 0x55cda0fcb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f75e7e00700 1 -- 172.21.15.99:0/1928420562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cda0d8f3b0 con 0x55cda0fbc7d0 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.361+0000 7f75ed000700 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f75e7e00700 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fcb330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f75d8000c00 tx=0x7f75d8005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0d6aa56574e74f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f75d2000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f75d2000700 1 -- 172.21.15.99:0/1928420562 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d80b9040 con 0x55cda0fbc7d0 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f75d2000700 1 -- 172.21.15.99:0/1928420562 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f75d80a3670 con 0x55cda0fbc7d0 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1928420562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75d8095e90 con 0x55cda0fbc7d0 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f75d2000700 1 -- 172.21.15.99:0/1928420562 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75d809a980 con 0x55cda0fbc7d0 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f7602ab8b80 10 client.15636 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1928420562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cda0ed3ea0 con 0x55cda0fbc7d0 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f75d2000700 1 -- 172.21.15.99:0/1928420562 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75d8095e90 con 0x55cda0fbc7d0 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f75d2000700 1 -- 172.21.15.99:0/1928420562 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f75d8097070 con 0x55cda0fbc7d0 2024-03-30T00:35:21.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f7602ab8b80 10 client.15636 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f7602ab8b80 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f7602ab8b80 20 client.15636 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cda1026890 0x55cda1028cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f7602ab8b80 4 client.15636 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:35:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.362+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1928420562 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cda0d5ccf0 con 0x55cda1026890 2024-03-30T00:35:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.363+0000 7f75e7400700 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cda1026890 0x55cda1028cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.363+0000 7f75e7400700 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cda1026890 0x55cda1028cb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33280569445744c5 server_cookie=f88a7851b39c8ec8 in_seq=0 out_seq=0 2024-03-30T00:35:21.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.363+0000 7f75d2000700 10 client.15636 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f75d2000700 1 -- 172.21.15.99:0/1928420562 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cda0d5ccf0 con 0x55cda1026890 2024-03-30T00:35:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f75d2000700 10 client.15636 handle_command_reply: tid=0 2024-03-30T00:35:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 1 client.15636 shutdown 2024-03-30T00:35:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1928420562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 msgr2=0x55cda0fcb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fcb330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f75d8000c00 tx=0x7f75d8005fb0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1928420562 shutdown_connections 2024-03-30T00:35:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cda1026890 0x55cda1028cb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cda0fbdfd0 0x55cda0fcc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cda0fbc7d0 0x55cda0fcb330 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 1 --2- 172.21.15.99:0/1928420562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cda0fbd130 0x55cda0fcb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f75d2000700 0 client.15636 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1928420562 >> 172.21.15.99:0/1928420562 conn(0x55cda0f1b6a0 msgr2=0x55cda0fc11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1928420562 shutdown_connections 2024-03-30T00:35:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 1 -- 172.21.15.99:0/1928420562 wait complete. 2024-03-30T00:35:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.364+0000 7f7602ab8b80 20 client.15636 trim_cache size 0 max 16384 2024-03-30T00:35:21.375 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:35:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:35:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": { 2024-03-30T00:35:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "1700173d-da29-48fc-ac7f-8c0f5aea4859": { 2024-03-30T00:35:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/", 2024-03-30T00:35:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tag": "1700173d-da29-48fc-ac7f-8c0f5aea4859", 2024-03-30T00:35:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "options": "recursive,force" 2024-03-30T00:35:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:35:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:35:21.376 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:35:21.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 -- 172.21.15.99:0/791648318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75cc0a11a0 msgr2=0x7f75cc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75cc0a11a0 0x7f75cc0a3660 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f75e0007db0 tx=0x7f75e00094a0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 -- 172.21.15.99:0/791648318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e8073810 msgr2=0x7f75e8198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:21.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e8073810 0x7f75e8198e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f75d8003f60 tx=0x7f75d8005740 comp rx=0 tx=0).stop 2024-03-30T00:35:21.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 -- 172.21.15.99:0/791648318 shutdown_connections 2024-03-30T00:35:21.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75e8108c40 0x7f75e81a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f75cc0a11a0 0x7f75cc0a3660 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75e8073810 0x7f75e8198e70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 --2- 172.21.15.99:0/791648318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75e8074df0 0x7f75e81993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:21.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 -- 172.21.15.99:0/791648318 >> 172.21.15.99:0/791648318 conn(0x7f75e80fc2f0 msgr2=0x7f75e8104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:21.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 -- 172.21.15.99:0/791648318 shutdown_connections 2024-03-30T00:35:21.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:21.365+0000 7f75ef000700 1 -- 172.21.15.99:0/791648318 wait complete. 2024-03-30T00:35:21.389 INFO:tasks.cephfs.filesystem:scrub status for tag:1700173d-da29-48fc-ac7f-8c0f5aea4859 - {'path': '/', 'tag': '1700173d-da29-48fc-ac7f-8c0f5aea4859', 'options': 'recursive,force'} 2024-03-30T00:35:21.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:35:21.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:35:21.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:21.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:35:21.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:35:21.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:21.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:35:21.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:35:21.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:35:22.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:22 smithi142 ceph-mon[41779]: pgmap v323: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 290 B/s wr, 7 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-30T00:35:22.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:22 smithi142 ceph-mon[41779]: scrub queued for path: / 2024-03-30T00:35:22.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:22 smithi142 ceph-mon[41779]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:22.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:22 smithi142 ceph-mon[41779]: scrub summary: active paths [/] 2024-03-30T00:35:22.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:22 smithi142 ceph-mon[41779]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:22.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:22 smithi161 ceph-mon[42221]: pgmap v323: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 290 B/s wr, 7 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-30T00:35:22.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:22 smithi161 ceph-mon[42221]: scrub queued for path: / 2024-03-30T00:35:22.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:22 smithi161 ceph-mon[42221]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:22.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:22 smithi161 ceph-mon[42221]: scrub summary: active paths [/] 2024-03-30T00:35:22.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:22 smithi161 ceph-mon[42221]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:22.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:22 smithi099 ceph-mon[38079]: pgmap v323: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 290 B/s wr, 7 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-03-30T00:35:22.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:22 smithi099 ceph-mon[38079]: scrub queued for path: / 2024-03-30T00:35:22.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:22 smithi099 ceph-mon[38079]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:22.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:22 smithi099 ceph-mon[38079]: scrub summary: active paths [/] 2024-03-30T00:35:22.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:22 smithi099 ceph-mon[38079]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:24.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:24 smithi142 ceph-mon[41779]: pgmap v324: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-03-30T00:35:24.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:24 smithi142 ceph-mon[41779]: scrub summary: idle 2024-03-30T00:35:24.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:24 smithi161 ceph-mon[42221]: pgmap v324: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-03-30T00:35:24.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:24 smithi161 ceph-mon[42221]: scrub summary: idle 2024-03-30T00:35:24.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:24 smithi099 ceph-mon[38079]: pgmap v324: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-03-30T00:35:24.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:24 smithi099 ceph-mon[38079]: scrub summary: idle 2024-03-30T00:35:26.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:26 smithi142 ceph-mon[41779]: pgmap v325: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T00:35:26.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:26 smithi161 ceph-mon[42221]: pgmap v325: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T00:35:26.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:26 smithi099 ceph-mon[38079]: pgmap v325: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T00:35:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.326+0000 7fef04a00700 1 -- 172.21.15.99:0/111319323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef00101780 msgr2=0x7fef00103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.326+0000 7fef04a00700 1 --2- 172.21.15.99:0/111319323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef00101780 0x7fef00103b70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7feef0009990 tx=0x7feef0009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.326+0000 7fef04a00700 1 -- 172.21.15.99:0/111319323 shutdown_connections 2024-03-30T00:35:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.326+0000 7fef04a00700 1 --2- 172.21.15.99:0/111319323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef001069e0 0x7fef00108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.326+0000 7fef04a00700 1 --2- 172.21.15.99:0/111319323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef00101780 0x7fef00103b70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.326+0000 7fef04a00700 1 --2- 172.21.15.99:0/111319323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef001040b0 0x7fef001064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.326+0000 7fef04a00700 1 -- 172.21.15.99:0/111319323 >> 172.21.15.99:0/111319323 conn(0x7fef000fb170 msgr2=0x7fef000fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:27.333 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.326+0000 7fef04a00700 1 -- 172.21.15.99:0/111319323 shutdown_connections 2024-03-30T00:35:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.326+0000 7fef04a00700 1 -- 172.21.15.99:0/111319323 wait complete. 2024-03-30T00:35:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.327+0000 7fef04a00700 1 Processor -- start 2024-03-30T00:35:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7fef04a00700 1 -- start start 2024-03-30T00:35:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7fef04a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef00101780 0x7fef00198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7fef04a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef001040b0 0x7fef00199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7fef04a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef001069e0 0x7fef001a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7fef04a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef001a08f0 con 0x7fef001040b0 2024-03-30T00:35:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7fef04a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef001a0a60 con 0x7fef00101780 2024-03-30T00:35:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7fef04a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef001a0c20 con 0x7fef001069e0 2024-03-30T00:35:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7feefd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef001040b0 0x7fef00199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7feefd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef001040b0 0x7fef00199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50432/0 (socket says 172.21.15.99:50432) 2024-03-30T00:35:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7feefd400700 1 -- 172.21.15.99:0/212770079 learned_addr learned my addr 172.21.15.99:0/212770079 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7feefe800700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef001069e0 0x7fef001a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.328+0000 7feefde00700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef00101780 0x7fef00198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.329+0000 7feefd400700 1 -- 172.21.15.99:0/212770079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef001069e0 msgr2=0x7fef001a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.329+0000 7feefd400700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef001069e0 0x7fef001a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.329+0000 7feefd400700 1 -- 172.21.15.99:0/212770079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef00101780 msgr2=0x7fef00198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.329+0000 7feefd400700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef00101780 0x7fef00198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.329+0000 7feefd400700 1 -- 172.21.15.99:0/212770079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef00095f0 con 0x7fef001040b0 2024-03-30T00:35:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.329+0000 7feefd400700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef001040b0 0x7fef00199340 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feeec00d950 tx=0x7feeec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.329+0000 7feef6a00700 1 -- 172.21.15.99:0/212770079 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec00f890 con 0x7fef001040b0 2024-03-30T00:35:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.329+0000 7feef6a00700 1 -- 172.21.15.99:0/212770079 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feeec00fe80 con 0x7fef001040b0 2024-03-30T00:35:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.329+0000 7feef6a00700 1 -- 172.21.15.99:0/212770079 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec00e3b0 con 0x7fef001040b0 2024-03-30T00:35:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.329+0000 7fef04a00700 1 -- 172.21.15.99:0/212770079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef001a0f00 con 0x7fef001040b0 2024-03-30T00:35:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.330+0000 7fef04a00700 1 -- 172.21.15.99:0/212770079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef001a12d0 con 0x7fef001040b0 2024-03-30T00:35:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.332+0000 7feef6a00700 1 -- 172.21.15.99:0/212770079 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7feeec010460 con 0x7fef001040b0 2024-03-30T00:35:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.335+0000 7feef6a00700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feee40a11a0 0x7feee40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.335+0000 7feef6a00700 1 -- 172.21.15.99:0/212770079 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7feeec0cc720 con 0x7fef001040b0 2024-03-30T00:35:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.335+0000 7fef185e9b80 1 Processor -- start 2024-03-30T00:35:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.335+0000 7fef185e9b80 1 -- start start 2024-03-30T00:35:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.335+0000 7feefde00700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feee40a11a0 0x7feee40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.335+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a4512fe620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.335+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a4512ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.335+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512ffa60 0x55a4512ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7fef185e9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a451256930 con 0x55a4512feb60 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7fef185e9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4512542f0 con 0x55a4512ffa60 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7fef185e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45124fbe0 con 0x55a4512fe200 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a4512ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a4512ff4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50436/0 (socket says 172.21.15.99:50436) 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefd400700 1 -- 172.21.15.99:0/2914647209 learned_addr learned my addr 172.21.15.99:0/2914647209 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefe800700 1 --2- 172.21.15.99:0/2914647209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512ffa60 0x55a4512ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefde00700 1 --2- 172.21.15.99:0/2914647209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a4512fe620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefd400700 1 -- 172.21.15.99:0/2914647209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 msgr2=0x55a4512fe620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefd400700 1 --2- 172.21.15.99:0/2914647209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a4512fe620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefd400700 1 -- 172.21.15.99:0/2914647209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512ffa60 msgr2=0x55a4512ffef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefd400700 1 --2- 172.21.15.99:0/2914647209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512ffa60 0x55a4512ffef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefd400700 1 -- 172.21.15.99:0/2914647209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4510d1350 con 0x55a4512feb60 2024-03-30T00:35:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefe800700 1 --2- 172.21.15.99:0/2914647209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512ffa60 0x55a4512ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.336+0000 7feefde00700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feee40a11a0 0x7feee40a3660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feef0000c00 tx=0x7feef0005c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.337+0000 7feefd400700 1 --2- 172.21.15.99:0/2914647209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a4512ff4d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feeec099a20 tx=0x7feeec0994c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acede4e5b903468 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.337+0000 7feeeb000700 1 -- 172.21.15.99:0/2914647209 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec099a90 con 0x55a4512feb60 2024-03-30T00:35:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.337+0000 7feeeb000700 1 -- 172.21.15.99:0/2914647209 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feeec09d8c0 con 0x55a4512feb60 2024-03-30T00:35:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.337+0000 7feeeb000700 1 -- 172.21.15.99:0/2914647209 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0a1a80 con 0x55a4512feb60 2024-03-30T00:35:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.337+0000 7fef185e9b80 1 -- 172.21.15.99:0/2914647209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 msgr2=0x55a4512ff4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.337+0000 7fef185e9b80 1 --2- 172.21.15.99:0/2914647209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a4512ff4d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feeec099a20 tx=0x7feeec0994c0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.338+0000 7fef185e9b80 1 -- 172.21.15.99:0/2914647209 shutdown_connections 2024-03-30T00:35:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.338+0000 7fef185e9b80 1 --2- 172.21.15.99:0/2914647209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512ffa60 0x55a4512ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.338+0000 7fef185e9b80 1 --2- 172.21.15.99:0/2914647209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a4512fe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.338+0000 7fef185e9b80 1 --2- 172.21.15.99:0/2914647209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a4512ff4d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.338+0000 7fef185e9b80 1 -- 172.21.15.99:0/2914647209 >> 172.21.15.99:0/2914647209 conn(0x55a45125d640 msgr2=0x55a4512f9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.338+0000 7fef185e9b80 1 -- 172.21.15.99:0/2914647209 shutdown_connections 2024-03-30T00:35:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.338+0000 7fef185e9b80 1 -- 172.21.15.99:0/2914647209 wait complete. 2024-03-30T00:35:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.338+0000 7fef185e9b80 1 -- start start 2024-03-30T00:35:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.339+0000 7fef185e9b80 1 Processor -- start 2024-03-30T00:35:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.339+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a451308160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.339+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512feb60 0x55a4513086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.339+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512ffa60 0x55a45130f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.339+0000 7fef185e9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4511ebd20 con 0x55a4512ffa60 2024-03-30T00:35:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.339+0000 7fef185e9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4511eb9c0 con 0x55a4512feb60 2024-03-30T00:35:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.339+0000 7fef185e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4510d3640 con 0x55a4512fe200 2024-03-30T00:35:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.339+0000 7fef185e9b80 10 client.0 _ll_register_callbacks cb 0x55a4511f5410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feefde00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a451308160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feefe800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512ffa60 0x55a45130f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feefde00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a451308160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:37378/0 (socket says 172.21.15.99:37378) 2024-03-30T00:35:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feefde00700 1 -- 172.21.15.99:0/851384448 learned_addr learned my addr 172.21.15.99:0/851384448 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feefd400700 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512feb60 0x55a4513086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feefe800700 1 -- 172.21.15.99:0/851384448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 msgr2=0x55a451308160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feefe800700 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a451308160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feefe800700 1 -- 172.21.15.99:0/851384448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512feb60 msgr2=0x55a4513086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feefe800700 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512feb60 0x55a4513086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feefe800700 1 -- 172.21.15.99:0/851384448 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef001a12d0 con 0x55a4512ffa60 2024-03-30T00:35:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feefe800700 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512ffa60 0x55a45130f6e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feef8004070 tx=0x7feef8004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb5d15e6ab34890 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feedf400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.340+0000 7feedf400700 1 -- 172.21.15.99:0/851384448 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8016870 con 0x55a4512ffa60 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7feedf400700 1 -- 172.21.15.99:0/851384448 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7feef80169d0 con 0x55a4512ffa60 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7fef185e9b80 1 -- 172.21.15.99:0/851384448 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a451210320 con 0x55a4512ffa60 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7feedf400700 1 -- 172.21.15.99:0/851384448 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feef8004bb0 con 0x55a4512ffa60 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7fef185e9b80 10 client.15648 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7fef185e9b80 1 -- 172.21.15.99:0/851384448 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a45107db00 con 0x55a4512ffa60 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7feedf400700 1 -- 172.21.15.99:0/851384448 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a451210320 con 0x55a4512ffa60 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7feedf400700 1 -- 172.21.15.99:0/851384448 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feef8014040 con 0x55a4512ffa60 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7fef185e9b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7fef185e9b80 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7fef185e9b80 20 client.15648 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7fef185e9b80 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a451314010 0x55a451316430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7fef185e9b80 4 client.15648 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:27.352 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.341+0000 7fef185e9b80 1 -- 172.21.15.99:0/851384448 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4512486a0 con 0x55a451314010 2024-03-30T00:35:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.342+0000 7feefde00700 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a451314010 0x55a451316430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.343+0000 7feefde00700 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a451314010 0x55a451316430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f4e7bb2d5ff1bd server_cookie=893cc059c20148a in_seq=0 out_seq=0 2024-03-30T00:35:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.343+0000 7feedf400700 10 client.15648 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.343+0000 7feedf400700 1 -- 172.21.15.99:0/851384448 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a4512486a0 con 0x55a451314010 2024-03-30T00:35:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.343+0000 7feedf400700 10 client.15648 handle_command_reply: tid=0 2024-03-30T00:35:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.343+0000 7fef185e9b80 1 client.15648 shutdown 2024-03-30T00:35:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7fef185e9b80 1 -- 172.21.15.99:0/851384448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512ffa60 msgr2=0x55a45130f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7fef185e9b80 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512ffa60 0x55a45130f6e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feef8004070 tx=0x7feef8004380 comp rx=0 tx=0).stop 2024-03-30T00:35:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7fef185e9b80 1 -- 172.21.15.99:0/851384448 shutdown_connections 2024-03-30T00:35:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7fef185e9b80 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a451314010 0x55a451316430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7fef185e9b80 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512ffa60 0x55a45130f6e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7fef185e9b80 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a451308160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7fef185e9b80 1 --2- 172.21.15.99:0/851384448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512feb60 0x55a4513086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7fef185e9b80 1 -- 172.21.15.99:0/851384448 >> 172.21.15.99:0/851384448 conn(0x55a45125d640 msgr2=0x55a4512fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7feedf400700 0 client.15648 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7fef185e9b80 1 -- 172.21.15.99:0/851384448 shutdown_connections 2024-03-30T00:35:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7fef185e9b80 1 -- 172.21.15.99:0/851384448 wait complete. 2024-03-30T00:35:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.344+0000 7fef185e9b80 20 client.15648 trim_cache size 0 max 16384 2024-03-30T00:35:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.361+0000 7fef185e9b80 1 Processor -- start 2024-03-30T00:35:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.361+0000 7fef185e9b80 1 -- start start 2024-03-30T00:35:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512fe200 0x55a4512fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512feb60 0x55a45130fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512ffa60 0x55a4512f3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7fef185e9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a451311470 con 0x55a4512fe200 2024-03-30T00:35:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7fef185e9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4512faa30 con 0x55a4512feb60 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7fef185e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a451305cc0 con 0x55a4512ffa60 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7feefde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512fe200 0x55a4512fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7feefde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512fe200 0x55a4512fe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50448/0 (socket says 172.21.15.99:50448) 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7feefde00700 1 -- 172.21.15.99:0/1212052669 learned_addr learned my addr 172.21.15.99:0/1212052669 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7feefd400700 1 --2- 172.21.15.99:0/1212052669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512feb60 0x55a45130fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7feefe800700 1 --2- 172.21.15.99:0/1212052669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512ffa60 0x55a4512f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7feefde00700 1 -- 172.21.15.99:0/1212052669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512ffa60 msgr2=0x55a4512f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7feefde00700 1 --2- 172.21.15.99:0/1212052669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512ffa60 0x55a4512f3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7feefde00700 1 -- 172.21.15.99:0/1212052669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512feb60 msgr2=0x55a45130fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7feefde00700 1 --2- 172.21.15.99:0/1212052669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512feb60 0x55a45130fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.362+0000 7feefde00700 1 -- 172.21.15.99:0/1212052669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a451215e40 con 0x55a4512fe200 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.363+0000 7feefde00700 1 --2- 172.21.15.99:0/1212052669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512fe200 0x55a4512fe5e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7feef000c010 tx=0x7feef0022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ded99c11c61227 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.363+0000 7feeeba00700 1 -- 172.21.15.99:0/1212052669 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef000b8b0 con 0x55a4512fe200 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.363+0000 7feeeba00700 1 -- 172.21.15.99:0/1212052669 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feef001e040 con 0x55a4512fe200 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.363+0000 7feeeba00700 1 -- 172.21.15.99:0/1212052669 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0003fc0 con 0x55a4512fe200 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.364+0000 7fef185e9b80 1 -- 172.21.15.99:0/1212052669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512fe200 msgr2=0x55a4512fe5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.364+0000 7fef185e9b80 1 --2- 172.21.15.99:0/1212052669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512fe200 0x55a4512fe5e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7feef000c010 tx=0x7feef0022000 comp rx=0 tx=0).stop 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.364+0000 7fef185e9b80 1 -- 172.21.15.99:0/1212052669 shutdown_connections 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.364+0000 7fef185e9b80 1 --2- 172.21.15.99:0/1212052669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512ffa60 0x55a4512f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.364+0000 7fef185e9b80 1 --2- 172.21.15.99:0/1212052669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512fe200 0x55a4512fe5e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.364+0000 7fef185e9b80 1 --2- 172.21.15.99:0/1212052669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512feb60 0x55a45130fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.364+0000 7fef185e9b80 1 -- 172.21.15.99:0/1212052669 >> 172.21.15.99:0/1212052669 conn(0x55a45125d640 msgr2=0x55a45131ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.364+0000 7fef185e9b80 1 -- 172.21.15.99:0/1212052669 shutdown_connections 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.364+0000 7fef185e9b80 1 -- 172.21.15.99:0/1212052669 wait complete. 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.364+0000 7fef185e9b80 1 -- start start 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.364+0000 7fef185e9b80 1 Processor -- start 2024-03-30T00:35:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a45130d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a45130d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7fef185e9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512ffa60 0x55a45130e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7fef185e9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45130ebc0 con 0x55a4512feb60 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7fef185e9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45130ed30 con 0x55a4512ffa60 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7fef185e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45130eef0 con 0x55a4512fe200 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7fef185e9b80 10 client.0 _ll_register_callbacks cb 0x55a4511f5410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7feefd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a45130d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7feefd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a45130d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50456/0 (socket says 172.21.15.99:50456) 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7feefd400700 1 -- 172.21.15.99:0/1977922085 learned_addr learned my addr 172.21.15.99:0/1977922085 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7feefde00700 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a45130d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.365+0000 7feefe800700 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512ffa60 0x55a45130e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7feefd400700 1 -- 172.21.15.99:0/1977922085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 msgr2=0x55a45130d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7feefd400700 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a45130d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7feefd400700 1 -- 172.21.15.99:0/1977922085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512ffa60 msgr2=0x55a45130e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7feefd400700 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512ffa60 0x55a45130e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7feefd400700 1 -- 172.21.15.99:0/1977922085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef00109a40 con 0x55a4512feb60 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7feefd400700 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a45130d6a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feeec096500 tx=0x7feeec00dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f445da325ae2de6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7feedf400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7feedf400700 1 -- 172.21.15.99:0/1977922085 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0a1f10 con 0x55a4512feb60 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7feedf400700 1 -- 172.21.15.99:0/1977922085 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7feeec0b2240 con 0x55a4512feb60 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7fef185e9b80 1 -- 172.21.15.99:0/1977922085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feeec09d8c0 con 0x55a4512feb60 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7feedf400700 1 -- 172.21.15.99:0/1977922085 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feeec0b2400 con 0x55a4512feb60 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7fef185e9b80 10 client.15654 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7fef185e9b80 1 -- 172.21.15.99:0/1977922085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a4510d5c70 con 0x55a4512feb60 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.366+0000 7feedf400700 1 -- 172.21.15.99:0/1977922085 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feeec09d8c0 con 0x55a4512feb60 2024-03-30T00:35:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.367+0000 7feedf400700 1 -- 172.21.15.99:0/1977922085 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feeec0a1b10 con 0x55a4512feb60 2024-03-30T00:35:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.367+0000 7fef185e9b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.367+0000 7fef185e9b80 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.367+0000 7fef185e9b80 20 client.15654 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.367+0000 7fef185e9b80 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a4513686c0 0x55a45136aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.367+0000 7fef185e9b80 4 client.15654 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:35:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.367+0000 7fef185e9b80 1 -- 172.21.15.99:0/1977922085 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a45109ec90 con 0x55a4513686c0 2024-03-30T00:35:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.367+0000 7feefde00700 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a4513686c0 0x55a45136aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.368+0000 7feefde00700 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a4513686c0 0x55a45136aab0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa624f60af3f698 server_cookie=1864c5f79e6f7c5d in_seq=0 out_seq=0 2024-03-30T00:35:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.368+0000 7feedf400700 10 client.15654 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.372+0000 7feedf400700 1 -- 172.21.15.99:0/1977922085 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a45109ec90 con 0x55a4513686c0 2024-03-30T00:35:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.372+0000 7feedf400700 10 client.15654 handle_command_reply: tid=0 2024-03-30T00:35:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.372+0000 7fef185e9b80 1 client.15654 shutdown 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.372+0000 7fef185e9b80 1 -- 172.21.15.99:0/1977922085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 msgr2=0x55a45130d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.372+0000 7fef185e9b80 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a45130d6a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feeec096500 tx=0x7feeec00dfd0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.373+0000 7fef185e9b80 1 -- 172.21.15.99:0/1977922085 shutdown_connections 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.373+0000 7fef185e9b80 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55a4513686c0 0x55a45136aab0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.373+0000 7fef185e9b80 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4512ffa60 0x55a45130e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.373+0000 7fef185e9b80 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4512fe200 0x55a45130d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.373+0000 7feedf400700 0 client.15654 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.373+0000 7fef185e9b80 1 --2- 172.21.15.99:0/1977922085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a4512feb60 0x55a45130d6a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.373+0000 7fef185e9b80 1 -- 172.21.15.99:0/1977922085 >> 172.21.15.99:0/1977922085 conn(0x55a45125d640 msgr2=0x55a451305fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.373+0000 7fef185e9b80 1 -- 172.21.15.99:0/1977922085 shutdown_connections 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.373+0000 7fef185e9b80 1 -- 172.21.15.99:0/1977922085 wait complete. 2024-03-30T00:35:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.373+0000 7fef185e9b80 20 client.15654 trim_cache size 0 max 16384 2024-03-30T00:35:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:35:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:35:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:35:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 -- 172.21.15.99:0/212770079 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feee40a11a0 msgr2=0x7feee40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feee40a11a0 0x7feee40a3660 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feef0000c00 tx=0x7feef0005c20 comp rx=0 tx=0).stop 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 -- 172.21.15.99:0/212770079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef001040b0 msgr2=0x7fef00199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef001040b0 0x7fef00199340 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feeec00d950 tx=0x7feeec00dc60 comp rx=0 tx=0).stop 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 -- 172.21.15.99:0/212770079 shutdown_connections 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef001069e0 0x7fef001a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7feee40a11a0 0x7feee40a3660 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef00101780 0x7fef00198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 --2- 172.21.15.99:0/212770079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef001040b0 0x7fef00199340 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 -- 172.21.15.99:0/212770079 >> 172.21.15.99:0/212770079 conn(0x7fef000fb170 msgr2=0x7fef000ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 -- 172.21.15.99:0/212770079 shutdown_connections 2024-03-30T00:35:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:27.374+0000 7fef04a00700 1 -- 172.21.15.99:0/212770079 wait complete. 2024-03-30T00:35:28.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:28 smithi142 ceph-mon[41779]: pgmap v326: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T00:35:28.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:28 smithi161 ceph-mon[42221]: pgmap v326: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T00:35:28.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:28 smithi099 ceph-mon[38079]: pgmap v326: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T00:35:30.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:30 smithi142 ceph-mon[41779]: pgmap v327: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-30T00:35:30.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:30 smithi161 ceph-mon[42221]: pgmap v327: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-30T00:35:30.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:30 smithi099 ceph-mon[38079]: pgmap v327: 97 pgs: 97 active+clean; 661 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-30T00:35:32.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:32 smithi142 ceph-mon[41779]: pgmap v328: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T00:35:32.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:32 smithi161 ceph-mon[42221]: pgmap v328: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T00:35:32.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:32 smithi099 ceph-mon[38079]: pgmap v328: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T00:35:33.969 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:33 smithi161 ceph-mon[42221]: pgmap v329: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:33.987 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:33 smithi099 ceph-mon[38079]: pgmap v329: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:34.066 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:33 smithi142 ceph-mon[41779]: pgmap v329: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:36.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:36 smithi161 ceph-mon[42221]: pgmap v330: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:36.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:36 smithi099 ceph-mon[38079]: pgmap v330: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:36.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:36 smithi142 ceph-mon[41779]: pgmap v330: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:38.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:38 smithi161 ceph-mon[42221]: pgmap v331: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:35:38.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:38 smithi099 ceph-mon[38079]: pgmap v331: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:35:38.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:38 smithi142 ceph-mon[41779]: pgmap v331: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:35:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:40 smithi161 ceph-mon[42221]: pgmap v332: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:35:40.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:40 smithi099 ceph-mon[38079]: pgmap v332: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:35:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.552+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4032801535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b801015d0 msgr2=0x7f4b801039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.552+0000 7f4b7fe00700 1 --2- 172.21.15.99:0/4032801535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b801015d0 0x7f4b801039c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009990 tx=0x7f4b70009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.552+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4032801535 shutdown_connections 2024-03-30T00:35:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.552+0000 7f4b7fe00700 1 --2- 172.21.15.99:0/4032801535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80106830 0x7f4b80108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.552+0000 7f4b7fe00700 1 --2- 172.21.15.99:0/4032801535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b801015d0 0x7f4b801039c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.552+0000 7f4b7fe00700 1 --2- 172.21.15.99:0/4032801535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b80103f00 0x7f4b801062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.552+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4032801535 >> 172.21.15.99:0/4032801535 conn(0x7f4b800fafc0 msgr2=0x7f4b800fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.552+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4032801535 shutdown_connections 2024-03-30T00:35:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.552+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4032801535 wait complete. 2024-03-30T00:35:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.554+0000 7f4b7fe00700 1 Processor -- start 2024-03-30T00:35:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.554+0000 7f4b7fe00700 1 -- start start 2024-03-30T00:35:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.554+0000 7f4b7fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b801015d0 0x7f4b80100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.554+0000 7f4b7fe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80103f00 0x7f4b800ff580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.554+0000 7f4b7fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b80106830 0x7f4b800ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.554+0000 7f4b7fe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b80100260 con 0x7f4b80106830 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.554+0000 7f4b7fe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801003d0 con 0x7f4b801015d0 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.554+0000 7f4b7fe00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801a0a10 con 0x7f4b80103f00 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b80106830 0x7f4b800ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b80106830 0x7f4b800ffac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41130/0 (socket says 172.21.15.99:41130) 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7f400700 1 -- 172.21.15.99:0/4060120685 learned_addr learned my addr 172.21.15.99:0/4060120685 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b801015d0 0x7f4b80100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7e000700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80103f00 0x7f4b800ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7f400700 1 -- 172.21.15.99:0/4060120685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80103f00 msgr2=0x7f4b800ff580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7f400700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80103f00 0x7f4b800ff580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7f400700 1 -- 172.21.15.99:0/4060120685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b801015d0 msgr2=0x7f4b80100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7f400700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b801015d0 0x7f4b80100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7f400700 1 -- 172.21.15.99:0/4060120685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700095f0 con 0x7f4b80106830 2024-03-30T00:35:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7e000700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80103f00 0x7f4b800ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b801015d0 0x7f4b80100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.555+0000 7f4b7f400700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b80106830 0x7f4b800ffac0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ef30 tx=0x7f4b7800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.556+0000 7f4b77400700 1 -- 172.21.15.99:0/4060120685 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78018070 con 0x7f4b80106830 2024-03-30T00:35:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.556+0000 7f4b77400700 1 -- 172.21.15.99:0/4060120685 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b7800f040 con 0x7f4b80106830 2024-03-30T00:35:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.556+0000 7f4b77400700 1 -- 172.21.15.99:0/4060120685 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7800ccc0 con 0x7f4b80106830 2024-03-30T00:35:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.556+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4060120685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b801a0c40 con 0x7f4b80106830 2024-03-30T00:35:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.556+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4060120685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b801a1100 con 0x7f4b80106830 2024-03-30T00:35:40.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:40 smithi142 ceph-mon[41779]: pgmap v332: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:35:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.561+0000 7f4b77400700 1 -- 172.21.15.99:0/4060120685 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f4b78007500 con 0x7f4b80106830 2024-03-30T00:35:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.562+0000 7f4b995d5b80 1 Processor -- start 2024-03-30T00:35:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.562+0000 7f4b995d5b80 1 -- start start 2024-03-30T00:35:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.562+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff44e0 0x555693ff4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.562+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff4e40 0x555693ff56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.562+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 0x555693ff6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.562+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555693f4c930 con 0x555693ff4e40 2024-03-30T00:35:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.562+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555693f4a2f0 con 0x555693ff5c90 2024-03-30T00:35:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.562+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555693f45be0 con 0x555693ff44e0 2024-03-30T00:35:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b7e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 0x555693ff6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b7e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 0x555693ff6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:36290/0 (socket says 172.21.15.99:36290) 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b7e000700 1 -- 172.21.15.99:0/2639805054 learned_addr learned my addr 172.21.15.99:0/2639805054 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/2639805054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff4e40 0x555693ff56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b77400700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4b640a1120 0x7f4b640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b7e000700 1 --2- 172.21.15.99:0/2639805054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff44e0 0x555693ff4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b77400700 1 -- 172.21.15.99:0/4060120685 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f4b780cc5f0 con 0x7f4b80106830 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b7e000700 1 -- 172.21.15.99:0/2639805054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff44e0 msgr2=0x555693ff4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b7e000700 1 --2- 172.21.15.99:0/2639805054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff44e0 0x555693ff4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b7e000700 1 -- 172.21.15.99:0/2639805054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff4e40 msgr2=0x555693ff56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b7e000700 1 --2- 172.21.15.99:0/2639805054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff4e40 0x555693ff56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b7e000700 1 -- 172.21.15.99:0/2639805054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555693dc7350 con 0x555693ff5c90 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.563+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/2639805054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff4e40 0x555693ff56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.564+0000 7f4b7e000700 1 --2- 172.21.15.99:0/2639805054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 0x555693ff6110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c012900 tx=0x7f4b6c009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de7a9ee68ec59879 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.564+0000 7f4b7f400700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4b640a1120 0x7f4b640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.564+0000 7f4b6be00700 1 -- 172.21.15.99:0/2639805054 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c00f450 con 0x555693ff5c90 2024-03-30T00:35:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.564+0000 7f4b6be00700 1 -- 172.21.15.99:0/2639805054 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b6c00fa40 con 0x555693ff5c90 2024-03-30T00:35:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.564+0000 7f4b6be00700 1 -- 172.21.15.99:0/2639805054 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c01e710 con 0x555693ff5c90 2024-03-30T00:35:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.565+0000 7f4b995d5b80 1 -- 172.21.15.99:0/2639805054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 msgr2=0x555693ff6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.565+0000 7f4b7f400700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4b640a1120 0x7f4b640a35e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4b78004b80 tx=0x7f4b78004ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.565+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/2639805054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 0x555693ff6110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c012900 tx=0x7f4b6c009820 comp rx=0 tx=0).stop 2024-03-30T00:35:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.565+0000 7f4b995d5b80 1 -- 172.21.15.99:0/2639805054 shutdown_connections 2024-03-30T00:35:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.565+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/2639805054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 0x555693ff6110 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.565+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/2639805054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff44e0 0x555693ff4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.565+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/2639805054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff4e40 0x555693ff56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.565+0000 7f4b995d5b80 1 -- 172.21.15.99:0/2639805054 >> 172.21.15.99:0/2639805054 conn(0x555693f53640 msgr2=0x555693fef360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.565+0000 7f4b995d5b80 1 -- 172.21.15.99:0/2639805054 shutdown_connections 2024-03-30T00:35:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.565+0000 7f4b995d5b80 1 -- 172.21.15.99:0/2639805054 wait complete. 2024-03-30T00:35:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.566+0000 7f4b995d5b80 1 -- start start 2024-03-30T00:35:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.566+0000 7f4b995d5b80 1 Processor -- start 2024-03-30T00:35:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.567+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x555694006c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.567+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff4e40 0x555694007190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.567+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff5c90 0x55569400e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.567+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555693ee1d20 con 0x555693ff44e0 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.567+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555693ee19c0 con 0x555693ff4e40 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.567+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555693dc9640 con 0x555693ff5c90 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.567+0000 7f4b995d5b80 10 client.0 _ll_register_callbacks cb 0x555693eeb410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.567+0000 7f4b7e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x555694006c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.568+0000 7f4b7e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x555694006c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41144/0 (socket says 172.21.15.99:41144) 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.568+0000 7f4b7e000700 1 -- 172.21.15.99:0/3969701796 learned_addr learned my addr 172.21.15.99:0/3969701796 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.568+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff4e40 0x555694007190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.568+0000 7f4b7e000700 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff5c90 0x55569400e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.568+0000 7f4b7e000700 1 -- 172.21.15.99:0/3969701796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff5c90 msgr2=0x55569400e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.568+0000 7f4b7e000700 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff5c90 0x55569400e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.568+0000 7f4b7e000700 1 -- 172.21.15.99:0/3969701796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff4e40 msgr2=0x555694007190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.568+0000 7f4b7e000700 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff4e40 0x555694007190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.568+0000 7f4b7e000700 1 -- 172.21.15.99:0/3969701796 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b80109890 con 0x555693ff44e0 2024-03-30T00:35:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.568+0000 7f4b7e000700 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x555694006c50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00f700 tx=0x7f4b6c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56475a79742244b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.568+0000 7f4b68c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.569+0000 7f4b68c00700 1 -- 172.21.15.99:0/3969701796 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c00fbb0 con 0x555693ff44e0 2024-03-30T00:35:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.569+0000 7f4b995d5b80 1 -- 172.21.15.99:0/3969701796 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555693f06320 con 0x555693ff44e0 2024-03-30T00:35:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.569+0000 7f4b68c00700 1 -- 172.21.15.99:0/3969701796 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f4b6c000e40 con 0x555693ff44e0 2024-03-30T00:35:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.569+0000 7f4b68c00700 1 -- 172.21.15.99:0/3969701796 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6c005110 con 0x555693ff44e0 2024-03-30T00:35:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.569+0000 7f4b995d5b80 10 client.15666 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.569+0000 7f4b995d5b80 1 -- 172.21.15.99:0/3969701796 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f4b6c01a040 con 0x555693ff44e0 2024-03-30T00:35:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.569+0000 7f4b68c00700 1 -- 172.21.15.99:0/3969701796 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555693f06320 con 0x555693ff44e0 2024-03-30T00:35:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.570+0000 7f4b68c00700 1 -- 172.21.15.99:0/3969701796 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4b6c01e510 con 0x555693ff44e0 2024-03-30T00:35:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.570+0000 7f4b995d5b80 10 client.15666 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.570+0000 7f4b995d5b80 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.570+0000 7f4b995d5b80 20 client.15666 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.570+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5556940127e0 0x555694014c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.570+0000 7f4b995d5b80 4 client.15666 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.570+0000 7f4b995d5b80 1 -- 172.21.15.99:0/3969701796 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555693f3e6a0 con 0x5556940127e0 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.570+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5556940127e0 0x555694014c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.571+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5556940127e0 0x555694014c00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8461128ff12973 server_cookie=6980db1f6f622a12 in_seq=0 out_seq=0 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.571+0000 7f4b68c00700 10 client.15666 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.571+0000 7f4b68c00700 1 -- 172.21.15.99:0/3969701796 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555693f3e6a0 con 0x5556940127e0 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.571+0000 7f4b68c00700 10 client.15666 handle_command_reply: tid=0 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 1 client.15666 shutdown 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 1 -- 172.21.15.99:0/3969701796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 msgr2=0x555694006c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x555694006c50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00f700 tx=0x7f4b6c009e90 comp rx=0 tx=0).stop 2024-03-30T00:35:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 1 -- 172.21.15.99:0/3969701796 shutdown_connections 2024-03-30T00:35:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5556940127e0 0x555694014c00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff5c90 0x55569400e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x555694006c50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/3969701796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff4e40 0x555694007190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 1 -- 172.21.15.99:0/3969701796 >> 172.21.15.99:0/3969701796 conn(0x555693f53640 msgr2=0x555693ff3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b68c00700 0 client.15666 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 1 -- 172.21.15.99:0/3969701796 shutdown_connections 2024-03-30T00:35:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 1 -- 172.21.15.99:0/3969701796 wait complete. 2024-03-30T00:35:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.572+0000 7f4b995d5b80 20 client.15666 trim_cache size 0 max 16384 2024-03-30T00:35:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.591+0000 7f4b995d5b80 1 Processor -- start 2024-03-30T00:35:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.591+0000 7f4b995d5b80 1 -- start start 2024-03-30T00:35:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.591+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x55569400e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.591+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff4e40 0x55569400e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.591+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff5c90 0x555693ffc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.591+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555694007c40 con 0x555693ff44e0 2024-03-30T00:35:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.591+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555693ff0c10 con 0x555693ff4e40 2024-03-30T00:35:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.591+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555693ff0570 con 0x555693ff5c90 2024-03-30T00:35:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.591+0000 7f4b7e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x55569400e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.591+0000 7f4b7e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x55569400e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41158/0 (socket says 172.21.15.99:41158) 2024-03-30T00:35:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.592+0000 7f4b7e000700 1 -- 172.21.15.99:0/2685862603 learned_addr learned my addr 172.21.15.99:0/2685862603 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.592+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/2685862603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff4e40 0x55569400e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.592+0000 7f4b7e000700 1 --2- 172.21.15.99:0/2685862603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff5c90 0x555693ffc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.592+0000 7f4b7ea00700 1 -- 172.21.15.99:0/2685862603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff5c90 msgr2=0x555693ffc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.592+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/2685862603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff5c90 0x555693ffc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.592+0000 7f4b7ea00700 1 -- 172.21.15.99:0/2685862603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 msgr2=0x55569400e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.592+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/2685862603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x55569400e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.592+0000 7f4b7ea00700 1 -- 172.21.15.99:0/2685862603 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555693d73b00 con 0x555693ff4e40 2024-03-30T00:35:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.592+0000 7f4b7e000700 1 --2- 172.21.15.99:0/2685862603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff5c90 0x555693ffc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:35:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.592+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/2685862603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff4e40 0x55569400e140 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b70003f60 tx=0x7f4b70005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26eb0d4887be9bad server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.593+0000 7f4b74c00700 1 -- 172.21.15.99:0/2685862603 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70029150 con 0x555693ff4e40 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.593+0000 7f4b74c00700 1 -- 172.21.15.99:0/2685862603 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b7001e040 con 0x555693ff4e40 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.593+0000 7f4b74c00700 1 -- 172.21.15.99:0/2685862603 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b700038e0 con 0x555693ff4e40 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.594+0000 7f4b995d5b80 1 -- 172.21.15.99:0/2685862603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff4e40 msgr2=0x55569400e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.594+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/2685862603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff4e40 0x55569400e140 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b70003f60 tx=0x7f4b70005ab0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.594+0000 7f4b995d5b80 1 -- 172.21.15.99:0/2685862603 shutdown_connections 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.594+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/2685862603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff5c90 0x555693ffc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.594+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/2685862603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x55569400e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.594+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/2685862603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff4e40 0x55569400e140 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.594+0000 7f4b995d5b80 1 -- 172.21.15.99:0/2685862603 >> 172.21.15.99:0/2685862603 conn(0x555693f53640 msgr2=0x555694019560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.594+0000 7f4b995d5b80 1 -- 172.21.15.99:0/2685862603 shutdown_connections 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.594+0000 7f4b995d5b80 1 -- 172.21.15.99:0/2685862603 wait complete. 2024-03-30T00:35:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.594+0000 7f4b995d5b80 1 -- start start 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.595+0000 7f4b995d5b80 1 Processor -- start 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.595+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x55569400c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.595+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff4e40 0x55569400c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.595+0000 7f4b995d5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 0x55569400d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.595+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569400dc50 con 0x555693ff44e0 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.595+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569400ddc0 con 0x555693ff5c90 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.595+0000 7f4b995d5b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569400df80 con 0x555693ff4e40 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.595+0000 7f4b995d5b80 10 client.0 _ll_register_callbacks cb 0x555693eeb410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.595+0000 7f4b7e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x55569400c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.596+0000 7f4b7e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x55569400c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41162/0 (socket says 172.21.15.99:41162) 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.596+0000 7f4b7e000700 1 -- 172.21.15.99:0/4150937958 learned_addr learned my addr 172.21.15.99:0/4150937958 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.596+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff4e40 0x55569400c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.596+0000 7f4b7e000700 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 0x55569400d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.596+0000 7f4b7e000700 1 -- 172.21.15.99:0/4150937958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff4e40 msgr2=0x55569400c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.596+0000 7f4b7e000700 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff4e40 0x55569400c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.596+0000 7f4b7e000700 1 -- 172.21.15.99:0/4150937958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 msgr2=0x55569400d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.596+0000 7f4b7e000700 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 0x55569400d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.596+0000 7f4b7e000700 1 -- 172.21.15.99:0/4150937958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b70004040 con 0x555693ff44e0 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.596+0000 7f4b7e000700 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x55569400c2b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00f700 tx=0x7f4b6c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cad6b1ce7f3293a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.596+0000 7f4b68c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.597+0000 7f4b68c00700 1 -- 172.21.15.99:0/4150937958 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c034070 con 0x555693ff44e0 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.597+0000 7f4b68c00700 1 -- 172.21.15.99:0/4150937958 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f4b6c000e40 con 0x555693ff44e0 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.597+0000 7f4b995d5b80 1 -- 172.21.15.99:0/4150937958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b6c00fa40 con 0x555693ff44e0 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.597+0000 7f4b68c00700 1 -- 172.21.15.99:0/4150937958 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6c01ee20 con 0x555693ff44e0 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.597+0000 7f4b995d5b80 10 client.15672 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.597+0000 7f4b995d5b80 1 -- 172.21.15.99:0/4150937958 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555693f0be40 con 0x555693ff44e0 2024-03-30T00:35:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.597+0000 7f4b68c00700 1 -- 172.21.15.99:0/4150937958 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b6c03e020 con 0x555693ff44e0 2024-03-30T00:35:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.598+0000 7f4b68c00700 1 -- 172.21.15.99:0/4150937958 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4b6c037030 con 0x555693ff44e0 2024-03-30T00:35:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.598+0000 7f4b995d5b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.598+0000 7f4b995d5b80 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.598+0000 7f4b995d5b80 20 client.15672 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.598+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5556940672b0 0x5556940696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.598+0000 7f4b995d5b80 4 client.15672 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:35:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.598+0000 7f4b995d5b80 1 -- 172.21.15.99:0/4150937958 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555693d94c90 con 0x5556940672b0 2024-03-30T00:35:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.598+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5556940672b0 0x5556940696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.599+0000 7f4b7ea00700 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5556940672b0 0x5556940696d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f33178d69d267d18 server_cookie=b67aa0d7ffc3cece in_seq=0 out_seq=0 2024-03-30T00:35:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.599+0000 7f4b68c00700 10 client.15672 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.603+0000 7f4b68c00700 1 -- 172.21.15.99:0/4150937958 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555693d94c90 con 0x5556940672b0 2024-03-30T00:35:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.603+0000 7f4b68c00700 10 client.15672 handle_command_reply: tid=0 2024-03-30T00:35:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.603+0000 7f4b995d5b80 1 client.15672 shutdown 2024-03-30T00:35:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.603+0000 7f4b995d5b80 1 -- 172.21.15.99:0/4150937958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 msgr2=0x55569400c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.603+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x55569400c2b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00f700 tx=0x7f4b6c000bc0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.604+0000 7f4b995d5b80 1 -- 172.21.15.99:0/4150937958 shutdown_connections 2024-03-30T00:35:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.604+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5556940672b0 0x5556940696d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.604+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555693ff5c90 0x55569400d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.604+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555693ff44e0 0x55569400c2b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.604+0000 7f4b995d5b80 1 --2- 172.21.15.99:0/4150937958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555693ff4e40 0x55569400c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.604+0000 7f4b995d5b80 1 -- 172.21.15.99:0/4150937958 >> 172.21.15.99:0/4150937958 conn(0x555693f53640 msgr2=0x555693ffc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.604+0000 7f4b68c00700 0 client.15672 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.604+0000 7f4b995d5b80 1 -- 172.21.15.99:0/4150937958 shutdown_connections 2024-03-30T00:35:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.604+0000 7f4b995d5b80 1 -- 172.21.15.99:0/4150937958 wait complete. 2024-03-30T00:35:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.604+0000 7f4b995d5b80 20 client.15672 trim_cache size 0 max 16384 2024-03-30T00:35:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:35:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:35:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:35:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:35:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4060120685 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4b640a1120 msgr2=0x7f4b640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4b640a1120 0x7f4b640a35e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4b78004b80 tx=0x7f4b78004ad0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4060120685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b80106830 msgr2=0x7f4b800ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b80106830 0x7f4b800ffac0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ef30 tx=0x7f4b7800c450 comp rx=0 tx=0).stop 2024-03-30T00:35:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4060120685 shutdown_connections 2024-03-30T00:35:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b80106830 0x7f4b800ffac0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f4b640a1120 0x7f4b640a35e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b801015d0 0x7f4b80100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 --2- 172.21.15.99:0/4060120685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4b80103f00 0x7f4b800ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4060120685 >> 172.21.15.99:0/4060120685 conn(0x7f4b800fafc0 msgr2=0x7f4b801034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4060120685 shutdown_connections 2024-03-30T00:35:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:40.605+0000 7f4b7fe00700 1 -- 172.21.15.99:0/4060120685 wait complete. 2024-03-30T00:35:42.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:42 smithi161 ceph-mon[42221]: pgmap v333: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:42.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:42 smithi099 ceph-mon[38079]: pgmap v333: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:42 smithi142 ceph-mon[41779]: pgmap v333: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:44.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:44 smithi161 ceph-mon[42221]: pgmap v334: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-30T00:35:44.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:44 smithi099 ceph-mon[38079]: pgmap v334: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-30T00:35:44.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:44 smithi142 ceph-mon[41779]: pgmap v334: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-30T00:35:46.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:46 smithi161 ceph-mon[42221]: pgmap v335: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-30T00:35:46.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:46 smithi099 ceph-mon[38079]: pgmap v335: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-30T00:35:46.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:46 smithi142 ceph-mon[41779]: pgmap v335: 97 pgs: 97 active+clean; 662 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-30T00:35:48.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:48 smithi161 ceph-mon[42221]: pgmap v336: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:35:48.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:48 smithi099 ceph-mon[38079]: pgmap v336: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:35:48.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:48 smithi142 ceph-mon[41779]: pgmap v336: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:35:50.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:50 smithi161 ceph-mon[42221]: pgmap v337: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:35:50.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:50 smithi099 ceph-mon[38079]: pgmap v337: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:35:50.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:50 smithi142 ceph-mon[41779]: pgmap v337: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:35:51.390 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T00:35:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.732+0000 7fa0b1600700 1 -- 172.21.15.99:0/2033675808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac104b20 msgr2=0x7fa0ac10b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.732+0000 7fa0b1600700 1 --2- 172.21.15.99:0/2033675808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac104b20 0x7fa0ac10b3d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400b0d0 tx=0x7fa0a400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.732+0000 7fa0b1600700 1 -- 172.21.15.99:0/2033675808 shutdown_connections 2024-03-30T00:35:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.732+0000 7fa0b1600700 1 --2- 172.21.15.99:0/2033675808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac104b20 0x7fa0ac10b3d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.732+0000 7fa0b1600700 1 --2- 172.21.15.99:0/2033675808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ac102f90 0x7fa0ac1033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.732+0000 7fa0b1600700 1 --2- 172.21.15.99:0/2033675808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0ac104180 0x7fa0ac1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.732+0000 7fa0b1600700 1 -- 172.21.15.99:0/2033675808 >> 172.21.15.99:0/2033675808 conn(0x7fa0ac0fe500 msgr2=0x7fa0ac100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.732+0000 7fa0b1600700 1 -- 172.21.15.99:0/2033675808 shutdown_connections 2024-03-30T00:35:51.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.732+0000 7fa0b1600700 1 -- 172.21.15.99:0/2033675808 wait complete. 2024-03-30T00:35:51.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.733+0000 7fa0b1600700 1 Processor -- start 2024-03-30T00:35:51.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.734+0000 7fa0b1600700 1 -- start start 2024-03-30T00:35:51.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.734+0000 7fa0b1600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0ac102f90 0x7fa0ac198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.734+0000 7fa0b1600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ac104180 0x7fa0ac199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.734+0000 7fa0b1600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac104b20 0x7fa0ac1a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.734+0000 7fa0b1600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac1a0a50 con 0x7fa0ac104b20 2024-03-30T00:35:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.734+0000 7fa0b1600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac1a0b90 con 0x7fa0ac102f90 2024-03-30T00:35:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.734+0000 7fa0b1600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac1a0d20 con 0x7fa0ac104180 2024-03-30T00:35:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.735+0000 7fa0aaa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0ac102f90 0x7fa0ac198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.735+0000 7fa0ab400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac104b20 0x7fa0ac1a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.735+0000 7fa0aaa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0ac102f90 0x7fa0ac198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45208/0 (socket says 172.21.15.99:45208) 2024-03-30T00:35:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.735+0000 7fa0aaa00700 1 -- 172.21.15.99:0/3983254210 learned_addr learned my addr 172.21.15.99:0/3983254210 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.735+0000 7fa0aa000700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ac104180 0x7fa0ac199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.735+0000 7fa0aaa00700 1 -- 172.21.15.99:0/3983254210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ac104180 msgr2=0x7fa0ac199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.735+0000 7fa0aaa00700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ac104180 0x7fa0ac199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.735+0000 7fa0aaa00700 1 -- 172.21.15.99:0/3983254210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac104b20 msgr2=0x7fa0ac1a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.735+0000 7fa0aaa00700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac104b20 0x7fa0ac1a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.735+0000 7fa0aaa00700 1 -- 172.21.15.99:0/3983254210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a4009d00 con 0x7fa0ac102f90 2024-03-30T00:35:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.735+0000 7fa0ab400700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac104b20 0x7fa0ac1a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.736+0000 7fa0aaa00700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0ac102f90 0x7fa0ac198f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00d980 tx=0x7fa09c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.736+0000 7fa0a3400700 1 -- 172.21.15.99:0/3983254210 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00f890 con 0x7fa0ac102f90 2024-03-30T00:35:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.736+0000 7fa0a3400700 1 -- 172.21.15.99:0/3983254210 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa09c00fe80 con 0x7fa0ac102f90 2024-03-30T00:35:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.736+0000 7fa0a3400700 1 -- 172.21.15.99:0/3983254210 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c019360 con 0x7fa0ac102f90 2024-03-30T00:35:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.736+0000 7fa0b1600700 1 -- 172.21.15.99:0/3983254210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac1a0fa0 con 0x7fa0ac102f90 2024-03-30T00:35:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.736+0000 7fa0b1600700 1 -- 172.21.15.99:0/3983254210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac1a13c0 con 0x7fa0ac102f90 2024-03-30T00:35:51.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.740+0000 7fa0c514cb80 1 Processor -- start 2024-03-30T00:35:51.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.741+0000 7fa0c514cb80 1 -- start start 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.741+0000 7fa0a3400700 1 -- 172.21.15.99:0/3983254210 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa09c00f9f0 con 0x7fa0ac102f90 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.741+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 0x55cc08d793a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.741+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d798e0 0x55cc08d7a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.741+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 0x55cc08d7ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.741+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08cc8990 con 0x55cc08d78f80 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.741+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08cc6350 con 0x55cc08d798e0 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.741+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08cc1c40 con 0x55cc08d7a7d0 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.742+0000 7fa0aa000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 0x55cc08d7ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.742+0000 7fa0aa000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 0x55cc08d7ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55540/0 (socket says 172.21.15.99:55540) 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.742+0000 7fa0aa000700 1 -- 172.21.15.99:0/689926304 learned_addr learned my addr 172.21.15.99:0/689926304 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.742+0000 7fa0ab400700 1 --2- 172.21.15.99:0/689926304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 0x55cc08d793a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.742+0000 7fa0aa000700 1 -- 172.21.15.99:0/689926304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d798e0 msgr2=0x55cc08d7a1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:35:51.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.742+0000 7fa0aa000700 1 --2- 172.21.15.99:0/689926304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d798e0 0x55cc08d7a1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.742+0000 7fa0aa000700 1 -- 172.21.15.99:0/689926304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 msgr2=0x55cc08d793a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:51.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.742+0000 7fa0aa000700 1 --2- 172.21.15.99:0/689926304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 0x55cc08d793a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.742+0000 7fa0aa000700 1 -- 172.21.15.99:0/689926304 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc08b433b0 con 0x55cc08d7a7d0 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.743+0000 7fa0aa000700 1 --2- 172.21.15.99:0/689926304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 0x55cc08d7ac50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa098009d00 tx=0x7fa098011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=466eabfd7e01be9f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.743+0000 7fa097400700 1 -- 172.21.15.99:0/689926304 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098004230 con 0x55cc08d7a7d0 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.743+0000 7fa097400700 1 -- 172.21.15.99:0/689926304 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa098004390 con 0x55cc08d7a7d0 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.743+0000 7fa097400700 1 -- 172.21.15.99:0/689926304 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0980196a0 con 0x55cc08d7a7d0 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.743+0000 7fa0a3400700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0900a11a0 0x7fa0900a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.743+0000 7fa0a3400700 1 -- 172.21.15.99:0/3983254210 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fa09c0cc3d0 con 0x7fa0ac102f90 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.744+0000 7fa0c514cb80 1 -- 172.21.15.99:0/689926304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 msgr2=0x55cc08d7ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.744+0000 7fa0ab400700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0900a11a0 0x7fa0900a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.744+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/689926304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 0x55cc08d7ac50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa098009d00 tx=0x7fa098011720 comp rx=0 tx=0).stop 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.744+0000 7fa0c514cb80 1 -- 172.21.15.99:0/689926304 shutdown_connections 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.744+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/689926304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 0x55cc08d7ac50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.744+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/689926304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 0x55cc08d793a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.744+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/689926304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d798e0 0x55cc08d7a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.744+0000 7fa0c514cb80 1 -- 172.21.15.99:0/689926304 >> 172.21.15.99:0/689926304 conn(0x55cc08ccf6a0 msgr2=0x55cc08d6b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:51.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.744+0000 7fa0c514cb80 1 -- 172.21.15.99:0/689926304 shutdown_connections 2024-03-30T00:35:51.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.744+0000 7fa0c514cb80 1 -- 172.21.15.99:0/689926304 wait complete. 2024-03-30T00:35:51.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.745+0000 7fa0c514cb80 1 -- start start 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.745+0000 7fa0ab400700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0900a11a0 0x7fa0900a3660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4007f80 tx=0x7fa0a400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.745+0000 7fa0c514cb80 1 Processor -- start 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 0x55cc08d93ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d798e0 0x55cc08d94400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 0x55cc08d9b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08c5dd80 con 0x55cc08d78f80 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08c5da20 con 0x55cc08d798e0 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08b456a0 con 0x55cc08d7a7d0 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0c514cb80 10 client.0 _ll_register_callbacks cb 0x55cc08c67470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0aaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 0x55cc08d93ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0aa000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d798e0 0x55cc08d94400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0aaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 0x55cc08d93ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56310/0 (socket says 172.21.15.99:56310) 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0aaa00700 1 -- 172.21.15.99:0/863029134 learned_addr learned my addr 172.21.15.99:0/863029134 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:51.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.746+0000 7fa0ab400700 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 0x55cc08d9b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.747+0000 7fa0aaa00700 1 -- 172.21.15.99:0/863029134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 msgr2=0x55cc08d9b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:51.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.747+0000 7fa0aaa00700 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 0x55cc08d9b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.747+0000 7fa0aaa00700 1 -- 172.21.15.99:0/863029134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d798e0 msgr2=0x55cc08d94400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:51.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.747+0000 7fa0aaa00700 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d798e0 0x55cc08d94400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.747+0000 7fa0aaa00700 1 -- 172.21.15.99:0/863029134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa098009960 con 0x55cc08d78f80 2024-03-30T00:35:51.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.747+0000 7fa0aaa00700 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 0x55cc08d93ec0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa09c095c40 tx=0x7fa09c099300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9dae97dfeeff37 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:51.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.747+0000 7fa08be00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:51.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.747+0000 7fa08be00700 1 -- 172.21.15.99:0/863029134 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c09d370 con 0x55cc08d78f80 2024-03-30T00:35:51.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.747+0000 7fa08be00700 1 -- 172.21.15.99:0/863029134 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fa09c00fca0 con 0x55cc08d78f80 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.747+0000 7fa0c514cb80 1 -- 172.21.15.99:0/863029134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc08c82380 con 0x55cc08d78f80 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.748+0000 7fa08be00700 1 -- 172.21.15.99:0/863029134 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa09c099e40 con 0x55cc08d78f80 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.748+0000 7fa0c514cb80 10 client.15684 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.748+0000 7fa0c514cb80 1 -- 172.21.15.99:0/863029134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cc08aefb60 con 0x55cc08d78f80 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.748+0000 7fa08be00700 1 -- 172.21.15.99:0/863029134 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cc08c82380 con 0x55cc08d78f80 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.748+0000 7fa08be00700 1 -- 172.21.15.99:0/863029134 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa09c0cb070 con 0x55cc08d78f80 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.748+0000 7fa0c514cb80 10 client.15684 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.748+0000 7fa0c514cb80 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.748+0000 7fa0c514cb80 20 client.15684 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.748+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cc08d9fcc0 0x55cc08da20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.748+0000 7fa0c514cb80 4 client.15684 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:51.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.748+0000 7fa0c514cb80 1 -- 172.21.15.99:0/863029134 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc08cba700 con 0x55cc08d9fcc0 2024-03-30T00:35:51.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.749+0000 7fa0aa000700 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cc08d9fcc0 0x55cc08da20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.749+0000 7fa08be00700 10 client.15684 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.749+0000 7fa0aa000700 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cc08d9fcc0 0x55cc08da20e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e56edcc20d177ec server_cookie=992bfcd9491ccc08 in_seq=0 out_seq=0 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.750+0000 7fa08be00700 1 -- 172.21.15.99:0/863029134 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cc08cba700 con 0x55cc08d9fcc0 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.750+0000 7fa08be00700 10 client.15684 handle_command_reply: tid=0 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.750+0000 7fa0c514cb80 1 client.15684 shutdown 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa0c514cb80 1 -- 172.21.15.99:0/863029134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 msgr2=0x55cc08d93ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 0x55cc08d93ec0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa09c095c40 tx=0x7fa09c099300 comp rx=0 tx=0).stop 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa0c514cb80 1 -- 172.21.15.99:0/863029134 shutdown_connections 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cc08d9fcc0 0x55cc08da20e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d7a7d0 0x55cc08d9b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d78f80 0x55cc08d93ec0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa08be00700 0 client.15684 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/863029134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d798e0 0x55cc08d94400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa0c514cb80 1 -- 172.21.15.99:0/863029134 >> 172.21.15.99:0/863029134 conn(0x55cc08ccf6a0 msgr2=0x55cc08d77980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:51.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa0c514cb80 1 -- 172.21.15.99:0/863029134 shutdown_connections 2024-03-30T00:35:51.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa0c514cb80 1 -- 172.21.15.99:0/863029134 wait complete. 2024-03-30T00:35:51.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.751+0000 7fa0c514cb80 20 client.15684 trim_cache size 0 max 16384 2024-03-30T00:35:51.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.769+0000 7fa0c514cb80 1 Processor -- start 2024-03-30T00:35:51.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.769+0000 7fa0c514cb80 1 -- start start 2024-03-30T00:35:51.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.769+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 0x55cc08d7abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.769+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d7a7d0 0x55cc08d9b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.769+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d94500 0x55cc08d64f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.769+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08d929c0 con 0x55cc08d7a7d0 2024-03-30T00:35:51.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.769+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08d7a650 con 0x55cc08d94500 2024-03-30T00:35:51.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.769+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08d9c5c0 con 0x55cc08d798e0 2024-03-30T00:35:51.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.769+0000 7fa0aaa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 0x55cc08d7abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.770+0000 7fa0aaa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 0x55cc08d7abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55552/0 (socket says 172.21.15.99:55552) 2024-03-30T00:35:51.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.770+0000 7fa0aaa00700 1 -- 172.21.15.99:0/652227901 learned_addr learned my addr 172.21.15.99:0/652227901 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:51.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.770+0000 7fa0ab400700 1 --2- 172.21.15.99:0/652227901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d94500 0x55cc08d64f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.770+0000 7fa0aaa00700 1 -- 172.21.15.99:0/652227901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d94500 msgr2=0x55cc08d64f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:51.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.770+0000 7fa0aaa00700 1 --2- 172.21.15.99:0/652227901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d94500 0x55cc08d64f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.770+0000 7fa0aaa00700 1 -- 172.21.15.99:0/652227901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d7a7d0 msgr2=0x55cc08d9b400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:35:51.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.770+0000 7fa0aaa00700 1 --2- 172.21.15.99:0/652227901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d7a7d0 0x55cc08d9b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.770+0000 7fa0aaa00700 1 -- 172.21.15.99:0/652227901 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc08c87ea0 con 0x55cc08d798e0 2024-03-30T00:35:51.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.770+0000 7fa0aaa00700 1 --2- 172.21.15.99:0/652227901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 0x55cc08d7abb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa09c097a90 tx=0x7fa09c0b2220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40055a693ed1259e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:51.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.771+0000 7fa097e00700 1 -- 172.21.15.99:0/652227901 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c09d370 con 0x55cc08d798e0 2024-03-30T00:35:51.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.771+0000 7fa097e00700 1 -- 172.21.15.99:0/652227901 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa09c0990a0 con 0x55cc08d798e0 2024-03-30T00:35:51.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.771+0000 7fa097e00700 1 -- 172.21.15.99:0/652227901 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c099b80 con 0x55cc08d798e0 2024-03-30T00:35:51.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.771+0000 7fa0c514cb80 1 -- 172.21.15.99:0/652227901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 msgr2=0x55cc08d7abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:51.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.771+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/652227901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 0x55cc08d7abb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa09c097a90 tx=0x7fa09c0b2220 comp rx=0 tx=0).stop 2024-03-30T00:35:51.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.771+0000 7fa0c514cb80 1 -- 172.21.15.99:0/652227901 shutdown_connections 2024-03-30T00:35:51.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.771+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/652227901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d94500 0x55cc08d64f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.771+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/652227901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 0x55cc08d7abb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.771+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/652227901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d7a7d0 0x55cc08d9b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.771+0000 7fa0c514cb80 1 -- 172.21.15.99:0/652227901 >> 172.21.15.99:0/652227901 conn(0x55cc08ccf6a0 msgr2=0x55cc08d645a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:51.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.772+0000 7fa0c514cb80 1 -- 172.21.15.99:0/652227901 shutdown_connections 2024-03-30T00:35:51.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.772+0000 7fa0c514cb80 1 -- 172.21.15.99:0/652227901 wait complete. 2024-03-30T00:35:51.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.772+0000 7fa0c514cb80 1 -- start start 2024-03-30T00:35:51.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.772+0000 7fa0c514cb80 1 Processor -- start 2024-03-30T00:35:51.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.773+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 0x55cc08d9b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.773+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d7a7d0 0x55cc08d997f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.773+0000 7fa0c514cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d94500 0x55cc08d99d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.773+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08d9a300 con 0x55cc08d7a7d0 2024-03-30T00:35:51.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.773+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08d9a470 con 0x55cc08d94500 2024-03-30T00:35:51.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.773+0000 7fa0c514cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08d9a630 con 0x55cc08d798e0 2024-03-30T00:35:51.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.773+0000 7fa0c514cb80 10 client.0 _ll_register_callbacks cb 0x55cc08c67470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:51.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.774+0000 7fa0aaa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 0x55cc08d9b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.774+0000 7fa0aa000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d7a7d0 0x55cc08d997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.774+0000 7fa0aa000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d7a7d0 0x55cc08d997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56314/0 (socket says 172.21.15.99:56314) 2024-03-30T00:35:51.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.774+0000 7fa0aa000700 1 -- 172.21.15.99:0/3797791752 learned_addr learned my addr 172.21.15.99:0/3797791752 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:51.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.774+0000 7fa0aaa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 0x55cc08d9b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55556/0 (socket says 172.21.15.99:55556) 2024-03-30T00:35:51.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.774+0000 7fa0ab400700 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d94500 0x55cc08d99d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.774+0000 7fa0aa000700 1 -- 172.21.15.99:0/3797791752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 msgr2=0x55cc08d9b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:51.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.774+0000 7fa0aa000700 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 0x55cc08d9b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.774+0000 7fa0aa000700 1 -- 172.21.15.99:0/3797791752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d94500 msgr2=0x55cc08d99d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:51.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.774+0000 7fa0aa000700 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d94500 0x55cc08d99d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.774+0000 7fa0aa000700 1 -- 172.21.15.99:0/3797791752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc08aefb60 con 0x55cc08d7a7d0 2024-03-30T00:35:51.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.775+0000 7fa0aa000700 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d7a7d0 0x55cc08d997f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa098009d00 tx=0x7fa098005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66666aeb82531bdb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:51.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.775+0000 7fa08be00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.775+0000 7fa08be00700 1 -- 172.21.15.99:0/3797791752 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098029070 con 0x55cc08d7a7d0 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.775+0000 7fa08be00700 1 -- 172.21.15.99:0/3797791752 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fa09802a250 con 0x55cc08d7a7d0 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.775+0000 7fa0c514cb80 1 -- 172.21.15.99:0/3797791752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc08d78500 con 0x55cc08d7a7d0 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.775+0000 7fa08be00700 1 -- 172.21.15.99:0/3797791752 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa098009260 con 0x55cc08d7a7d0 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.775+0000 7fa0c514cb80 10 client.15687 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.775+0000 7fa0c514cb80 1 -- 172.21.15.99:0/3797791752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cc08b47cd0 con 0x55cc08d7a7d0 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.776+0000 7fa08be00700 1 -- 172.21.15.99:0/3797791752 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cc08d78500 con 0x55cc08d7a7d0 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.776+0000 7fa08be00700 1 -- 172.21.15.99:0/3797791752 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa098011440 con 0x55cc08d7a7d0 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.776+0000 7fa0c514cb80 10 client.15687 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.776+0000 7fa0c514cb80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.776+0000 7fa0c514cb80 20 client.15687 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.776+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cc08df5330 0x55cc08df7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:51.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.776+0000 7fa0c514cb80 4 client.15687 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.776+0000 7fa0c514cb80 1 -- 172.21.15.99:0/3797791752 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fa098004390 con 0x55cc08df5330 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.777+0000 7fa0aaa00700 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cc08df5330 0x55cc08df7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.777+0000 7fa08be00700 10 client.15687 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.777+0000 7fa0aaa00700 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cc08df5330 0x55cc08df7750 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b45b053cb9ec35 server_cookie=f9e59cae3b906869 in_seq=0 out_seq=0 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.777+0000 7fa08be00700 1 -- 172.21.15.99:0/3797791752 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fa098004390 con 0x55cc08df5330 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.777+0000 7fa08be00700 10 client.15687 handle_command_reply: tid=0 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.777+0000 7fa0c514cb80 1 client.15687 shutdown 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa0c514cb80 1 -- 172.21.15.99:0/3797791752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d7a7d0 msgr2=0x55cc08d997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d7a7d0 0x55cc08d997f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa098009d00 tx=0x7fa098005b00 comp rx=0 tx=0).stop 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa0c514cb80 1 -- 172.21.15.99:0/3797791752 shutdown_connections 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55cc08df5330 0x55cc08df7750 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc08d94500 0x55cc08d99d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cc08d798e0 0x55cc08d9b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa0c514cb80 1 --2- 172.21.15.99:0/3797791752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc08d7a7d0 0x55cc08d997f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa0c514cb80 1 -- 172.21.15.99:0/3797791752 >> 172.21.15.99:0/3797791752 conn(0x55cc08ccf6a0 msgr2=0x55cc08d93c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:51.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa08be00700 0 client.15687 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:51.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa0c514cb80 1 -- 172.21.15.99:0/3797791752 shutdown_connections 2024-03-30T00:35:51.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa0c514cb80 1 -- 172.21.15.99:0/3797791752 wait complete. 2024-03-30T00:35:51.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.778+0000 7fa0c514cb80 20 client.15687 trim_cache size 0 max 16384 2024-03-30T00:35:51.787 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:35:51.787 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "no active scrubs running", 2024-03-30T00:35:51.787 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": {} 2024-03-30T00:35:51.787 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:35:51.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.779+0000 7fa0b1600700 1 -- 172.21.15.99:0/3983254210 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0900a11a0 msgr2=0x7fa0900a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:51.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.779+0000 7fa0b1600700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0900a11a0 0x7fa0900a3660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4007f80 tx=0x7fa0a400b040 comp rx=0 tx=0).stop 2024-03-30T00:35:51.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.779+0000 7fa0b1600700 1 -- 172.21.15.99:0/3983254210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0ac102f90 msgr2=0x7fa0ac198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:51.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.779+0000 7fa0b1600700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0ac102f90 0x7fa0ac198f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00d980 tx=0x7fa09c00dc90 comp rx=0 tx=0).stop 2024-03-30T00:35:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.779+0000 7fa0b1600700 1 -- 172.21.15.99:0/3983254210 shutdown_connections 2024-03-30T00:35:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.779+0000 7fa0b1600700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0ac104b20 0x7fa0ac1a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.779+0000 7fa0b1600700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa0900a11a0 0x7fa0900a3660 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.779+0000 7fa0b1600700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0ac102f90 0x7fa0ac198f20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.779+0000 7fa0b1600700 1 --2- 172.21.15.99:0/3983254210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0ac104180 0x7fa0ac199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.779+0000 7fa0b1600700 1 -- 172.21.15.99:0/3983254210 >> 172.21.15.99:0/3983254210 conn(0x7fa0ac0fe500 msgr2=0x7fa0ac107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.780+0000 7fa0b1600700 1 -- 172.21.15.99:0/3983254210 shutdown_connections 2024-03-30T00:35:51.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:51.780+0000 7fa0b1600700 1 -- 172.21.15.99:0/3983254210 wait complete. 2024-03-30T00:35:51.806 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:35:51.806 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:35:52.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.009+0000 7fc1c1400700 1 -- 172.21.15.99:0/1022193337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc101e70 msgr2=0x7fc1bc1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.009+0000 7fc1c1400700 1 --2- 172.21.15.99:0/1022193337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc101e70 0x7fc1bc1022d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80099c0 tx=0x7fc1a8009cd0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.009+0000 7fc1c1400700 1 -- 172.21.15.99:0/1022193337 shutdown_connections 2024-03-30T00:35:52.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.009+0000 7fc1c1400700 1 --2- 172.21.15.99:0/1022193337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc102810 0x7fc1bc1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.009+0000 7fc1c1400700 1 --2- 172.21.15.99:0/1022193337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc100c80 0x7fc1bc1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.009+0000 7fc1c1400700 1 --2- 172.21.15.99:0/1022193337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc101e70 0x7fc1bc1022d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.009+0000 7fc1c1400700 1 -- 172.21.15.99:0/1022193337 >> 172.21.15.99:0/1022193337 conn(0x7fc1bc0fc1f0 msgr2=0x7fc1bc0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.009+0000 7fc1c1400700 1 -- 172.21.15.99:0/1022193337 shutdown_connections 2024-03-30T00:35:52.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.009+0000 7fc1c1400700 1 -- 172.21.15.99:0/1022193337 wait complete. 2024-03-30T00:35:52.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.010+0000 7fc1c1400700 1 Processor -- start 2024-03-30T00:35:52.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.011+0000 7fc1c1400700 1 -- start start 2024-03-30T00:35:52.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.011+0000 7fc1c1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc100c80 0x7fc1bc198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.011+0000 7fc1c1400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc101e70 0x7fc1bc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.011+0000 7fc1c1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc102810 0x7fc1bc1a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.011+0000 7fc1c1400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc1a08d0 con 0x7fc1bc102810 2024-03-30T00:35:52.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.011+0000 7fc1c1400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc06a7d0 con 0x7fc1bc100c80 2024-03-30T00:35:52.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.011+0000 7fc1c1400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc06a960 con 0x7fc1bc101e70 2024-03-30T00:35:52.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.012+0000 7fc1bb200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc102810 0x7fc1bc1a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.012+0000 7fc1bb200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc102810 0x7fc1bc1a0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56324/0 (socket says 172.21.15.99:56324) 2024-03-30T00:35:52.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.012+0000 7fc1bb200700 1 -- 172.21.15.99:0/415426327 learned_addr learned my addr 172.21.15.99:0/415426327 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:52.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.012+0000 7fc1ba800700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc100c80 0x7fc1bc198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.012+0000 7fc1b9e00700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc101e70 0x7fc1bc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.012+0000 7fc1bb200700 1 -- 172.21.15.99:0/415426327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc101e70 msgr2=0x7fc1bc199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.012+0000 7fc1bb200700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc101e70 0x7fc1bc199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.012+0000 7fc1bb200700 1 -- 172.21.15.99:0/415426327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc100c80 msgr2=0x7fc1bc198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.012+0000 7fc1bb200700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc100c80 0x7fc1bc198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.012+0000 7fc1bb200700 1 -- 172.21.15.99:0/415426327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a80095f0 con 0x7fc1bc102810 2024-03-30T00:35:52.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.012+0000 7fc1bb200700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc102810 0x7fc1bc1a0300 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400ef30 tx=0x7fc1b400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.013+0000 7fc1b3400700 1 -- 172.21.15.99:0/415426327 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4018070 con 0x7fc1bc102810 2024-03-30T00:35:52.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.013+0000 7fc1c1400700 1 -- 172.21.15.99:0/415426327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc06abe0 con 0x7fc1bc102810 2024-03-30T00:35:52.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.013+0000 7fc1c1400700 1 -- 172.21.15.99:0/415426327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc06b0f0 con 0x7fc1bc102810 2024-03-30T00:35:52.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.013+0000 7fc1b3400700 1 -- 172.21.15.99:0/415426327 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1b400f040 con 0x7fc1bc102810 2024-03-30T00:35:52.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.013+0000 7fc1b3400700 1 -- 172.21.15.99:0/415426327 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4003d00 con 0x7fc1bc102810 2024-03-30T00:35:52.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.014+0000 7fc1c1400700 1 -- 172.21.15.99:0/415426327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c0052f0 con 0x7fc1bc102810 2024-03-30T00:35:52.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.018+0000 7fc1b3400700 1 -- 172.21.15.99:0/415426327 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc1b4004ad0 con 0x7fc1bc102810 2024-03-30T00:35:52.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.020+0000 7fc1b3400700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1a00a11a0 0x7fc1a00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.020+0000 7fc1b3400700 1 -- 172.21.15.99:0/415426327 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fc1b40ccd40 con 0x7fc1bc102810 2024-03-30T00:35:52.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.020+0000 7fc1ba800700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1a00a11a0 0x7fc1a00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.021+0000 7fc1ba800700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1a00a11a0 0x7fc1a00a3660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc101cd0 tx=0x7fc1ac00b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.022+0000 7fc1b3400700 1 -- 172.21.15.99:0/415426327 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc1b40964e0 con 0x7fc1bc102810 2024-03-30T00:35:52.174 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:52 smithi099 ceph-mon[38079]: pgmap v338: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:35:52.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.381+0000 7fc1c1400700 1 -- 172.21.15.99:0/415426327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc19c005160 con 0x7fc1bc102810 2024-03-30T00:35:52.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.382+0000 7fc1b3400700 1 -- 172.21.15.99:0/415426327 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10860 (secure 0 0 0) 0x7fc1b4099b00 con 0x7fc1bc102810 2024-03-30T00:35:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:35:52.389 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:34:39.366251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 -- 172.21.15.99:0/415426327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1a00a11a0 msgr2=0x7fc1a00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1a00a11a0 0x7fc1a00a3660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc101cd0 tx=0x7fc1ac00b5b0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 -- 172.21.15.99:0/415426327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc102810 msgr2=0x7fc1bc1a0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc102810 0x7fc1bc1a0300 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400ef30 tx=0x7fc1b400c450 comp rx=0 tx=0).stop 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 -- 172.21.15.99:0/415426327 shutdown_connections 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc102810 0x7fc1bc1a0300 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1a00a11a0 0x7fc1a00a3660 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1bc100c80 0x7fc1bc198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 --2- 172.21.15.99:0/415426327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1bc101e70 0x7fc1bc199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 -- 172.21.15.99:0/415426327 >> 172.21.15.99:0/415426327 conn(0x7fc1bc0fc1f0 msgr2=0x7fc1bc072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 -- 172.21.15.99:0/415426327 shutdown_connections 2024-03-30T00:35:52.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.385+0000 7fc1c1400700 1 -- 172.21.15.99:0/415426327 wait complete. 2024-03-30T00:35:52.393 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2024-03-30T00:35:52.419 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T00:35:52.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:52 smithi161 ceph-mon[42221]: pgmap v338: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:35:52.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:52 smithi142 ceph-mon[41779]: pgmap v338: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:35:52.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.641+0000 7ff784800700 1 -- 172.21.15.99:0/2208589874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c101840 msgr2=0x7ff77c103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.641+0000 7ff784800700 1 --2- 172.21.15.99:0/2208589874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c101840 0x7ff77c103c30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff770009990 tx=0x7ff770009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.641+0000 7ff784800700 1 -- 172.21.15.99:0/2208589874 shutdown_connections 2024-03-30T00:35:52.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.641+0000 7ff784800700 1 --2- 172.21.15.99:0/2208589874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff77c106aa0 0x7ff77c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.641+0000 7ff784800700 1 --2- 172.21.15.99:0/2208589874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c101840 0x7ff77c103c30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.641+0000 7ff784800700 1 --2- 172.21.15.99:0/2208589874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c104170 0x7ff77c106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.641+0000 7ff784800700 1 -- 172.21.15.99:0/2208589874 >> 172.21.15.99:0/2208589874 conn(0x7ff77c0fb1d0 msgr2=0x7ff77c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.642+0000 7ff784800700 1 -- 172.21.15.99:0/2208589874 shutdown_connections 2024-03-30T00:35:52.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.642+0000 7ff784800700 1 -- 172.21.15.99:0/2208589874 wait complete. 2024-03-30T00:35:52.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.643+0000 7ff784800700 1 Processor -- start 2024-03-30T00:35:52.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.643+0000 7ff784800700 1 -- start start 2024-03-30T00:35:52.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.643+0000 7ff784800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff77c101840 0x7ff77c194a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.643+0000 7ff784800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c104170 0x7ff77c194f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.643+0000 7ff784800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c106aa0 0x7ff77c1a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.643+0000 7ff784800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c1956c0 con 0x7ff77c106aa0 2024-03-30T00:35:52.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.643+0000 7ff784800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c1a4e60 con 0x7ff77c101840 2024-03-30T00:35:52.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.643+0000 7ff784800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c1a5020 con 0x7ff77c104170 2024-03-30T00:35:52.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.644+0000 7ff782800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c106aa0 0x7ff77c1a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.644+0000 7ff782800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c106aa0 0x7ff77c1a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56340/0 (socket says 172.21.15.99:56340) 2024-03-30T00:35:52.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.644+0000 7ff782800700 1 -- 172.21.15.99:0/2305628369 learned_addr learned my addr 172.21.15.99:0/2305628369 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:52.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.644+0000 7ff781400700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c104170 0x7ff77c194f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.644+0000 7ff782800700 1 -- 172.21.15.99:0/2305628369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c104170 msgr2=0x7ff77c194f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.644+0000 7ff782800700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c104170 0x7ff77c194f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.644+0000 7ff781e00700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff77c101840 0x7ff77c194a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.644+0000 7ff782800700 1 -- 172.21.15.99:0/2305628369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff77c101840 msgr2=0x7ff77c194a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.644+0000 7ff782800700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff77c101840 0x7ff77c194a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.644+0000 7ff782800700 1 -- 172.21.15.99:0/2305628369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7700095f0 con 0x7ff77c106aa0 2024-03-30T00:35:52.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.645+0000 7ff782800700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c106aa0 0x7ff77c1a48f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff778007ee0 tx=0x7ff77800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.645+0000 7ff776a00700 1 -- 172.21.15.99:0/2305628369 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778017070 con 0x7ff77c106aa0 2024-03-30T00:35:52.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.645+0000 7ff776a00700 1 -- 172.21.15.99:0/2305628369 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff77800f040 con 0x7ff77c106aa0 2024-03-30T00:35:52.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.645+0000 7ff784800700 1 -- 172.21.15.99:0/2305628369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c1a5300 con 0x7ff77c106aa0 2024-03-30T00:35:52.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.645+0000 7ff776a00700 1 -- 172.21.15.99:0/2305628369 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778005710 con 0x7ff77c106aa0 2024-03-30T00:35:52.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.646+0000 7ff784800700 1 -- 172.21.15.99:0/2305628369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c1a5810 con 0x7ff77c106aa0 2024-03-30T00:35:52.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.650+0000 7ff776a00700 1 -- 172.21.15.99:0/2305628369 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ff778005870 con 0x7ff77c106aa0 2024-03-30T00:35:52.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.651+0000 7ff79829bb80 1 Processor -- start 2024-03-30T00:35:52.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.651+0000 7ff79829bb80 1 -- start start 2024-03-30T00:35:52.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.651+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 0x5590f9195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.652+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 0x5590f91969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.652+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196fd0 0x5590f9197450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.652+0000 7ff79829bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f90ed990 con 0x5590f9196fd0 2024-03-30T00:35:52.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.652+0000 7ff79829bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f90eb350 con 0x5590f91957d0 2024-03-30T00:35:52.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.652+0000 7ff79829bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f90e6c40 con 0x5590f9196130 2024-03-30T00:35:52.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.652+0000 7ff781e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 0x5590f91969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.652+0000 7ff781e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 0x5590f91969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55612/0 (socket says 172.21.15.99:55612) 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.652+0000 7ff776a00700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7640a11a0 0x7ff7640a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.652+0000 7ff781e00700 1 -- 172.21.15.99:0/1771570172 learned_addr learned my addr 172.21.15.99:0/1771570172 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.652+0000 7ff781400700 1 --2- 172.21.15.99:0/1771570172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 0x5590f9195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.653+0000 7ff781400700 1 --2- 172.21.15.99:0/1771570172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196fd0 0x5590f9197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.653+0000 7ff776a00700 1 -- 172.21.15.99:0/2305628369 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7ff7780cbb20 con 0x7ff77c106aa0 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.653+0000 7ff782800700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7640a11a0 0x7ff7640a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.653+0000 7ff781400700 1 -- 172.21.15.99:0/1771570172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 msgr2=0x5590f91969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.653+0000 7ff781400700 1 --2- 172.21.15.99:0/1771570172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 0x5590f91969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.653+0000 7ff781400700 1 -- 172.21.15.99:0/1771570172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196fd0 msgr2=0x5590f9197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.653+0000 7ff781400700 1 --2- 172.21.15.99:0/1771570172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196fd0 0x5590f9197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.653+0000 7ff781400700 1 -- 172.21.15.99:0/1771570172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590f8f683b0 con 0x5590f91957d0 2024-03-30T00:35:52.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.653+0000 7ff781400700 1 --2- 172.21.15.99:0/1771570172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 0x5590f9195bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0097c0 tx=0x7ff76c009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e883e3beb3d0f17 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.654+0000 7ff782800700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7640a11a0 0x7ff7640a3660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff7780953f0 tx=0x7ff77800df60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.654+0000 7ff76b000700 1 -- 172.21.15.99:0/1771570172 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c004050 con 0x5590f91957d0 2024-03-30T00:35:52.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.654+0000 7ff76b000700 1 -- 172.21.15.99:0/1771570172 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff76c004830 con 0x5590f91957d0 2024-03-30T00:35:52.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.654+0000 7ff76b000700 1 -- 172.21.15.99:0/1771570172 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0053a0 con 0x5590f91957d0 2024-03-30T00:35:52.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.655+0000 7ff79829bb80 1 -- 172.21.15.99:0/1771570172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 msgr2=0x5590f9195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.655+0000 7ff79829bb80 1 --2- 172.21.15.99:0/1771570172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 0x5590f9195bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0097c0 tx=0x7ff76c009750 comp rx=0 tx=0).stop 2024-03-30T00:35:52.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.655+0000 7ff79829bb80 1 -- 172.21.15.99:0/1771570172 shutdown_connections 2024-03-30T00:35:52.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.655+0000 7ff79829bb80 1 --2- 172.21.15.99:0/1771570172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196fd0 0x5590f9197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.655+0000 7ff79829bb80 1 --2- 172.21.15.99:0/1771570172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 0x5590f9195bf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.655+0000 7ff79829bb80 1 --2- 172.21.15.99:0/1771570172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 0x5590f91969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.655+0000 7ff79829bb80 1 -- 172.21.15.99:0/1771570172 >> 172.21.15.99:0/1771570172 conn(0x5590f90f46a0 msgr2=0x5590f91909a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.655+0000 7ff79829bb80 1 -- 172.21.15.99:0/1771570172 shutdown_connections 2024-03-30T00:35:52.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.655+0000 7ff79829bb80 1 -- 172.21.15.99:0/1771570172 wait complete. 2024-03-30T00:35:52.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.656+0000 7ff79829bb80 1 -- start start 2024-03-30T00:35:52.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.656+0000 7ff79829bb80 1 Processor -- start 2024-03-30T00:35:52.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.657+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f91957d0 0x5590f919f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.657+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 0x5590f919fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.657+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f9196fd0 0x5590f91a6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.657+0000 7ff79829bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f9082d80 con 0x5590f91957d0 2024-03-30T00:35:52.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.657+0000 7ff79829bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f9082a20 con 0x5590f9196fd0 2024-03-30T00:35:52.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.657+0000 7ff79829bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f8f6a6a0 con 0x5590f9196130 2024-03-30T00:35:52.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.657+0000 7ff79829bb80 10 client.0 _ll_register_callbacks cb 0x5590f908c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:52.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.657+0000 7ff781400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f91957d0 0x5590f919f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.657+0000 7ff781e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 0x5590f919fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.657+0000 7ff781400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f91957d0 0x5590f919f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56364/0 (socket says 172.21.15.99:56364) 2024-03-30T00:35:52.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.657+0000 7ff781400700 1 -- 172.21.15.99:0/678461336 learned_addr learned my addr 172.21.15.99:0/678461336 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:52.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.658+0000 7ff781400700 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f9196fd0 0x5590f91a6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.658+0000 7ff781e00700 1 -- 172.21.15.99:0/678461336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f9196fd0 msgr2=0x5590f91a6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.658+0000 7ff781e00700 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f9196fd0 0x5590f91a6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.658+0000 7ff781e00700 1 -- 172.21.15.99:0/678461336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f91957d0 msgr2=0x5590f919f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.658+0000 7ff781e00700 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f91957d0 0x5590f919f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.658+0000 7ff781e00700 1 -- 172.21.15.99:0/678461336 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c019040 con 0x5590f9196130 2024-03-30T00:35:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.659+0000 7ff781e00700 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 0x5590f919fbf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff770006010 tx=0x7ff770004230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c755ccf286d1c74 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.659+0000 7ff75f400700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:35:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.659+0000 7ff75f400700 1 -- 172.21.15.99:0/678461336 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77000b710 con 0x5590f9196130 2024-03-30T00:35:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.659+0000 7ff75f400700 1 -- 172.21.15.99:0/678461336 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7ff77000b870 con 0x5590f9196130 2024-03-30T00:35:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.659+0000 7ff79829bb80 1 -- 172.21.15.99:0/678461336 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590f90a7380 con 0x5590f9196130 2024-03-30T00:35:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.659+0000 7ff75f400700 1 -- 172.21.15.99:0/678461336 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff77000ba50 con 0x5590f9196130 2024-03-30T00:35:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.659+0000 7ff79829bb80 10 client.25426 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:52.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.659+0000 7ff79829bb80 1 -- 172.21.15.99:0/678461336 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5590f8f14b60 con 0x5590f9196130 2024-03-30T00:35:52.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.660+0000 7ff75f400700 1 -- 172.21.15.99:0/678461336 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff770031020 con 0x5590f9196130 2024-03-30T00:35:52.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.660+0000 7ff75f400700 1 -- 172.21.15.99:0/678461336 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff77000fa80 con 0x5590f9196130 2024-03-30T00:35:52.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.660+0000 7ff79829bb80 10 client.25426 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:52.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.660+0000 7ff79829bb80 10 client.25426 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-03-30T00:35:52.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.660+0000 7ff79829bb80 20 client.25426 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:52.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.660+0000 7ff79829bb80 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5590f91ab410 0x5590f91ad830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.660+0000 7ff79829bb80 4 client.25426 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.660+0000 7ff79829bb80 1 -- 172.21.15.99:0/678461336 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590f90df700 con 0x5590f91ab410 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.661+0000 7ff781400700 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5590f91ab410 0x5590f91ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.661+0000 7ff781400700 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5590f91ab410 0x5590f91ad830 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d2ec833dffae7ce server_cookie=d68fc66318176e31 in_seq=0 out_seq=0 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.661+0000 7ff75f400700 10 client.25426 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.662+0000 7ff75f400700 1 -- 172.21.15.99:0/678461336 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5590f90df700 con 0x5590f91ab410 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.662+0000 7ff75f400700 10 client.25426 handle_command_reply: tid=0 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.662+0000 7ff79829bb80 1 client.25426 shutdown 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.662+0000 7ff79829bb80 1 -- 172.21.15.99:0/678461336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 msgr2=0x5590f919fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.662+0000 7ff79829bb80 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 0x5590f919fbf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff770006010 tx=0x7ff770004230 comp rx=0 tx=0).stop 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.663+0000 7ff79829bb80 1 -- 172.21.15.99:0/678461336 shutdown_connections 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.663+0000 7ff79829bb80 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5590f91ab410 0x5590f91ad830 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.663+0000 7ff79829bb80 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f9196fd0 0x5590f91a6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.663+0000 7ff79829bb80 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f91957d0 0x5590f919f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.663+0000 7ff75f400700 0 client.25426 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:35:52.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.663+0000 7ff79829bb80 1 --2- 172.21.15.99:0/678461336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196130 0x5590f919fbf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.663+0000 7ff79829bb80 1 -- 172.21.15.99:0/678461336 >> 172.21.15.99:0/678461336 conn(0x5590f90f46a0 msgr2=0x5590f9194170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.663+0000 7ff79829bb80 1 -- 172.21.15.99:0/678461336 shutdown_connections 2024-03-30T00:35:52.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.663+0000 7ff79829bb80 1 -- 172.21.15.99:0/678461336 wait complete. 2024-03-30T00:35:52.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.663+0000 7ff79829bb80 20 client.25426 trim_cache size 0 max 16384 2024-03-30T00:35:52.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.685+0000 7ff79829bb80 1 Processor -- start 2024-03-30T00:35:52.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.685+0000 7ff79829bb80 1 -- start start 2024-03-30T00:35:52.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.685+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f91957d0 0x5590f91a7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.685+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f9196130 0x5590f91a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.685+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196fd0 0x5590f91973b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.685+0000 7ff79829bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f919f220 con 0x5590f9196fd0 2024-03-30T00:35:52.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.685+0000 7ff79829bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f9196850 con 0x5590f9196130 2024-03-30T00:35:52.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.685+0000 7ff79829bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f918c780 con 0x5590f91957d0 2024-03-30T00:35:52.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.686+0000 7ff781400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196fd0 0x5590f91973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.686+0000 7ff781400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f91957d0 0x5590f91a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.686+0000 7ff781400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f91957d0 0x5590f91a7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55626/0 (socket says 172.21.15.99:55626) 2024-03-30T00:35:52.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.686+0000 7ff781400700 1 -- 172.21.15.99:0/3976110567 learned_addr learned my addr 172.21.15.99:0/3976110567 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:52.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.686+0000 7ff781400700 1 -- 172.21.15.99:0/3976110567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f9196130 msgr2=0x5590f91a6ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:35:52.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.686+0000 7ff781400700 1 --2- 172.21.15.99:0/3976110567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f9196130 0x5590f91a6ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.686+0000 7ff781400700 1 -- 172.21.15.99:0/3976110567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196fd0 msgr2=0x5590f91973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.686+0000 7ff781400700 1 --2- 172.21.15.99:0/3976110567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196fd0 0x5590f91973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.686+0000 7ff781400700 1 -- 172.21.15.99:0/3976110567 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590f90acea0 con 0x5590f91957d0 2024-03-30T00:35:52.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.686+0000 7ff781400700 1 --2- 172.21.15.99:0/3976110567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f91957d0 0x5590f91a7290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009a70 tx=0x7ff76c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab347d00d9266218 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.687+0000 7ff76ba00700 1 -- 172.21.15.99:0/3976110567 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c004f90 con 0x5590f91957d0 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.687+0000 7ff76ba00700 1 -- 172.21.15.99:0/3976110567 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff76c0056d0 con 0x5590f91957d0 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.687+0000 7ff76ba00700 1 -- 172.21.15.99:0/3976110567 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c003680 con 0x5590f91957d0 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.688+0000 7ff79829bb80 1 -- 172.21.15.99:0/3976110567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f91957d0 msgr2=0x5590f91a7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.688+0000 7ff79829bb80 1 --2- 172.21.15.99:0/3976110567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f91957d0 0x5590f91a7290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009a70 tx=0x7ff76c004750 comp rx=0 tx=0).stop 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.688+0000 7ff79829bb80 1 -- 172.21.15.99:0/3976110567 shutdown_connections 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.688+0000 7ff79829bb80 1 --2- 172.21.15.99:0/3976110567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196fd0 0x5590f91973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.688+0000 7ff79829bb80 1 --2- 172.21.15.99:0/3976110567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f91957d0 0x5590f91a7290 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.688+0000 7ff79829bb80 1 --2- 172.21.15.99:0/3976110567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f9196130 0x5590f91a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.688+0000 7ff79829bb80 1 -- 172.21.15.99:0/3976110567 >> 172.21.15.99:0/3976110567 conn(0x5590f90f46a0 msgr2=0x5590f91895a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.688+0000 7ff79829bb80 1 -- 172.21.15.99:0/3976110567 shutdown_connections 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.688+0000 7ff79829bb80 1 -- 172.21.15.99:0/3976110567 wait complete. 2024-03-30T00:35:52.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.688+0000 7ff79829bb80 1 -- start start 2024-03-30T00:35:52.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.689+0000 7ff79829bb80 1 Processor -- start 2024-03-30T00:35:52.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.689+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 0x5590f91a6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.689+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196130 0x5590f91a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.689+0000 7ff79829bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196fd0 0x5590f91a5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.689+0000 7ff79829bb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f91a59a0 con 0x5590f9196130 2024-03-30T00:35:52.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.689+0000 7ff79829bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f91a5b10 con 0x5590f91957d0 2024-03-30T00:35:52.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.689+0000 7ff79829bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f91a5cd0 con 0x5590f9196fd0 2024-03-30T00:35:52.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.689+0000 7ff79829bb80 10 client.0 _ll_register_callbacks cb 0x5590f908c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:52.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.690+0000 7ff781400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 0x5590f91a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.690+0000 7ff781400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 0x5590f91a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45296/0 (socket says 172.21.15.99:45296) 2024-03-30T00:35:52.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.690+0000 7ff781400700 1 -- 172.21.15.99:0/1657586782 learned_addr learned my addr 172.21.15.99:0/1657586782 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:52.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.690+0000 7ff781400700 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196fd0 0x5590f91a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.690+0000 7ff781e00700 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196130 0x5590f91a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.690+0000 7ff781400700 1 -- 172.21.15.99:0/1657586782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196fd0 msgr2=0x5590f91a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.690+0000 7ff781400700 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196fd0 0x5590f91a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.690+0000 7ff781400700 1 -- 172.21.15.99:0/1657586782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196130 msgr2=0x5590f91a4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.690+0000 7ff781400700 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196130 0x5590f91a4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.690+0000 7ff781400700 1 -- 172.21.15.99:0/1657586782 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590f8f683b0 con 0x5590f91957d0 2024-03-30T00:35:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.690+0000 7ff781e00700 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196130 0x5590f91a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.691+0000 7ff781400700 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 0x5590f91a6820 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009790 tx=0x7ff76c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=867b83ef1439c593 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.691+0000 7ff75f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:35:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.691+0000 7ff75f400700 1 -- 172.21.15.99:0/1657586782 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c037070 con 0x5590f91957d0 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.691+0000 7ff79829bb80 1 -- 172.21.15.99:0/1657586782 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff76c004830 con 0x5590f91957d0 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.691+0000 7ff75f400700 1 -- 172.21.15.99:0/1657586782 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7ff76c005240 con 0x5590f91957d0 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.691+0000 7ff75f400700 1 -- 172.21.15.99:0/1657586782 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff76c005400 con 0x5590f91957d0 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.691+0000 7ff79829bb80 10 client.25421 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.691+0000 7ff79829bb80 1 -- 172.21.15.99:0/1657586782 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5590f8f6ccd0 con 0x5590f91957d0 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.692+0000 7ff75f400700 1 -- 172.21.15.99:0/1657586782 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff76c004830 con 0x5590f91957d0 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.692+0000 7ff75f400700 1 -- 172.21.15.99:0/1657586782 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff76c01e900 con 0x5590f91957d0 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.692+0000 7ff79829bb80 10 client.25421 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.692+0000 7ff79829bb80 10 client.25421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.692+0000 7ff79829bb80 20 client.25421 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.692+0000 7ff79829bb80 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5590f9200500 0x5590f9202920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.692+0000 7ff79829bb80 4 client.25421 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.692+0000 7ff79829bb80 1 -- 172.21.15.99:0/1657586782 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5590f8f35cf0 con 0x5590f9200500 2024-03-30T00:35:52.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.693+0000 7ff781e00700 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5590f9200500 0x5590f9202920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff75f400700 10 client.25421 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff781e00700 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5590f9200500 0x5590f9202920 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e742f7ec1e625ca server_cookie=2fa4c64097150450 in_seq=0 out_seq=0 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff75f400700 1 -- 172.21.15.99:0/1657586782 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5590f8f35cf0 con 0x5590f9200500 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff75f400700 10 client.25421 handle_command_reply: tid=0 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff79829bb80 1 client.25421 shutdown 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff79829bb80 1 -- 172.21.15.99:0/1657586782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 msgr2=0x5590f91a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff79829bb80 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 0x5590f91a6820 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009790 tx=0x7ff76c0058e0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff79829bb80 1 -- 172.21.15.99:0/1657586782 shutdown_connections 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff79829bb80 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x5590f9200500 0x5590f9202920 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff79829bb80 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5590f9196fd0 0x5590f91a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff75f400700 0 client.25421 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:35:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff79829bb80 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590f91957d0 0x5590f91a6820 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff79829bb80 1 --2- 172.21.15.99:0/1657586782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5590f9196130 0x5590f91a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.694+0000 7ff79829bb80 1 -- 172.21.15.99:0/1657586782 >> 172.21.15.99:0/1657586782 conn(0x5590f90f46a0 msgr2=0x5590f919d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.695+0000 7ff79829bb80 1 -- 172.21.15.99:0/1657586782 shutdown_connections 2024-03-30T00:35:52.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.695+0000 7ff79829bb80 1 -- 172.21.15.99:0/1657586782 wait complete. 2024-03-30T00:35:52.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.695+0000 7ff79829bb80 20 client.25421 trim_cache size 0 max 16384 2024-03-30T00:35:52.703 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:35:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 -- 172.21.15.99:0/2305628369 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7640a11a0 msgr2=0x7ff7640a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7640a11a0 0x7ff7640a3660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff7780953f0 tx=0x7ff77800df60 comp rx=0 tx=0).stop 2024-03-30T00:35:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 -- 172.21.15.99:0/2305628369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c106aa0 msgr2=0x7ff77c1a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c106aa0 0x7ff77c1a48f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff778007ee0 tx=0x7ff77800d3b0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 -- 172.21.15.99:0/2305628369 shutdown_connections 2024-03-30T00:35:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77c106aa0 0x7ff77c1a48f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff7640a11a0 0x7ff7640a3660 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff77c101840 0x7ff77c194a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 --2- 172.21.15.99:0/2305628369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff77c104170 0x7ff77c194f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 -- 172.21.15.99:0/2305628369 >> 172.21.15.99:0/2305628369 conn(0x7ff77c0fb1d0 msgr2=0x7ff77c1010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 -- 172.21.15.99:0/2305628369 shutdown_connections 2024-03-30T00:35:52.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.696+0000 7ff784800700 1 -- 172.21.15.99:0/2305628369 wait complete. 2024-03-30T00:35:52.723 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T00:35:52.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.929+0000 7f9ec1000700 1 -- 172.21.15.99:0/3948149108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ebc0ffc60 msgr2=0x7f9ebc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.929+0000 7f9ec1000700 1 --2- 172.21.15.99:0/3948149108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ebc0ffc60 0x7f9ebc100080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009990 tx=0x7f9eac009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.929+0000 7f9ec1000700 1 -- 172.21.15.99:0/3948149108 shutdown_connections 2024-03-30T00:35:52.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.929+0000 7f9ec1000700 1 --2- 172.21.15.99:0/3948149108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ebc1017f0 0x7f9ebc1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.929+0000 7f9ec1000700 1 --2- 172.21.15.99:0/3948149108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ebc0ffc60 0x7f9ebc100080 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.929+0000 7f9ec1000700 1 --2- 172.21.15.99:0/3948149108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ebc100e50 0x7f9ebc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.929+0000 7f9ec1000700 1 -- 172.21.15.99:0/3948149108 >> 172.21.15.99:0/3948149108 conn(0x7f9ebc0fb1d0 msgr2=0x7f9ebc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.929+0000 7f9ec1000700 1 -- 172.21.15.99:0/3948149108 shutdown_connections 2024-03-30T00:35:52.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.929+0000 7f9ec1000700 1 -- 172.21.15.99:0/3948149108 wait complete. 2024-03-30T00:35:52.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.931+0000 7f9ec1000700 1 Processor -- start 2024-03-30T00:35:52.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.931+0000 7f9ec1000700 1 -- start start 2024-03-30T00:35:52.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.931+0000 7f9ec1000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ebc0ffc60 0x7f9ebc19d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.931+0000 7f9ec1000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ebc100e50 0x7f9ebc19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9ec1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ebc1017f0 0x7f9ebc1a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9ec1000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc1a4dd0 con 0x7f9ebc100e50 2024-03-30T00:35:52.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9ec1000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc1a4f10 con 0x7f9ebc1017f0 2024-03-30T00:35:52.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9ec1000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc1a50a0 con 0x7f9ebc0ffc60 2024-03-30T00:35:52.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9eba800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ebc1017f0 0x7f9ebc1a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9eb9e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ebc0ffc60 0x7f9ebc19d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9eb9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ebc100e50 0x7f9ebc19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9eba800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ebc1017f0 0x7f9ebc1a4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45316/0 (socket says 172.21.15.99:45316) 2024-03-30T00:35:52.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9eb9e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ebc0ffc60 0x7f9ebc19d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55636/0 (socket says 172.21.15.99:55636) 2024-03-30T00:35:52.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9eba800700 1 -- 172.21.15.99:0/1625879575 learned_addr learned my addr 172.21.15.99:0/1625879575 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:52.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9eb9e00700 1 -- 172.21.15.99:0/1625879575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ebc1017f0 msgr2=0x7f9ebc1a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ebc1017f0 0x7f9ebc1a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9eb9e00700 1 -- 172.21.15.99:0/1625879575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ebc100e50 msgr2=0x7f9ebc19d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ebc100e50 0x7f9ebc19d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.932+0000 7f9eb9e00700 1 -- 172.21.15.99:0/1625879575 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4009c40 con 0x7f9ebc0ffc60 2024-03-30T00:35:52.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.933+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ebc0ffc60 0x7f9ebc19d2a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0098e0 tx=0x7f9eac005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.933+0000 7f9eb2a00700 1 -- 172.21.15.99:0/1625879575 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac01d070 con 0x7f9ebc0ffc60 2024-03-30T00:35:52.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.933+0000 7f9ec1000700 1 -- 172.21.15.99:0/1625879575 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac0095f0 con 0x7f9ebc0ffc60 2024-03-30T00:35:52.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.933+0000 7f9eb2a00700 1 -- 172.21.15.99:0/1625879575 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9eac00bdb0 con 0x7f9ebc0ffc60 2024-03-30T00:35:52.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.933+0000 7f9eb2a00700 1 -- 172.21.15.99:0/1625879575 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac022d40 con 0x7f9ebc0ffc60 2024-03-30T00:35:52.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.934+0000 7f9ec1000700 1 -- 172.21.15.99:0/1625879575 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc1a55b0 con 0x7f9ebc0ffc60 2024-03-30T00:35:52.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.937+0000 7f9eb2a00700 1 -- 172.21.15.99:0/1625879575 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f9eac022470 con 0x7f9ebc0ffc60 2024-03-30T00:35:52.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.938+0000 7f9eb2a00700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea00a11a0 0x7f9ea00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.938+0000 7f9eb2a00700 1 -- 172.21.15.99:0/1625879575 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f9eac0cdee0 con 0x7f9ebc0ffc60 2024-03-30T00:35:52.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.939+0000 7f9eb9400700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea00a11a0 0x7f9ea00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.939+0000 7f9eb9400700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea00a11a0 0x7f9ea00a3660 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009c60 tx=0x7f9ea8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.940+0000 7f9ed4b66b80 1 Processor -- start 2024-03-30T00:35:52.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.940+0000 7f9ed4b66b80 1 -- start start 2024-03-30T00:35:52.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.941+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880dcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.941+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dd130 0x55be880dd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.941+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880de450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.941+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be88034990 con 0x55be880dd130 2024-03-30T00:35:52.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.941+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be88032350 con 0x55be880dc7d0 2024-03-30T00:35:52.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.941+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8802dc40 con 0x55be880ddfd0 2024-03-30T00:35:52.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.942+0000 7f9eb9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880dcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.942+0000 7f9eb9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880dcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45330/0 (socket says 172.21.15.99:45330) 2024-03-30T00:35:52.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.942+0000 7f9eb9e00700 1 -- 172.21.15.99:0/2229930538 learned_addr learned my addr 172.21.15.99:0/2229930538 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:52.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.942+0000 7f9eb9400700 1 --2- 172.21.15.99:0/2229930538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dd130 0x55be880dd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.942+0000 7f9eba800700 1 --2- 172.21.15.99:0/2229930538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880de450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.942+0000 7f9eb9e00700 1 -- 172.21.15.99:0/2229930538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 msgr2=0x55be880de450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.942+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/2229930538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880de450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.942+0000 7f9eb9e00700 1 -- 172.21.15.99:0/2229930538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dd130 msgr2=0x55be880dd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.942+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/2229930538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dd130 0x55be880dd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.942+0000 7f9eb9e00700 1 -- 172.21.15.99:0/2229930538 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be87eaf3b0 con 0x55be880dc7d0 2024-03-30T00:35:52.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.942+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/2229930538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880dcbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0977e0 tx=0x7f9eac009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2bd235053c1dfd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.943+0000 7f9ea7400700 1 -- 172.21.15.99:0/2229930538 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac022ea0 con 0x55be880dc7d0 2024-03-30T00:35:52.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.943+0000 7f9ea7400700 1 -- 172.21.15.99:0/2229930538 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9eac09b9e0 con 0x55be880dc7d0 2024-03-30T00:35:52.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.943+0000 7f9ea7400700 1 -- 172.21.15.99:0/2229930538 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac09a7d0 con 0x55be880dc7d0 2024-03-30T00:35:52.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.944+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2229930538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 msgr2=0x55be880dcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.944+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/2229930538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880dcbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0977e0 tx=0x7f9eac009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.944+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2229930538 shutdown_connections 2024-03-30T00:35:52.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.944+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/2229930538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880de450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.944+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/2229930538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880dcbf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.944+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/2229930538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dd130 0x55be880dd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.944+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2229930538 >> 172.21.15.99:0/2229930538 conn(0x55be8803b6a0 msgr2=0x55be880d79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.944+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2229930538 shutdown_connections 2024-03-30T00:35:52.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.944+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2229930538 wait complete. 2024-03-30T00:35:52.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.945+0000 7f9ed4b66b80 1 -- start start 2024-03-30T00:35:52.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.945+0000 7f9ed4b66b80 1 Processor -- start 2024-03-30T00:35:52.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.946+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.946+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dd130 0x55be880e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.946+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880ed920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.946+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be87fc9d80 con 0x55be880dd130 2024-03-30T00:35:52.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.946+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be87fc9a20 con 0x55be880dc7d0 2024-03-30T00:35:52.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.946+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be87eb16a0 con 0x55be880ddfd0 2024-03-30T00:35:52.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.946+0000 7f9ed4b66b80 10 client.0 _ll_register_callbacks cb 0x55be87fd3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.946+0000 7f9eb9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eb9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880e63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45344/0 (socket says 172.21.15.99:45344) 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eb9e00700 1 -- 172.21.15.99:0/176051929 learned_addr learned my addr 172.21.15.99:0/176051929 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eba800700 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eb9400700 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dd130 0x55be880e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eb9e00700 1 -- 172.21.15.99:0/176051929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 msgr2=0x55be880ed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880ed920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eb9e00700 1 -- 172.21.15.99:0/176051929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dd130 msgr2=0x55be880e68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dd130 0x55be880e68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eb9e00700 1 -- 172.21.15.99:0/176051929 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc1a55b0 con 0x55be880dc7d0 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eb9400700 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dd130 0x55be880e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:52.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eba800700 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:52.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880e63a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0a31a0 tx=0x7f9eac09ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1380edf09d072ead server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.947+0000 7f9e9be00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:35:52.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.948+0000 7f9e9be00700 1 -- 172.21.15.99:0/176051929 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac022ea0 con 0x55be880dc7d0 2024-03-30T00:35:52.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.948+0000 7f9e9be00700 1 -- 172.21.15.99:0/176051929 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f9eac099680 con 0x55be880dc7d0 2024-03-30T00:35:52.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.948+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/176051929 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be87fee380 con 0x55be880dc7d0 2024-03-30T00:35:52.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.948+0000 7f9e9be00700 1 -- 172.21.15.99:0/176051929 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9eac0a3450 con 0x55be880dc7d0 2024-03-30T00:35:52.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.948+0000 7f9ed4b66b80 10 client.25433 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:52.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.948+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/176051929 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55be87e5bb60 con 0x55be880dc7d0 2024-03-30T00:35:52.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.948+0000 7f9e9be00700 1 -- 172.21.15.99:0/176051929 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55be87fee380 con 0x55be880dc7d0 2024-03-30T00:35:52.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.949+0000 7f9e9be00700 1 -- 172.21.15.99:0/176051929 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9eac064020 con 0x55be880dc7d0 2024-03-30T00:35:52.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.949+0000 7f9ed4b66b80 10 client.25433 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:52.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.949+0000 7f9ed4b66b80 10 client.25433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-30T00:35:52.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.949+0000 7f9ed4b66b80 20 client.25433 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:52.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.949+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55be880f1cd0 0x55be880f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.950+0000 7f9ed4b66b80 4 client.25433 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:52.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.950+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/176051929 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be88026700 con 0x55be880f1cd0 2024-03-30T00:35:52.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.950+0000 7f9eb9400700 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55be880f1cd0 0x55be880f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.950+0000 7f9eb9400700 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55be880f1cd0 0x55be880f40f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b4c0144089ff05f server_cookie=43e80096d95e7b41 in_seq=0 out_seq=0 2024-03-30T00:35:52.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.950+0000 7f9e9be00700 10 client.25433 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:35:52.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.951+0000 7f9e9be00700 1 -- 172.21.15.99:0/176051929 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55be88026700 con 0x55be880f1cd0 2024-03-30T00:35:52.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.951+0000 7f9e9be00700 10 client.25433 handle_command_reply: tid=0 2024-03-30T00:35:52.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.951+0000 7f9ed4b66b80 1 client.25433 shutdown 2024-03-30T00:35:52.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/176051929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 msgr2=0x55be880e63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880e63a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0a31a0 tx=0x7f9eac09ad70 comp rx=0 tx=0).stop 2024-03-30T00:35:52.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/176051929 shutdown_connections 2024-03-30T00:35:52.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55be880f1cd0 0x55be880f40f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880e63a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/176051929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dd130 0x55be880e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9e9be00700 0 client.25433 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:35:52.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/176051929 >> 172.21.15.99:0/176051929 conn(0x55be8803b6a0 msgr2=0x55be880db170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/176051929 shutdown_connections 2024-03-30T00:35:52.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/176051929 wait complete. 2024-03-30T00:35:52.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.952+0000 7f9ed4b66b80 20 client.25433 trim_cache size 0 max 16384 2024-03-30T00:35:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.978+0000 7f9ed4b66b80 1 Processor -- start 2024-03-30T00:35:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.978+0000 7f9ed4b66b80 1 -- start start 2024-03-30T00:35:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.978+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.978+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880dd130 0x55be880ed8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.978+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880ddfd0 0x55be880e5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.978+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be88026700 con 0x55be880ddfd0 2024-03-30T00:35:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.978+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be87fee380 con 0x55be880dc7d0 2024-03-30T00:35:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.978+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be880dde50 con 0x55be880dd130 2024-03-30T00:35:52.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.979+0000 7f9eba800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880ddfd0 0x55be880e5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.979+0000 7f9eb9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.979+0000 7f9eb9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880edfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45350/0 (socket says 172.21.15.99:45350) 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.979+0000 7f9eb9e00700 1 -- 172.21.15.99:0/3278651314 learned_addr learned my addr 172.21.15.99:0/3278651314 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.979+0000 7f9eb9400700 1 --2- 172.21.15.99:0/3278651314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880dd130 0x55be880ed8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.979+0000 7f9eb9e00700 1 -- 172.21.15.99:0/3278651314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880dd130 msgr2=0x55be880ed8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.979+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/3278651314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880dd130 0x55be880ed8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.979+0000 7f9eb9e00700 1 -- 172.21.15.99:0/3278651314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880ddfd0 msgr2=0x55be880e5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.979+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/3278651314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880ddfd0 0x55be880e5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.979+0000 7f9eb9e00700 1 -- 172.21.15.99:0/3278651314 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be87ff3ea0 con 0x55be880dc7d0 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.979+0000 7f9eba800700 1 --2- 172.21.15.99:0/3278651314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880ddfd0 0x55be880e5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.980+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/3278651314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880edfd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9eac003f60 tx=0x7f9eac00b2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb445dae1db733d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.980+0000 7f9ea7e00700 1 -- 172.21.15.99:0/3278651314 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac022ea0 con 0x55be880dc7d0 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.980+0000 7f9ea7e00700 1 -- 172.21.15.99:0/3278651314 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9eac0a3450 con 0x55be880dc7d0 2024-03-30T00:35:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.980+0000 7f9ea7e00700 1 -- 172.21.15.99:0/3278651314 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0b9dc0 con 0x55be880dc7d0 2024-03-30T00:35:52.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.981+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/3278651314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 msgr2=0x55be880edfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.981+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/3278651314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880edfd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9eac003f60 tx=0x7f9eac00b2e0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.981+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/3278651314 shutdown_connections 2024-03-30T00:35:52.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.981+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/3278651314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880ddfd0 0x55be880e5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.981+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/3278651314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dc7d0 0x55be880edfd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.981+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/3278651314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880dd130 0x55be880ed8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.981+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/3278651314 >> 172.21.15.99:0/3278651314 conn(0x55be8803b6a0 msgr2=0x55be880d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.981+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/3278651314 shutdown_connections 2024-03-30T00:35:52.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.981+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/3278651314 wait complete. 2024-03-30T00:35:52.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.982+0000 7f9ed4b66b80 1 -- start start 2024-03-30T00:35:52.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.982+0000 7f9ed4b66b80 1 Processor -- start 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.983+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dc7d0 0x55be880eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.983+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dd130 0x55be880eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.983+0000 7f9ed4b66b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880ec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.983+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be880ec7a0 con 0x55be880dc7d0 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.983+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be880ec910 con 0x55be880dd130 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.983+0000 7f9ed4b66b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be880ecad0 con 0x55be880ddfd0 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.983+0000 7f9ed4b66b80 10 client.0 _ll_register_callbacks cb 0x55be87fd3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.983+0000 7f9eb9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dc7d0 0x55be880eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.983+0000 7f9eb9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dc7d0 0x55be880eaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56438/0 (socket says 172.21.15.99:56438) 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.983+0000 7f9eb9e00700 1 -- 172.21.15.99:0/2461143447 learned_addr learned my addr 172.21.15.99:0/2461143447 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.983+0000 7f9eba800700 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9eb9400700 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dd130 0x55be880eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9eb9e00700 1 -- 172.21.15.99:0/2461143447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 msgr2=0x55be880ec260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880ec260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9eb9e00700 1 -- 172.21.15.99:0/2461143447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dd130 msgr2=0x55be880eb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dd130 0x55be880eb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9eb9e00700 1 -- 172.21.15.99:0/2461143447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb40114c0 con 0x55be880dc7d0 2024-03-30T00:35:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9eba800700 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9eb9e00700 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dc7d0 0x55be880eaf40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0a31a0 tx=0x7f9eac097620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b7d77a2268655b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9e9be00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9e9be00700 1 -- 172.21.15.99:0/2461143447 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac022ea0 con 0x55be880dc7d0 2024-03-30T00:35:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9e9be00700 1 -- 172.21.15.99:0/2461143447 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f9eac0b9840 con 0x55be880dc7d0 2024-03-30T00:35:52.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.984+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2461143447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9eac09b9e0 con 0x55be880dc7d0 2024-03-30T00:35:52.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.985+0000 7f9e9be00700 1 -- 172.21.15.99:0/2461143447 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9eac0a12e0 con 0x55be880dc7d0 2024-03-30T00:35:52.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.985+0000 7f9ed4b66b80 10 client.15729 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:52.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.985+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2461143447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55be87e5bb60 con 0x55be880dc7d0 2024-03-30T00:35:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.985+0000 7f9e9be00700 1 -- 172.21.15.99:0/2461143447 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9eac09b9e0 con 0x55be880dc7d0 2024-03-30T00:35:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.985+0000 7f9e9be00700 1 -- 172.21.15.99:0/2461143447 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9eac064030 con 0x55be880dc7d0 2024-03-30T00:35:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.986+0000 7f9ed4b66b80 10 client.15729 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.986+0000 7f9ed4b66b80 10 client.15729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-30T00:35:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.986+0000 7f9ed4b66b80 20 client.15729 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.986+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55be88145af0 0x55be88147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.986+0000 7f9ed4b66b80 4 client.15729 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:35:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.986+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2461143447 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be87e7ccf0 con 0x55be88145af0 2024-03-30T00:35:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.986+0000 7f9eb9400700 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55be88145af0 0x55be88147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.987+0000 7f9eb9400700 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55be88145af0 0x55be88147ee0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ad7e0de9a7a2f2f server_cookie=4061e918b6963338 in_seq=0 out_seq=0 2024-03-30T00:35:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.987+0000 7f9e9be00700 10 client.15729 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:35:52.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.987+0000 7f9e9be00700 1 -- 172.21.15.99:0/2461143447 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be87e7ccf0 con 0x55be88145af0 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.987+0000 7f9e9be00700 10 client.15729 handle_command_reply: tid=0 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.987+0000 7f9ed4b66b80 1 client.15729 shutdown 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2461143447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dc7d0 msgr2=0x55be880eaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dc7d0 0x55be880eaf40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0a31a0 tx=0x7f9eac097620 comp rx=0 tx=0).stop 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2461143447 shutdown_connections 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55be88145af0 0x55be88147ee0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55be880ddfd0 0x55be880ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be880dc7d0 0x55be880eaf40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9ed4b66b80 1 --2- 172.21.15.99:0/2461143447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be880dd130 0x55be880eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2461143447 >> 172.21.15.99:0/2461143447 conn(0x55be8803b6a0 msgr2=0x55be880e5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9e9be00700 0 client.15729 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2461143447 shutdown_connections 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9ed4b66b80 1 -- 172.21.15.99:0/2461143447 wait complete. 2024-03-30T00:35:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.988+0000 7f9ed4b66b80 20 client.15729 trim_cache size 0 max 16384 2024-03-30T00:35:52.997 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:35:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 -- 172.21.15.99:0/1625879575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea00a11a0 msgr2=0x7f9ea00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea00a11a0 0x7f9ea00a3660 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009c60 tx=0x7f9ea8009260 comp rx=0 tx=0).stop 2024-03-30T00:35:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 -- 172.21.15.99:0/1625879575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ebc0ffc60 msgr2=0x7f9ebc19d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ebc0ffc60 0x7f9ebc19d2a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0098e0 tx=0x7f9eac005740 comp rx=0 tx=0).stop 2024-03-30T00:35:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 -- 172.21.15.99:0/1625879575 shutdown_connections 2024-03-30T00:35:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f9ea00a11a0 0x7f9ea00a3660 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ebc1017f0 0x7f9ebc1a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ebc0ffc60 0x7f9ebc19d2a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 --2- 172.21.15.99:0/1625879575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ebc100e50 0x7f9ebc19d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 -- 172.21.15.99:0/1625879575 >> 172.21.15.99:0/1625879575 conn(0x7f9ebc0fb1d0 msgr2=0x7f9ebc10d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 -- 172.21.15.99:0/1625879575 shutdown_connections 2024-03-30T00:35:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:52.989+0000 7f9ec1000700 1 -- 172.21.15.99:0/1625879575 wait complete. 2024-03-30T00:35:53.015 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T00:35:53.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.228+0000 7fb6c9600700 1 -- 172.21.15.99:0/1485210522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4108c70 msgr2=0x7fb6c410b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.228+0000 7fb6c9600700 1 --2- 172.21.15.99:0/1485210522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4108c70 0x7fb6c410b100 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00b0a0 tx=0x7fb6bc00b3b0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.228+0000 7fb6c9600700 1 -- 172.21.15.99:0/1485210522 shutdown_connections 2024-03-30T00:35:53.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.228+0000 7fb6c9600700 1 --2- 172.21.15.99:0/1485210522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4108c70 0x7fb6c410b100 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.228+0000 7fb6c9600700 1 --2- 172.21.15.99:0/1485210522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c4074d70 0x7fb6c40731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.228+0000 7fb6c9600700 1 --2- 172.21.15.99:0/1485210522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c4073790 0x7fb6c4073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.228+0000 7fb6c9600700 1 -- 172.21.15.99:0/1485210522 >> 172.21.15.99:0/1485210522 conn(0x7fb6c40fc2f0 msgr2=0x7fb6c40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.228+0000 7fb6c9600700 1 -- 172.21.15.99:0/1485210522 shutdown_connections 2024-03-30T00:35:53.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.228+0000 7fb6c9600700 1 -- 172.21.15.99:0/1485210522 wait complete. 2024-03-30T00:35:53.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.229+0000 7fb6c9600700 1 Processor -- start 2024-03-30T00:35:53.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.230+0000 7fb6c9600700 1 -- start start 2024-03-30T00:35:53.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.230+0000 7fb6c9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c4073790 0x7fb6c4198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.230+0000 7fb6c9600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4074d70 0x7fb6c41993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.230+0000 7fb6c9600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c4108c70 0x7fb6c41a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.230+0000 7fb6c9600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41a09e0 con 0x7fb6c4074d70 2024-03-30T00:35:53.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.230+0000 7fb6c9600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41a0b20 con 0x7fb6c4073790 2024-03-30T00:35:53.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.230+0000 7fb6c9600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41a0cb0 con 0x7fb6c4108c70 2024-03-30T00:35:53.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.231+0000 7fb6c2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c4073790 0x7fb6c4198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.231+0000 7fb6c2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4074d70 0x7fb6c41993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.231+0000 7fb6c2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c4073790 0x7fb6c4198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45386/0 (socket says 172.21.15.99:45386) 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.231+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1581509013 learned_addr learned my addr 172.21.15.99:0/1581509013 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.231+0000 7fb6c3400700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c4108c70 0x7fb6c41a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.231+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1581509013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c4108c70 msgr2=0x7fb6c41a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.231+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c4108c70 0x7fb6c41a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.231+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1581509013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4074d70 msgr2=0x7fb6c41993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.231+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4074d70 0x7fb6c41993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.231+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1581509013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc009d00 con 0x7fb6c4073790 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.231+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c4073790 0x7fb6c4198eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400e9a0 tx=0x7fb6b400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.232+0000 7fb6bb400700 1 -- 172.21.15.99:0/1581509013 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4009790 con 0x7fb6c4073790 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.232+0000 7fb6bb400700 1 -- 172.21.15.99:0/1581509013 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6b400cbd0 con 0x7fb6c4073790 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.232+0000 7fb6c9600700 1 -- 172.21.15.99:0/1581509013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c41a0f90 con 0x7fb6c4073790 2024-03-30T00:35:53.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.232+0000 7fb6bb400700 1 -- 172.21.15.99:0/1581509013 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4018680 con 0x7fb6c4073790 2024-03-30T00:35:53.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.232+0000 7fb6c9600700 1 -- 172.21.15.99:0/1581509013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c41a13b0 con 0x7fb6c4073790 2024-03-30T00:35:53.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.235+0000 7fb6bb400700 1 -- 172.21.15.99:0/1581509013 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fb6b400cd40 con 0x7fb6c4073790 2024-03-30T00:35:53.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.237+0000 7fb6dd0b7b80 1 Processor -- start 2024-03-30T00:35:53.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.237+0000 7fb6dd0b7b80 1 -- start start 2024-03-30T00:35:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.237+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559d7d0 0x55e75559dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.237+0000 7fb6bb400700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb6a80a1120 0x7fb6a80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.237+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559e130 0x55e75559e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.237+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559efd0 0x55e75559f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.237+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7554f5990 con 0x55e75559efd0 2024-03-30T00:35:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.237+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7554f3350 con 0x55e75559d7d0 2024-03-30T00:35:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.237+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7554eec40 con 0x55e75559e130 2024-03-30T00:35:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.237+0000 7fb6bb400700 1 -- 172.21.15.99:0/1581509013 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fb6b4014070 con 0x7fb6c4073790 2024-03-30T00:35:53.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.238+0000 7fb6c2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559d7d0 0x55e75559dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.238+0000 7fb6c2000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559e130 0x55e75559e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.238+0000 7fb6c3400700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb6a80a1120 0x7fb6a80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.238+0000 7fb6c2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559d7d0 0x55e75559dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45398/0 (socket says 172.21.15.99:45398) 2024-03-30T00:35:53.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.238+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1367428647 learned_addr learned my addr 172.21.15.99:0/1367428647 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:53.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.238+0000 7fb6c3400700 1 --2- 172.21.15.99:0/1367428647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559efd0 0x55e75559f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.238+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1367428647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559e130 msgr2=0x55e75559e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.238+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/1367428647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559e130 0x55e75559e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.238+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1367428647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559efd0 msgr2=0x55e75559f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.238+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/1367428647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559efd0 0x55e75559f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.238+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1367428647 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7553703b0 con 0x55e75559d7d0 2024-03-30T00:35:53.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.239+0000 7fb6c3400700 1 --2- 172.21.15.99:0/1367428647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559efd0 0x55e75559f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:53.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.239+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/1367428647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559d7d0 0x55e75559dbf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4099f10 tx=0x7fb6b4099b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ed3d7c910a9407e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.239+0000 7fb6c3400700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb6a80a1120 0x7fb6a80a35e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009850 tx=0x7fb6bc0097a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.239+0000 7fb6af400700 1 -- 172.21.15.99:0/1367428647 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b409a680 con 0x55e75559d7d0 2024-03-30T00:35:53.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.239+0000 7fb6af400700 1 -- 172.21.15.99:0/1367428647 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6b4099f80 con 0x55e75559d7d0 2024-03-30T00:35:53.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.239+0000 7fb6af400700 1 -- 172.21.15.99:0/1367428647 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4082d00 con 0x55e75559d7d0 2024-03-30T00:35:53.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.240+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/1367428647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559d7d0 msgr2=0x55e75559dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.240+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/1367428647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559d7d0 0x55e75559dbf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4099f10 tx=0x7fb6b4099b40 comp rx=0 tx=0).stop 2024-03-30T00:35:53.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.240+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/1367428647 shutdown_connections 2024-03-30T00:35:53.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.240+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/1367428647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559efd0 0x55e75559f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.240+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/1367428647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559d7d0 0x55e75559dbf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.240+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/1367428647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559e130 0x55e75559e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.240+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/1367428647 >> 172.21.15.99:0/1367428647 conn(0x55e7554fc6a0 msgr2=0x55e7555989a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.241+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/1367428647 shutdown_connections 2024-03-30T00:35:53.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.241+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/1367428647 wait complete. 2024-03-30T00:35:53.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.241+0000 7fb6dd0b7b80 1 -- start start 2024-03-30T00:35:53.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.242+0000 7fb6dd0b7b80 1 Processor -- start 2024-03-30T00:35:53.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.242+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559d7d0 0x55e7555a7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.242+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 0x55e7555a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.242+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 0x55e7555aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.242+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75548ad80 con 0x55e75559e130 2024-03-30T00:35:53.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.242+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75548aa20 con 0x55e75559efd0 2024-03-30T00:35:53.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.242+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7553726a0 con 0x55e75559d7d0 2024-03-30T00:35:53.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.243+0000 7fb6dd0b7b80 10 client.0 _ll_register_callbacks cb 0x55e755494470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.243+0000 7fb6c2a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559d7d0 0x55e7555a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.243+0000 7fb6c2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 0x55e7555a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.243+0000 7fb6c2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 0x55e7555a7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56470/0 (socket says 172.21.15.99:56470) 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.243+0000 7fb6c2000700 1 -- 172.21.15.99:0/179303749 learned_addr learned my addr 172.21.15.99:0/179303749 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.243+0000 7fb6c3400700 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 0x55e7555aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.243+0000 7fb6c2000700 1 -- 172.21.15.99:0/179303749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559d7d0 msgr2=0x55e7555a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.243+0000 7fb6c2000700 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559d7d0 0x55e7555a7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.243+0000 7fb6c2000700 1 -- 172.21.15.99:0/179303749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 msgr2=0x55e7555aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.243+0000 7fb6c2000700 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 0x55e7555aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.244+0000 7fb6c2000700 1 -- 172.21.15.99:0/179303749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c41a13b0 con 0x55e75559e130 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.244+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559d7d0 0x55e7555a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.244+0000 7fb6c2000700 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 0x55e7555a7ba0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0012760 tx=0x7fb6b0012a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb268cad53254668 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.244+0000 7fb6a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.244+0000 7fb6a3e00700 1 -- 172.21.15.99:0/179303749 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0017070 con 0x55e75559e130 2024-03-30T00:35:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.244+0000 7fb6a3e00700 1 -- 172.21.15.99:0/179303749 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fb6b000f480 con 0x55e75559e130 2024-03-30T00:35:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.244+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/179303749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7554af380 con 0x55e75559e130 2024-03-30T00:35:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.244+0000 7fb6a3e00700 1 -- 172.21.15.99:0/179303749 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6b00122e0 con 0x55e75559e130 2024-03-30T00:35:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.244+0000 7fb6dd0b7b80 10 client.15741 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.245+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/179303749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e75531cb60 con 0x55e75559e130 2024-03-30T00:35:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.245+0000 7fb6a3e00700 1 -- 172.21.15.99:0/179303749 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e7554af380 con 0x55e75559e130 2024-03-30T00:35:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.245+0000 7fb6a3e00700 1 -- 172.21.15.99:0/179303749 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6b00267e0 con 0x55e75559e130 2024-03-30T00:35:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.245+0000 7fb6dd0b7b80 10 client.15741 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:53.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.245+0000 7fb6dd0b7b80 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.245+0000 7fb6dd0b7b80 20 client.15741 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.245+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7555b33f0 0x55e7555b5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.245+0000 7fb6dd0b7b80 4 client.15741 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.245+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/179303749 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7554e7700 con 0x55e7555b33f0 2024-03-30T00:35:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.246+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7555b33f0 0x55e7555b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.246+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7555b33f0 0x55e7555b5810 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0a860bbba50017 server_cookie=9a6991d1a82506a3 in_seq=0 out_seq=0 2024-03-30T00:35:53.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.246+0000 7fb6a3e00700 10 client.15741 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:53.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6a3e00700 1 -- 172.21.15.99:0/179303749 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e7554e7700 con 0x55e7555b33f0 2024-03-30T00:35:53.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6a3e00700 10 client.15741 handle_command_reply: tid=0 2024-03-30T00:35:53.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6dd0b7b80 1 client.15741 shutdown 2024-03-30T00:35:53.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/179303749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 msgr2=0x55e7555a7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 0x55e7555a7ba0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0012760 tx=0x7fb6b0012a70 comp rx=0 tx=0).stop 2024-03-30T00:35:53.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/179303749 shutdown_connections 2024-03-30T00:35:53.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7555b33f0 0x55e7555b5810 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 0x55e7555aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559d7d0 0x55e7555a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/179303749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 0x55e7555a7ba0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/179303749 >> 172.21.15.99:0/179303749 conn(0x55e7554fc6a0 msgr2=0x55e75559c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.247+0000 7fb6a3e00700 0 client.15741 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:53.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.248+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/179303749 shutdown_connections 2024-03-30T00:35:53.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.248+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/179303749 wait complete. 2024-03-30T00:35:53.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.248+0000 7fb6dd0b7b80 20 client.15741 trim_cache size 0 max 16384 2024-03-30T00:35:53.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.262+0000 7fb6dd0b7b80 1 Processor -- start 2024-03-30T00:35:53.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.262+0000 7fb6dd0b7b80 1 -- start start 2024-03-30T00:35:53.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.262+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559d7d0 0x55e7555af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.262+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559e130 0x55e7555a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.262+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559efd0 0x55e7555aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.262+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7555afd90 con 0x55e75559d7d0 2024-03-30T00:35:53.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.262+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7555a7200 con 0x55e75559e130 2024-03-30T00:35:53.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.262+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e75559dfc0 con 0x55e75559efd0 2024-03-30T00:35:53.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.263+0000 7fb6c2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559d7d0 0x55e7555af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.263+0000 7fb6c2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559d7d0 0x55e7555af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56480/0 (socket says 172.21.15.99:56480) 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.263+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1074365526 learned_addr learned my addr 172.21.15.99:0/1074365526 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.263+0000 7fb6c3400700 1 --2- 172.21.15.99:0/1074365526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559efd0 0x55e7555aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.263+0000 7fb6c2000700 1 --2- 172.21.15.99:0/1074365526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559e130 0x55e7555a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.263+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1074365526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559efd0 msgr2=0x55e7555aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.263+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/1074365526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559efd0 0x55e7555aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.263+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1074365526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559e130 msgr2=0x55e7555a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.263+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/1074365526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559e130 0x55e7555a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.263+0000 7fb6c2a00700 1 -- 172.21.15.99:0/1074365526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7554b4ea0 con 0x55e75559d7d0 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.263+0000 7fb6c3400700 1 --2- 172.21.15.99:0/1074365526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559efd0 0x55e7555aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.264+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/1074365526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559d7d0 0x55e7555af290 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40969f0 tx=0x7fb6b401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0d7802145065e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.264+0000 7fb6afe00700 1 -- 172.21.15.99:0/1074365526 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b409a680 con 0x55e75559d7d0 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.264+0000 7fb6afe00700 1 -- 172.21.15.99:0/1074365526 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6b40a2400 con 0x55e75559d7d0 2024-03-30T00:35:53.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.264+0000 7fb6afe00700 1 -- 172.21.15.99:0/1074365526 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4099740 con 0x55e75559d7d0 2024-03-30T00:35:53.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.265+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/1074365526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559d7d0 msgr2=0x55e7555af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.265+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/1074365526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559d7d0 0x55e7555af290 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40969f0 tx=0x7fb6b401f000 comp rx=0 tx=0).stop 2024-03-30T00:35:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.265+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/1074365526 shutdown_connections 2024-03-30T00:35:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.265+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/1074365526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e75559efd0 0x55e7555aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.265+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/1074365526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559d7d0 0x55e7555af290 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.265+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/1074365526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559e130 0x55e7555a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.265+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/1074365526 >> 172.21.15.99:0/1074365526 conn(0x55e7554fc6a0 msgr2=0x55e7555a55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.265+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/1074365526 shutdown_connections 2024-03-30T00:35:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.265+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/1074365526 wait complete. 2024-03-30T00:35:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.265+0000 7fb6dd0b7b80 1 -- start start 2024-03-30T00:35:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.266+0000 7fb6dd0b7b80 1 Processor -- start 2024-03-30T00:35:53.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.266+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 0x55e7555ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.266+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 0x55e7555acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.266+0000 7fb6dd0b7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7555adea0 0x55e7555ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.266+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7555ae860 con 0x55e75559e130 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.266+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7555ae9d0 con 0x55e75559efd0 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.266+0000 7fb6dd0b7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7555aeb90 con 0x55e7555adea0 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.266+0000 7fb6dd0b7b80 10 client.0 _ll_register_callbacks cb 0x55e755494470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.267+0000 7fb6c2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 0x55e7555ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.267+0000 7fb6c3400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7555adea0 0x55e7555ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.267+0000 7fb6c2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 0x55e7555acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.267+0000 7fb6c3400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7555adea0 0x55e7555ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55756/0 (socket says 172.21.15.99:55756) 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.267+0000 7fb6c2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 0x55e7555acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45432/0 (socket says 172.21.15.99:45432) 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.267+0000 7fb6c2000700 1 -- 172.21.15.99:0/3592441711 learned_addr learned my addr 172.21.15.99:0/3592441711 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.267+0000 7fb6c2000700 1 -- 172.21.15.99:0/3592441711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7555adea0 msgr2=0x55e7555ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.267+0000 7fb6c2000700 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7555adea0 0x55e7555ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.267+0000 7fb6c2000700 1 -- 172.21.15.99:0/3592441711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 msgr2=0x55e7555ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.267+0000 7fb6c2000700 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 0x55e7555ac9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.268+0000 7fb6c2000700 1 -- 172.21.15.99:0/3592441711 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c410bcf0 con 0x55e75559efd0 2024-03-30T00:35:53.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.268+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 0x55e7555ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:53.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.268+0000 7fb6c2000700 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 0x55e7555acee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0026bc0 tx=0x7fb6b0005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd20ea260358f6c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.268+0000 7fb6a3e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:35:53.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.268+0000 7fb6a3e00700 1 -- 172.21.15.99:0/3592441711 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0017040 con 0x55e75559efd0 2024-03-30T00:35:53.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.268+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/3592441711 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6b4099f80 con 0x55e75559efd0 2024-03-30T00:35:53.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.268+0000 7fb6a3e00700 1 -- 172.21.15.99:0/3592441711 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fb6b002b030 con 0x55e75559efd0 2024-03-30T00:35:53.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.269+0000 7fb6a3e00700 1 -- 172.21.15.99:0/3592441711 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6b0005680 con 0x55e75559efd0 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.269+0000 7fb6dd0b7b80 10 client.25457 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.269+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/3592441711 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e7553703b0 con 0x55e75559efd0 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.269+0000 7fb6a3e00700 1 -- 172.21.15.99:0/3592441711 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6b4099f80 con 0x55e75559efd0 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.269+0000 7fb6a3e00700 1 -- 172.21.15.99:0/3592441711 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb6b0010480 con 0x55e75559efd0 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.270+0000 7fb6dd0b7b80 10 client.25457 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.270+0000 7fb6dd0b7b80 10 client.25457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.270+0000 7fb6dd0b7b80 20 client.25457 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.270+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e755610ca0 0x55e7556130c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.270+0000 7fb6dd0b7b80 4 client.25457 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.270+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/3592441711 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e75533dcf0 con 0x55e755610ca0 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.270+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e755610ca0 0x55e7556130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6a3e00700 10 client.25457 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6c2a00700 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e755610ca0 0x55e7556130c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a49b87913084f server_cookie=95c350c7bcce16fa in_seq=0 out_seq=0 2024-03-30T00:35:53.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6a3e00700 1 -- 172.21.15.99:0/3592441711 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e75533dcf0 con 0x55e755610ca0 2024-03-30T00:35:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6a3e00700 10 client.25457 handle_command_reply: tid=0 2024-03-30T00:35:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6dd0b7b80 1 client.25457 shutdown 2024-03-30T00:35:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/3592441711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 msgr2=0x55e7555acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 0x55e7555acee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0026bc0 tx=0x7fb6b0005280 comp rx=0 tx=0).stop 2024-03-30T00:35:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/3592441711 shutdown_connections 2024-03-30T00:35:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e755610ca0 0x55e7556130c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7555adea0 0x55e7555ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e75559e130 0x55e7555ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6a3e00700 0 client.25457 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6dd0b7b80 1 --2- 172.21.15.99:0/3592441711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e75559efd0 0x55e7555acee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.271+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/3592441711 >> 172.21.15.99:0/3592441711 conn(0x55e7554fc6a0 msgr2=0x55e7555a7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.272+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/3592441711 shutdown_connections 2024-03-30T00:35:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.272+0000 7fb6dd0b7b80 1 -- 172.21.15.99:0/3592441711 wait complete. 2024-03-30T00:35:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.272+0000 7fb6dd0b7b80 20 client.25457 trim_cache size 0 max 16384 2024-03-30T00:35:53.281 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:35:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 -- 172.21.15.99:0/1581509013 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb6a80a1120 msgr2=0x7fb6a80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb6a80a1120 0x7fb6a80a35e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009850 tx=0x7fb6bc0097a0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 -- 172.21.15.99:0/1581509013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c4073790 msgr2=0x7fb6c4198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c4073790 0x7fb6c4198eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400e9a0 tx=0x7fb6b400ecb0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 -- 172.21.15.99:0/1581509013 shutdown_connections 2024-03-30T00:35:53.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6c4108c70 0x7fb6c41a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb6a80a1120 0x7fb6a80a35e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6c4073790 0x7fb6c4198eb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 --2- 172.21.15.99:0/1581509013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4074d70 0x7fb6c41993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 -- 172.21.15.99:0/1581509013 >> 172.21.15.99:0/1581509013 conn(0x7fb6c40fc2f0 msgr2=0x7fb6c4104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 -- 172.21.15.99:0/1581509013 shutdown_connections 2024-03-30T00:35:53.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:53.273+0000 7fb6c9600700 1 -- 172.21.15.99:0/1581509013 wait complete. 2024-03-30T00:35:53.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:53 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/415426327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:35:53.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:53 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/415426327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:35:53.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:53 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/415426327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:35:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.833+0000 7f7452a00700 1 -- 172.21.15.99:0/1441535182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c101f00 msgr2=0x7f744c102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.833+0000 7f7452a00700 1 --2- 172.21.15.99:0/1441535182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c101f00 0x7f744c102360 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f74380099c0 tx=0x7f7438009cd0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.833+0000 7f7452a00700 1 -- 172.21.15.99:0/1441535182 shutdown_connections 2024-03-30T00:35:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.833+0000 7f7452a00700 1 --2- 172.21.15.99:0/1441535182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f744c1028a0 0x7f744c109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.833+0000 7f7452a00700 1 --2- 172.21.15.99:0/1441535182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c100d10 0x7f744c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.833+0000 7f7452a00700 1 --2- 172.21.15.99:0/1441535182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c101f00 0x7f744c102360 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.833+0000 7f7452a00700 1 -- 172.21.15.99:0/1441535182 >> 172.21.15.99:0/1441535182 conn(0x7f744c0fc280 msgr2=0x7f744c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.833+0000 7f7452a00700 1 -- 172.21.15.99:0/1441535182 shutdown_connections 2024-03-30T00:35:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.833+0000 7f7452a00700 1 -- 172.21.15.99:0/1441535182 wait complete. 2024-03-30T00:35:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.834+0000 7f7452a00700 1 Processor -- start 2024-03-30T00:35:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.835+0000 7f7452a00700 1 -- start start 2024-03-30T00:35:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.835+0000 7f7452a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f744c100d10 0x7f744c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.835+0000 7f7452a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c101f00 0x7f744c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.842 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.835+0000 7f7452a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c1028a0 0x7f744c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.835+0000 7f7452a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c073e80 con 0x7f744c1028a0 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.835+0000 7f7452a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c073ff0 con 0x7f744c100d10 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.835+0000 7f7452a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c1a0b50 con 0x7f744c101f00 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.835+0000 7f7450a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c1028a0 0x7f744c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.836+0000 7f7450a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c1028a0 0x7f744c0736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56490/0 (socket says 172.21.15.99:56490) 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.836+0000 7f7450a00700 1 -- 172.21.15.99:0/727840875 learned_addr learned my addr 172.21.15.99:0/727840875 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.836+0000 7f744be00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f744c100d10 0x7f744c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.836+0000 7f744b400700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c101f00 0x7f744c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.836+0000 7f7450a00700 1 -- 172.21.15.99:0/727840875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c101f00 msgr2=0x7f744c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.836+0000 7f7450a00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c101f00 0x7f744c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.836+0000 7f7450a00700 1 -- 172.21.15.99:0/727840875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f744c100d10 msgr2=0x7f744c074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.836+0000 7f7450a00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f744c100d10 0x7f744c074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.836+0000 7f7450a00700 1 -- 172.21.15.99:0/727840875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74380095f0 con 0x7f744c1028a0 2024-03-30T00:35:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.837+0000 7f7450a00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c1028a0 0x7f744c0736e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7444007d70 tx=0x7f744400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.837+0000 7f7448c00700 1 -- 172.21.15.99:0/727840875 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744400ac90 con 0x7f744c1028a0 2024-03-30T00:35:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.837+0000 7f7448c00700 1 -- 172.21.15.99:0/727840875 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7444007e60 con 0x7f744c1028a0 2024-03-30T00:35:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.837+0000 7f7448c00700 1 -- 172.21.15.99:0/727840875 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440115e0 con 0x7f744c1028a0 2024-03-30T00:35:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.837+0000 7f7452a00700 1 -- 172.21.15.99:0/727840875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744c1a0de0 con 0x7f744c1028a0 2024-03-30T00:35:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.837+0000 7f7452a00700 1 -- 172.21.15.99:0/727840875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744c1a1210 con 0x7f744c1028a0 2024-03-30T00:35:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.841+0000 7f7448c00700 1 -- 172.21.15.99:0/727840875 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7444012420 con 0x7f744c1028a0 2024-03-30T00:35:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.842+0000 7f7448c00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74300a11a0 0x7f74300a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.842+0000 7f7448c00700 1 -- 172.21.15.99:0/727840875 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f74440ccb50 con 0x7f744c1028a0 2024-03-30T00:35:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.843+0000 7f744be00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74300a11a0 0x7f74300a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.844+0000 7f744be00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74300a11a0 0x7f74300a3660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f744c101d60 tx=0x7f743c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.844+0000 7f74665ccb80 1 Processor -- start 2024-03-30T00:35:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.844+0000 7f74665ccb80 1 -- start start 2024-03-30T00:35:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.844+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fc200 0x5644b38fc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.844+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b38fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.845+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 0x5644b38fdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.845+0000 7f74665ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b3854930 con 0x5644b38fcb60 2024-03-30T00:35:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.845+0000 7f74665ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b38522f0 con 0x5644b38fda60 2024-03-30T00:35:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.845+0000 7f74665ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b384dbe0 con 0x5644b38fc200 2024-03-30T00:35:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.845+0000 7f744b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b38fd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.845+0000 7f744be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fc200 0x5644b38fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.845+0000 7f744be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fc200 0x5644b38fc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55772/0 (socket says 172.21.15.99:55772) 2024-03-30T00:35:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.845+0000 7f744be00700 1 -- 172.21.15.99:0/1209395415 learned_addr learned my addr 172.21.15.99:0/1209395415 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.845+0000 7f7450a00700 1 --2- 172.21.15.99:0/1209395415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 0x5644b38fdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.846+0000 7f7450a00700 1 -- 172.21.15.99:0/1209395415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fc200 msgr2=0x5644b38fc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.846+0000 7f7450a00700 1 --2- 172.21.15.99:0/1209395415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fc200 0x5644b38fc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.846+0000 7f7450a00700 1 -- 172.21.15.99:0/1209395415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 msgr2=0x5644b38fd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.846+0000 7f7450a00700 1 --2- 172.21.15.99:0/1209395415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b38fd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.846+0000 7f7450a00700 1 -- 172.21.15.99:0/1209395415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644b36cf350 con 0x5644b38fda60 2024-03-30T00:35:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.846+0000 7f744b400700 1 --2- 172.21.15.99:0/1209395415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b38fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.846+0000 7f744be00700 1 --2- 172.21.15.99:0/1209395415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fc200 0x5644b38fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.846+0000 7f7450a00700 1 --2- 172.21.15.99:0/1209395415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 0x5644b38fdef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7444007d40 tx=0x7f7444099b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29f463daafbe5afe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.846+0000 7f7441200700 1 -- 172.21.15.99:0/1209395415 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444099ec0 con 0x5644b38fda60 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.846+0000 7f7441200700 1 -- 172.21.15.99:0/1209395415 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f744409dcf0 con 0x5644b38fda60 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.846+0000 7f7441200700 1 -- 172.21.15.99:0/1209395415 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444099610 con 0x5644b38fda60 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.847+0000 7f74665ccb80 1 -- 172.21.15.99:0/1209395415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 msgr2=0x5644b38fdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.848+0000 7f74665ccb80 1 --2- 172.21.15.99:0/1209395415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 0x5644b38fdef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7444007d40 tx=0x7f7444099b30 comp rx=0 tx=0).stop 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.848+0000 7f74665ccb80 1 -- 172.21.15.99:0/1209395415 shutdown_connections 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.848+0000 7f74665ccb80 1 --2- 172.21.15.99:0/1209395415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 0x5644b38fdef0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.848+0000 7f74665ccb80 1 --2- 172.21.15.99:0/1209395415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fc200 0x5644b38fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.848+0000 7f74665ccb80 1 --2- 172.21.15.99:0/1209395415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b38fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.848+0000 7f74665ccb80 1 -- 172.21.15.99:0/1209395415 >> 172.21.15.99:0/1209395415 conn(0x5644b385b640 msgr2=0x5644b38f7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.848+0000 7f74665ccb80 1 -- 172.21.15.99:0/1209395415 shutdown_connections 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.848+0000 7f74665ccb80 1 -- 172.21.15.99:0/1209395415 wait complete. 2024-03-30T00:35:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.849+0000 7f74665ccb80 1 -- start start 2024-03-30T00:35:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.849+0000 7f74665ccb80 1 Processor -- start 2024-03-30T00:35:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fc200 0x5644b3906010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b3906550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fda60 0x5644b390d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f74665ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b37e9d20 con 0x5644b38fcb60 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f74665ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b37e99c0 con 0x5644b38fc200 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f74665ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b36d1640 con 0x5644b38fda60 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f74665ccb80 10 client.0 _ll_register_callbacks cb 0x5644b37f3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f744b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b3906550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f744b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b3906550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56502/0 (socket says 172.21.15.99:56502) 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f7450a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fda60 0x5644b390d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f744b400700 1 -- 172.21.15.99:0/1361067437 learned_addr learned my addr 172.21.15.99:0/1361067437 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.850+0000 7f744be00700 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fc200 0x5644b3906010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.851+0000 7f744b400700 1 -- 172.21.15.99:0/1361067437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fda60 msgr2=0x5644b390d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.851+0000 7f744b400700 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fda60 0x5644b390d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.851+0000 7f744b400700 1 -- 172.21.15.99:0/1361067437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fc200 msgr2=0x5644b3906010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.851+0000 7f744b400700 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fc200 0x5644b3906010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.851+0000 7f744b400700 1 -- 172.21.15.99:0/1361067437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744c109aa0 con 0x5644b38fcb60 2024-03-30T00:35:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.851+0000 7f744b400700 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b3906550 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f743800b580 tx=0x7f7438005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4404a1c1081dd128 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.851+0000 7f7435600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.851+0000 7f7435600700 1 -- 172.21.15.99:0/1361067437 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438003760 con 0x5644b38fcb60 2024-03-30T00:35:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.851+0000 7f7435600700 1 -- 172.21.15.99:0/1361067437 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f74380038c0 con 0x5644b38fcb60 2024-03-30T00:35:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.851+0000 7f74665ccb80 1 -- 172.21.15.99:0/1361067437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644b380e320 con 0x5644b38fcb60 2024-03-30T00:35:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.852+0000 7f7435600700 1 -- 172.21.15.99:0/1361067437 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7438003a80 con 0x5644b38fcb60 2024-03-30T00:35:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.852+0000 7f74665ccb80 10 client.15759 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.852+0000 7f74665ccb80 1 -- 172.21.15.99:0/1361067437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f744c1a1210 con 0x5644b38fcb60 2024-03-30T00:35:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.852+0000 7f7435600700 1 -- 172.21.15.99:0/1361067437 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7438032020 con 0x5644b38fcb60 2024-03-30T00:35:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.852+0000 7f7435600700 1 -- 172.21.15.99:0/1361067437 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7438024c50 con 0x5644b38fcb60 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.853+0000 7f74665ccb80 10 client.15759 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.853+0000 7f74665ccb80 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.853+0000 7f74665ccb80 20 client.15759 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.853+0000 7f74665ccb80 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5644b3911cb0 0x5644b39140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.853+0000 7f74665ccb80 4 client.15759 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.853+0000 7f74665ccb80 1 -- 172.21.15.99:0/1361067437 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644b380e320 con 0x5644b3911cb0 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.853+0000 7f744be00700 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5644b3911cb0 0x5644b39140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.854+0000 7f7435600700 10 client.15759 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.854+0000 7f744be00700 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5644b3911cb0 0x5644b39140d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=229cba20348cd40d server_cookie=6f7a1a8fb4e4d692 in_seq=0 out_seq=0 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.855+0000 7f7435600700 1 -- 172.21.15.99:0/1361067437 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5644b380e320 con 0x5644b3911cb0 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.855+0000 7f7435600700 10 client.15759 handle_command_reply: tid=0 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.855+0000 7f74665ccb80 1 client.15759 shutdown 2024-03-30T00:35:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.855+0000 7f74665ccb80 1 -- 172.21.15.99:0/1361067437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 msgr2=0x5644b3906550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.855+0000 7f74665ccb80 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b3906550 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f743800b580 tx=0x7f7438005390 comp rx=0 tx=0).stop 2024-03-30T00:35:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.857+0000 7f74665ccb80 1 -- 172.21.15.99:0/1361067437 shutdown_connections 2024-03-30T00:35:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.857+0000 7f74665ccb80 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5644b3911cb0 0x5644b39140d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.857+0000 7f74665ccb80 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fda60 0x5644b390d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.857+0000 7f74665ccb80 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fc200 0x5644b3906010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.857+0000 7f74665ccb80 1 --2- 172.21.15.99:0/1361067437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b3906550 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.857+0000 7f74665ccb80 1 -- 172.21.15.99:0/1361067437 >> 172.21.15.99:0/1361067437 conn(0x5644b385b640 msgr2=0x5644b38fafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.857+0000 7f7435600700 0 client.15759 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.858+0000 7f74665ccb80 1 -- 172.21.15.99:0/1361067437 shutdown_connections 2024-03-30T00:35:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.858+0000 7f74665ccb80 1 -- 172.21.15.99:0/1361067437 wait complete. 2024-03-30T00:35:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.858+0000 7f74665ccb80 20 client.15759 trim_cache size 0 max 16384 2024-03-30T00:35:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.874+0000 7f74665ccb80 1 Processor -- start 2024-03-30T00:35:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.874+0000 7f74665ccb80 1 -- start start 2024-03-30T00:35:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.874+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fc200 0x5644b38f1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.874+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b390d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.874+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fda60 0x5644b3905520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.874+0000 7f74665ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b38fde40 con 0x5644b38fcb60 2024-03-30T00:35:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.874+0000 7f74665ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b3906c90 con 0x5644b38fc200 2024-03-30T00:35:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.874+0000 7f74665ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b38fe2e0 con 0x5644b38fda60 2024-03-30T00:35:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.875+0000 7f744be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fc200 0x5644b38f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.875+0000 7f744be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fc200 0x5644b38f1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45458/0 (socket says 172.21.15.99:45458) 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.875+0000 7f744be00700 1 -- 172.21.15.99:0/4191661622 learned_addr learned my addr 172.21.15.99:0/4191661622 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.875+0000 7f7450a00700 1 --2- 172.21.15.99:0/4191661622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fda60 0x5644b3905520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.875+0000 7f744b400700 1 --2- 172.21.15.99:0/4191661622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b390d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.875+0000 7f7450a00700 1 -- 172.21.15.99:0/4191661622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fc200 msgr2=0x5644b38f1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.875+0000 7f7450a00700 1 --2- 172.21.15.99:0/4191661622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fc200 0x5644b38f1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.875+0000 7f7450a00700 1 -- 172.21.15.99:0/4191661622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 msgr2=0x5644b390d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.875+0000 7f7450a00700 1 --2- 172.21.15.99:0/4191661622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b390d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.875+0000 7f7450a00700 1 -- 172.21.15.99:0/4191661622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644b367bb00 con 0x5644b38fda60 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.876+0000 7f7450a00700 1 --2- 172.21.15.99:0/4191661622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fda60 0x5644b3905520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f744400a690 tx=0x7f74440962b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc25d777c0357edf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.876+0000 7f7441c00700 1 -- 172.21.15.99:0/4191661622 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444099ec0 con 0x5644b38fda60 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.876+0000 7f7441c00700 1 -- 172.21.15.99:0/4191661622 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74440a2340 con 0x5644b38fda60 2024-03-30T00:35:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.876+0000 7f7441c00700 1 -- 172.21.15.99:0/4191661622 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744409a3f0 con 0x5644b38fda60 2024-03-30T00:35:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.877+0000 7f74665ccb80 1 -- 172.21.15.99:0/4191661622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fda60 msgr2=0x5644b3905520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.877+0000 7f74665ccb80 1 --2- 172.21.15.99:0/4191661622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fda60 0x5644b3905520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f744400a690 tx=0x7f74440962b0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.877+0000 7f74665ccb80 1 -- 172.21.15.99:0/4191661622 shutdown_connections 2024-03-30T00:35:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.877+0000 7f74665ccb80 1 --2- 172.21.15.99:0/4191661622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fda60 0x5644b3905520 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.877+0000 7f74665ccb80 1 --2- 172.21.15.99:0/4191661622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fc200 0x5644b38f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.877+0000 7f74665ccb80 1 --2- 172.21.15.99:0/4191661622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fcb60 0x5644b390d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.877+0000 7f74665ccb80 1 -- 172.21.15.99:0/4191661622 >> 172.21.15.99:0/4191661622 conn(0x5644b385b640 msgr2=0x5644b3918a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.877+0000 7f74665ccb80 1 -- 172.21.15.99:0/4191661622 shutdown_connections 2024-03-30T00:35:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.877+0000 7f74665ccb80 1 -- 172.21.15.99:0/4191661622 wait complete. 2024-03-30T00:35:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.878+0000 7f74665ccb80 1 -- start start 2024-03-30T00:35:53.885 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.878+0000 7f74665ccb80 1 Processor -- start 2024-03-30T00:35:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fc200 0x5644b390b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fcb60 0x5644b390b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f74665ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 0x5644b390c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.886 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f74665ccb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b390cc20 con 0x5644b38fc200 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f74665ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b390cd90 con 0x5644b38fda60 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f74665ccb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b390cf50 con 0x5644b38fcb60 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f74665ccb80 10 client.0 _ll_register_callbacks cb 0x5644b37f3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f744be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fc200 0x5644b390b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f744b400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fcb60 0x5644b390b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f744be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fc200 0x5644b390b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56526/0 (socket says 172.21.15.99:56526) 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f744b400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fcb60 0x5644b390b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55786/0 (socket says 172.21.15.99:55786) 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f744be00700 1 -- 172.21.15.99:0/2436595775 learned_addr learned my addr 172.21.15.99:0/2436595775 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.879+0000 7f7450a00700 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 0x5644b390c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.880+0000 7f7450a00700 1 -- 172.21.15.99:0/2436595775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fcb60 msgr2=0x5644b390b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.880+0000 7f7450a00700 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fcb60 0x5644b390b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.880+0000 7f7450a00700 1 -- 172.21.15.99:0/2436595775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fc200 msgr2=0x5644b390b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.880+0000 7f7450a00700 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fc200 0x5644b390b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.887 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.880+0000 7f7450a00700 1 -- 172.21.15.99:0/2436595775 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644b36cf350 con 0x5644b38fda60 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.880+0000 7f7450a00700 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 0x5644b390c6e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f744409a220 tx=0x7f74440998a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd2a4b5852cbe6e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.880+0000 7f7435600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.880+0000 7f7435600700 1 -- 172.21.15.99:0/2436595775 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440b8080 con 0x5644b38fda60 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.881+0000 7f74665ccb80 1 -- 172.21.15.99:0/2436595775 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f744409dcf0 con 0x5644b38fda60 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.881+0000 7f7435600700 1 -- 172.21.15.99:0/2436595775 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f7444099ec0 con 0x5644b38fda60 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.881+0000 7f7435600700 1 -- 172.21.15.99:0/2436595775 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74440995c0 con 0x5644b38fda60 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.881+0000 7f74665ccb80 10 client.25472 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.881+0000 7f74665ccb80 1 -- 172.21.15.99:0/2436595775 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5644b3813e40 con 0x5644b38fda60 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.881+0000 7f7435600700 1 -- 172.21.15.99:0/2436595775 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f744409dcf0 con 0x5644b38fda60 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.882+0000 7f7435600700 1 -- 172.21.15.99:0/2436595775 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7444004ad0 con 0x5644b38fda60 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.882+0000 7f74665ccb80 10 client.25472 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.882+0000 7f74665ccb80 10 client.25472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.882+0000 7f74665ccb80 20 client.25472 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.882+0000 7f74665ccb80 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5644b3966570 0x5644b3968990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.882+0000 7f74665ccb80 4 client.25472 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:35:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.882+0000 7f74665ccb80 1 -- 172.21.15.99:0/2436595775 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5644b38466a0 con 0x5644b3966570 2024-03-30T00:35:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.882+0000 7f744be00700 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5644b3966570 0x5644b3968990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.883+0000 7f744be00700 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5644b3966570 0x5644b3968990 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3006d500e0f9c9da server_cookie=1af428ca3b73c756 in_seq=0 out_seq=0 2024-03-30T00:35:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.883+0000 7f7435600700 10 client.25472 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.887+0000 7f7435600700 1 -- 172.21.15.99:0/2436595775 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5644b38466a0 con 0x5644b3966570 2024-03-30T00:35:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.887+0000 7f7435600700 10 client.25472 handle_command_reply: tid=0 2024-03-30T00:35:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.887+0000 7f74665ccb80 1 client.25472 shutdown 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f74665ccb80 1 -- 172.21.15.99:0/2436595775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 msgr2=0x5644b390c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f74665ccb80 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 0x5644b390c6e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f744409a220 tx=0x7f74440998a0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f74665ccb80 1 -- 172.21.15.99:0/2436595775 shutdown_connections 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f74665ccb80 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5644b3966570 0x5644b3968990 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f74665ccb80 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5644b38fda60 0x5644b390c6e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f74665ccb80 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644b38fc200 0x5644b390b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f74665ccb80 1 --2- 172.21.15.99:0/2436595775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644b38fcb60 0x5644b390b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f7435600700 0 client.25472 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f74665ccb80 1 -- 172.21.15.99:0/2436595775 >> 172.21.15.99:0/2436595775 conn(0x5644b385b640 msgr2=0x5644b3905520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f74665ccb80 1 -- 172.21.15.99:0/2436595775 shutdown_connections 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f74665ccb80 1 -- 172.21.15.99:0/2436595775 wait complete. 2024-03-30T00:35:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.888+0000 7f74665ccb80 20 client.25472 trim_cache size 0 max 16384 2024-03-30T00:35:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:35:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:35:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:35:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:35:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.889+0000 7f7452a00700 1 -- 172.21.15.99:0/727840875 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74300a11a0 msgr2=0x7f74300a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.889+0000 7f7452a00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74300a11a0 0x7f74300a3660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f744c101d60 tx=0x7f743c009260 comp rx=0 tx=0).stop 2024-03-30T00:35:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.889+0000 7f7452a00700 1 -- 172.21.15.99:0/727840875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c1028a0 msgr2=0x7f744c0736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.889+0000 7f7452a00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c1028a0 0x7f744c0736e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7444007d70 tx=0x7f744400a340 comp rx=0 tx=0).stop 2024-03-30T00:35:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.889+0000 7f7452a00700 1 -- 172.21.15.99:0/727840875 shutdown_connections 2024-03-30T00:35:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.889+0000 7f7452a00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744c1028a0 0x7f744c0736e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.889+0000 7f7452a00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74300a11a0 0x7f74300a3660 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.889+0000 7f7452a00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f744c100d10 0x7f744c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.889+0000 7f7452a00700 1 --2- 172.21.15.99:0/727840875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f744c101f00 0x7f744c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.889+0000 7f7452a00700 1 -- 172.21.15.99:0/727840875 >> 172.21.15.99:0/727840875 conn(0x7f744c0fc280 msgr2=0x7f744c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.890+0000 7f7452a00700 1 -- 172.21.15.99:0/727840875 shutdown_connections 2024-03-30T00:35:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:35:53.890+0000 7f7452a00700 1 -- 172.21.15.99:0/727840875 wait complete. 2024-03-30T00:35:54.298 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T00:35:54.299 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T00:35:54.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:54 smithi161 ceph-mon[42221]: pgmap v339: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:35:54.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:54 smithi099 ceph-mon[38079]: pgmap v339: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:35:54.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:54 smithi142 ceph-mon[41779]: pgmap v339: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:35:54.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.621+0000 7f12fb200700 1 -- 172.21.15.99:0/243501366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc101db0 msgr2=0x7f12fc102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:54.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.621+0000 7f12fb200700 1 --2- 172.21.15.99:0/243501366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc101db0 0x7f12fc102210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f12e80098f0 tx=0x7f12e8009c00 comp rx=0 tx=0).stop 2024-03-30T00:35:54.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.621+0000 7f12fb200700 1 -- 172.21.15.99:0/243501366 shutdown_connections 2024-03-30T00:35:54.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.621+0000 7f12fb200700 1 --2- 172.21.15.99:0/243501366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12fc102750 0x7f12fc109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.621+0000 7f12fb200700 1 --2- 172.21.15.99:0/243501366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc100bc0 0x7f12fc100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.621+0000 7f12fb200700 1 --2- 172.21.15.99:0/243501366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc101db0 0x7f12fc102210 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.622+0000 7f12fb200700 1 -- 172.21.15.99:0/243501366 >> 172.21.15.99:0/243501366 conn(0x7f12fc0fc110 msgr2=0x7f12fc0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:54.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.622+0000 7f12fb200700 1 -- 172.21.15.99:0/243501366 shutdown_connections 2024-03-30T00:35:54.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.623+0000 7f12fb200700 1 -- 172.21.15.99:0/243501366 wait complete. 2024-03-30T00:35:54.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.624+0000 7f12fb200700 1 Processor -- start 2024-03-30T00:35:54.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.624+0000 7f12fb200700 1 -- start start 2024-03-30T00:35:54.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12fb200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12fc100bc0 0x7f12fc074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12fb200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc101db0 0x7f12fc0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12fb200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc102750 0x7f12fc0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12fb200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc073e80 con 0x7f12fc101db0 2024-03-30T00:35:54.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12fb200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc073ff0 con 0x7f12fc100bc0 2024-03-30T00:35:54.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12fb200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc1a0ac0 con 0x7f12fc102750 2024-03-30T00:35:54.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12fa800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc102750 0x7f12fc0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12f9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12fc100bc0 0x7f12fc074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12fa800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc102750 0x7f12fc0736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55798/0 (socket says 172.21.15.99:55798) 2024-03-30T00:35:54.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12f9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12fc100bc0 0x7f12fc074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45486/0 (socket says 172.21.15.99:45486) 2024-03-30T00:35:54.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12fa800700 1 -- 172.21.15.99:0/763978889 learned_addr learned my addr 172.21.15.99:0/763978889 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:54.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.625+0000 7f12f9400700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc101db0 0x7f12fc0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.626+0000 7f12fa800700 1 -- 172.21.15.99:0/763978889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12fc100bc0 msgr2=0x7f12fc074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:54.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.626+0000 7f12fa800700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12fc100bc0 0x7f12fc074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.626+0000 7f12fa800700 1 -- 172.21.15.99:0/763978889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc101db0 msgr2=0x7f12fc0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:54.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.626+0000 7f12fa800700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc101db0 0x7f12fc0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.626+0000 7f12fa800700 1 -- 172.21.15.99:0/763978889 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e8009520 con 0x7f12fc102750 2024-03-30T00:35:54.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.626+0000 7f12f9400700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc101db0 0x7f12fc0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:54.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.626+0000 7f12fa800700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc102750 0x7f12fc0736e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12f400ef30 tx=0x7f12f400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:54.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.626+0000 7f12f2a00700 1 -- 172.21.15.99:0/763978889 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f4018070 con 0x7f12fc102750 2024-03-30T00:35:54.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.626+0000 7f12f2a00700 1 -- 172.21.15.99:0/763978889 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12f400f040 con 0x7f12fc102750 2024-03-30T00:35:54.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.626+0000 7f12f2a00700 1 -- 172.21.15.99:0/763978889 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f400cc20 con 0x7f12fc102750 2024-03-30T00:35:54.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.627+0000 7f12fb200700 1 -- 172.21.15.99:0/763978889 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc196680 con 0x7f12fc102750 2024-03-30T00:35:54.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.627+0000 7f12fb200700 1 -- 172.21.15.99:0/763978889 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc196ae0 con 0x7f12fc102750 2024-03-30T00:35:54.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.632+0000 7f12f2a00700 1 -- 172.21.15.99:0/763978889 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f12f4007500 con 0x7f12fc102750 2024-03-30T00:35:54.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.633+0000 7f1314590b80 1 Processor -- start 2024-03-30T00:35:54.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.634+0000 7f1314590b80 1 -- start start 2024-03-30T00:35:54.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.634+0000 7f1314590b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db3a60 0x562204db3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.634+0000 7f1314590b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204db4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.634+0000 7f1314590b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db5210 0x562204db5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.634+0000 7f1314590b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204d0ba10 con 0x562204db3a60 2024-03-30T00:35:54.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.634+0000 7f1314590b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204d093d0 con 0x562204db43c0 2024-03-30T00:35:54.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.634+0000 7f1314590b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204d04cc0 con 0x562204db5210 2024-03-30T00:35:54.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.634+0000 7f12f2a00700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f12e00a1120 0x7f12e00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.634+0000 7f12f2a00700 1 -- 172.21.15.99:0/763978889 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f12f40cc550 con 0x7f12fc102750 2024-03-30T00:35:54.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12f9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204db4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12f9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204db4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:45488/0 (socket says 172.21.15.99:45488) 2024-03-30T00:35:54.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12f9e00700 1 -- 172.21.15.99:0/1291028254 learned_addr learned my addr 172.21.15.99:0/1291028254 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:54.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12f9400700 1 --2- 172.21.15.99:0/1291028254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db3a60 0x562204db3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12f9e00700 1 -- 172.21.15.99:0/1291028254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db5210 msgr2=0x562204db5690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:35:54.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12f9e00700 1 --2- 172.21.15.99:0/1291028254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db5210 0x562204db5690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12f9e00700 1 -- 172.21.15.99:0/1291028254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db3a60 msgr2=0x562204db3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:54.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12f9e00700 1 --2- 172.21.15.99:0/1291028254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db3a60 0x562204db3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12f9e00700 1 -- 172.21.15.99:0/1291028254 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562204b86430 con 0x562204db43c0 2024-03-30T00:35:54.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12fa800700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f12e00a1120 0x7f12e00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12f9400700 1 --2- 172.21.15.99:0/1291028254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db3a60 0x562204db3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:54.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.635+0000 7f12f9e00700 1 --2- 172.21.15.99:0/1291028254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204db4c30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009d30 tx=0x7f12ec011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c296fd3b833dbc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:54.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.636+0000 7f12e7000700 1 -- 172.21.15.99:0/1291028254 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec004220 con 0x562204db43c0 2024-03-30T00:35:54.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.636+0000 7f12e7000700 1 -- 172.21.15.99:0/1291028254 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12ec004cc0 con 0x562204db43c0 2024-03-30T00:35:54.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.636+0000 7f12e7000700 1 -- 172.21.15.99:0/1291028254 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0054c0 con 0x562204db43c0 2024-03-30T00:35:54.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.637+0000 7f1314590b80 1 -- 172.21.15.99:0/1291028254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 msgr2=0x562204db4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:54.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.637+0000 7f1314590b80 1 --2- 172.21.15.99:0/1291028254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204db4c30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009d30 tx=0x7f12ec011720 comp rx=0 tx=0).stop 2024-03-30T00:35:54.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.637+0000 7f12fa800700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f12e00a1120 0x7f12e00a35e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f12f40090a0 tx=0x7f12f4095570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:54.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.637+0000 7f1314590b80 1 -- 172.21.15.99:0/1291028254 shutdown_connections 2024-03-30T00:35:54.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.637+0000 7f1314590b80 1 --2- 172.21.15.99:0/1291028254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db5210 0x562204db5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.637+0000 7f1314590b80 1 --2- 172.21.15.99:0/1291028254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db3a60 0x562204db3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.637+0000 7f1314590b80 1 --2- 172.21.15.99:0/1291028254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204db4c30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.637+0000 7f1314590b80 1 -- 172.21.15.99:0/1291028254 >> 172.21.15.99:0/1291028254 conn(0x562204d12720 msgr2=0x562204daeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:54.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.637+0000 7f1314590b80 1 -- 172.21.15.99:0/1291028254 shutdown_connections 2024-03-30T00:35:54.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.637+0000 7f1314590b80 1 -- 172.21.15.99:0/1291028254 wait complete. 2024-03-30T00:35:54.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.638+0000 7f1314590b80 1 -- start start 2024-03-30T00:35:54.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.638+0000 7f1314590b80 1 Processor -- start 2024-03-30T00:35:54.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.639+0000 7f1314590b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204dbd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.639+0000 7f1314590b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204dbde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.639+0000 7f1314590b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 0x562204dc4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.639+0000 7f1314590b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204ca0e00 con 0x562204db5210 2024-03-30T00:35:54.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.639+0000 7f1314590b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204ca0aa0 con 0x562204db43c0 2024-03-30T00:35:54.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.639+0000 7f1314590b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204b88720 con 0x562204db3a60 2024-03-30T00:35:54.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.639+0000 7f1314590b80 10 client.0 _ll_register_callbacks cb 0x562204caa4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:54.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.640+0000 7f12f9400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204dbd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.640+0000 7f12f9400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204dbd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55812/0 (socket says 172.21.15.99:55812) 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.640+0000 7f12f9400700 1 -- 172.21.15.99:0/1105745027 learned_addr learned my addr 172.21.15.99:0/1105745027 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.640+0000 7f12f9e00700 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204dbde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.640+0000 7f12f9400700 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 0x562204dc4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.640+0000 7f12f9e00700 1 -- 172.21.15.99:0/1105745027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 msgr2=0x562204dbd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.640+0000 7f12f9e00700 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204dbd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.640+0000 7f12f9e00700 1 -- 172.21.15.99:0/1105745027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 msgr2=0x562204dc4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.640+0000 7f12f9e00700 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 0x562204dc4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.640+0000 7f12f9e00700 1 -- 172.21.15.99:0/1105745027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ec009960 con 0x562204db43c0 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.640+0000 7f12f9400700 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 0x562204dc4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.641+0000 7f12f9e00700 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204dbde40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12ec01b040 tx=0x7f12ec0054c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b3b9cd0607a635b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.641+0000 7f12db400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:35:54.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.641+0000 7f12db400700 1 -- 172.21.15.99:0/1105745027 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0056e0 con 0x562204db43c0 2024-03-30T00:35:54.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.641+0000 7f1314590b80 1 -- 172.21.15.99:0/1105745027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562204cc5400 con 0x562204db43c0 2024-03-30T00:35:54.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.641+0000 7f12db400700 1 -- 172.21.15.99:0/1105745027 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f12ec01f040 con 0x562204db43c0 2024-03-30T00:35:54.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.642+0000 7f12db400700 1 -- 172.21.15.99:0/1105745027 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12ec024640 con 0x562204db43c0 2024-03-30T00:35:54.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.642+0000 7f1314590b80 10 client.25484 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:54.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.642+0000 7f1314590b80 1 -- 172.21.15.99:0/1105745027 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562204b32be0 con 0x562204db43c0 2024-03-30T00:35:54.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.642+0000 7f12db400700 1 -- 172.21.15.99:0/1105745027 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562204cc5400 con 0x562204db43c0 2024-03-30T00:35:54.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.642+0000 7f12db400700 1 -- 172.21.15.99:0/1105745027 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f12ec019430 con 0x562204db43c0 2024-03-30T00:35:54.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.643+0000 7f1314590b80 10 client.25484 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:54.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.643+0000 7f1314590b80 10 client.25484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:54.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.643+0000 7f1314590b80 20 client.25484 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:54.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.643+0000 7f1314590b80 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562204dc95a0 0x562204dcb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.643+0000 7f1314590b80 4 client.25484 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:54.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.643+0000 7f1314590b80 1 -- 172.21.15.99:0/1105745027 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562204cfd780 con 0x562204dc95a0 2024-03-30T00:35:54.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.643+0000 7f12f9400700 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562204dc95a0 0x562204dcb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.644+0000 7f12f9400700 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562204dc95a0 0x562204dcb990 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5eee7edd7c3b07 server_cookie=fbdea2c7bbab3a4e in_seq=0 out_seq=0 2024-03-30T00:35:54.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.644+0000 7f12db400700 10 client.25484 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:54.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f12db400700 1 -- 172.21.15.99:0/1105745027 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562204cfd780 con 0x562204dc95a0 2024-03-30T00:35:54.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f12db400700 10 client.25484 handle_command_reply: tid=0 2024-03-30T00:35:54.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f1314590b80 1 client.25484 shutdown 2024-03-30T00:35:54.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f1314590b80 1 -- 172.21.15.99:0/1105745027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 msgr2=0x562204dbde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:54.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f1314590b80 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204dbde40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12ec01b040 tx=0x7f12ec0054c0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f1314590b80 1 -- 172.21.15.99:0/1105745027 shutdown_connections 2024-03-30T00:35:54.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f1314590b80 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562204dc95a0 0x562204dcb990 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f1314590b80 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 0x562204dc4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f1314590b80 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204dbd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f1314590b80 1 --2- 172.21.15.99:0/1105745027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204dbde40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f1314590b80 1 -- 172.21.15.99:0/1105745027 >> 172.21.15.99:0/1105745027 conn(0x562204d12720 msgr2=0x562204db2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:54.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f12db400700 0 client.25484 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:54.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.645+0000 7f1314590b80 1 -- 172.21.15.99:0/1105745027 shutdown_connections 2024-03-30T00:35:54.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.646+0000 7f1314590b80 1 -- 172.21.15.99:0/1105745027 wait complete. 2024-03-30T00:35:54.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.648+0000 7f1314590b80 20 client.25484 trim_cache size 0 max 16384 2024-03-30T00:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.681+0000 7f1314590b80 1 Processor -- start 2024-03-30T00:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.681+0000 7f1314590b80 1 -- start start 2024-03-30T00:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.682+0000 7f1314590b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204da8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.682+0000 7f1314590b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db43c0 0x562204dc4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.682+0000 7f1314590b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db5210 0x562204dbd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.682+0000 7f1314590b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204db5090 con 0x562204db43c0 2024-03-30T00:35:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.682+0000 7f1314590b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204dbd490 con 0x562204db5210 2024-03-30T00:35:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.682+0000 7f1314590b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204da8e60 con 0x562204db3a60 2024-03-30T00:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.682+0000 7f12f9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db43c0 0x562204dc4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.682+0000 7f12f9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db43c0 0x562204dc4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56586/0 (socket says 172.21.15.99:56586) 2024-03-30T00:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.682+0000 7f12f9e00700 1 -- 172.21.15.99:0/1402732635 learned_addr learned my addr 172.21.15.99:0/1402732635 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.682+0000 7f12f9400700 1 --2- 172.21.15.99:0/1402732635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204da8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.682+0000 7f12f9400700 1 --2- 172.21.15.99:0/1402732635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db5210 0x562204dbd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.683+0000 7f12f9e00700 1 -- 172.21.15.99:0/1402732635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 msgr2=0x562204da8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.683+0000 7f12f9e00700 1 --2- 172.21.15.99:0/1402732635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204da8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.683+0000 7f12f9e00700 1 -- 172.21.15.99:0/1402732635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db5210 msgr2=0x562204dbd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.683+0000 7f12f9e00700 1 --2- 172.21.15.99:0/1402732635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db5210 0x562204dbd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.683+0000 7f12f9e00700 1 -- 172.21.15.99:0/1402732635 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562204ccaf20 con 0x562204db43c0 2024-03-30T00:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.683+0000 7f12f9e00700 1 --2- 172.21.15.99:0/1402732635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db43c0 0x562204dc4e40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f12ec011760 tx=0x7f12ec004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f8fcec21667d22 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:54.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.684+0000 7f12e7a00700 1 -- 172.21.15.99:0/1402732635 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec009c30 con 0x562204db43c0 2024-03-30T00:35:54.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.684+0000 7f12e7a00700 1 -- 172.21.15.99:0/1402732635 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12ec011b20 con 0x562204db43c0 2024-03-30T00:35:54.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.684+0000 7f12e7a00700 1 -- 172.21.15.99:0/1402732635 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec004e30 con 0x562204db43c0 2024-03-30T00:35:54.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.685+0000 7f1314590b80 1 -- 172.21.15.99:0/1402732635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db43c0 msgr2=0x562204dc4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:54.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.685+0000 7f1314590b80 1 --2- 172.21.15.99:0/1402732635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db43c0 0x562204dc4e40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f12ec011760 tx=0x7f12ec004bd0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.685+0000 7f1314590b80 1 -- 172.21.15.99:0/1402732635 shutdown_connections 2024-03-30T00:35:54.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.685+0000 7f1314590b80 1 --2- 172.21.15.99:0/1402732635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db5210 0x562204dbd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.685+0000 7f1314590b80 1 --2- 172.21.15.99:0/1402732635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204da8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.685+0000 7f1314590b80 1 --2- 172.21.15.99:0/1402732635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db43c0 0x562204dc4e40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.685+0000 7f1314590b80 1 -- 172.21.15.99:0/1402732635 >> 172.21.15.99:0/1402732635 conn(0x562204d12720 msgr2=0x562204db8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:54.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.685+0000 7f1314590b80 1 -- 172.21.15.99:0/1402732635 shutdown_connections 2024-03-30T00:35:54.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.685+0000 7f1314590b80 1 -- 172.21.15.99:0/1402732635 wait complete. 2024-03-30T00:35:54.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.686+0000 7f1314590b80 1 -- start start 2024-03-30T00:35:54.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.686+0000 7f1314590b80 1 Processor -- start 2024-03-30T00:35:54.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.686+0000 7f1314590b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204dc4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.686+0000 7f1314590b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204dc3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.687+0000 7f1314590b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 0x562204dc3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.687+0000 7f1314590b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204dc3cc0 con 0x562204db5210 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.687+0000 7f1314590b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204dc3e30 con 0x562204db43c0 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.687+0000 7f1314590b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562204dc3ff0 con 0x562204db3a60 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.687+0000 7f1314590b80 10 client.0 _ll_register_callbacks cb 0x562204caa4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.687+0000 7f12f9400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204dc4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.687+0000 7f12f9400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204dc4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:55836/0 (socket says 172.21.15.99:55836) 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.687+0000 7f12f9400700 1 -- 172.21.15.99:0/2767211171 learned_addr learned my addr 172.21.15.99:0/2767211171 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.687+0000 7f12f9400700 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 0x562204dc3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.688+0000 7f12f9e00700 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204dc3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.688+0000 7f12f9400700 1 -- 172.21.15.99:0/2767211171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 msgr2=0x562204dc4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:54.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.688+0000 7f12f9400700 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204dc4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.688+0000 7f12f9400700 1 -- 172.21.15.99:0/2767211171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 msgr2=0x562204dc3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:54.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.688+0000 7f12f9400700 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204dc3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.688+0000 7f12f9400700 1 -- 172.21.15.99:0/2767211171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562204b32be0 con 0x562204db5210 2024-03-30T00:35:54.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.688+0000 7f12f9400700 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 0x562204dc3740 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f12e800fa00 tx=0x7f12e8019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9278a96c18089e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:54.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.688+0000 7f12db400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:35:54.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.688+0000 7f12db400700 1 -- 172.21.15.99:0/2767211171 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e800f780 con 0x562204db5210 2024-03-30T00:35:54.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.688+0000 7f12db400700 1 -- 172.21.15.99:0/2767211171 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f12e8000e40 con 0x562204db5210 2024-03-30T00:35:54.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.688+0000 7f1314590b80 1 -- 172.21.15.99:0/2767211171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12ec004cc0 con 0x562204db5210 2024-03-30T00:35:54.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.689+0000 7f12db400700 1 -- 172.21.15.99:0/2767211171 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12e80269b0 con 0x562204db5210 2024-03-30T00:35:54.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.689+0000 7f1314590b80 10 client.15783 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.689+0000 7f1314590b80 1 -- 172.21.15.99:0/2767211171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562204b8ad50 con 0x562204db5210 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.689+0000 7f12db400700 1 -- 172.21.15.99:0/2767211171 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12ec004cc0 con 0x562204db5210 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.689+0000 7f12db400700 1 -- 172.21.15.99:0/2767211171 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f12e8015020 con 0x562204db5210 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.689+0000 7f1314590b80 10 client.15783 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.689+0000 7f1314590b80 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.689+0000 7f1314590b80 20 client.15783 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.689+0000 7f1314590b80 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562204e1ec30 0x562204e21050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.689+0000 7f1314590b80 4 client.15783 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.689+0000 7f1314590b80 1 -- 172.21.15.99:0/2767211171 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562204b53d70 con 0x562204e1ec30 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.690+0000 7f12f9e00700 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562204e1ec30 0x562204e21050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.690+0000 7f12db400700 10 client.15783 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:54.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.690+0000 7f12f9e00700 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562204e1ec30 0x562204e21050 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68ed38d3b8a85b82 server_cookie=8450f8b3aeae11a7 in_seq=0 out_seq=0 2024-03-30T00:35:54.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f12db400700 1 -- 172.21.15.99:0/2767211171 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562204b53d70 con 0x562204e1ec30 2024-03-30T00:35:54.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f12db400700 10 client.15783 handle_command_reply: tid=0 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f1314590b80 1 client.15783 shutdown 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f1314590b80 1 -- 172.21.15.99:0/2767211171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 msgr2=0x562204dc3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f1314590b80 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 0x562204dc3740 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f12e800fa00 tx=0x7f12e8019040 comp rx=0 tx=0).stop 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f1314590b80 1 -- 172.21.15.99:0/2767211171 shutdown_connections 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f1314590b80 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562204e1ec30 0x562204e21050 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f1314590b80 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562204db5210 0x562204dc3740 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f1314590b80 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562204db3a60 0x562204dc4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f1314590b80 1 --2- 172.21.15.99:0/2767211171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562204db43c0 0x562204dc3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f1314590b80 1 -- 172.21.15.99:0/2767211171 >> 172.21.15.99:0/2767211171 conn(0x562204d12720 msgr2=0x562204dbd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.692+0000 7f12db400700 0 client.15783 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.693+0000 7f1314590b80 1 -- 172.21.15.99:0/2767211171 shutdown_connections 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.693+0000 7f1314590b80 1 -- 172.21.15.99:0/2767211171 wait complete. 2024-03-30T00:35:54.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.693+0000 7f1314590b80 20 client.15783 trim_cache size 0 max 16384 2024-03-30T00:35:54.701 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:35:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: "return_code": 0, 2024-03-30T00:35:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_tag": "84bceab3-f785-4e03-aa79-c4cb2c275237", 2024-03-30T00:35:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": "asynchronous" 2024-03-30T00:35:54.701 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:35:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 -- 172.21.15.99:0/763978889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f12e00a1120 msgr2=0x7f12e00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:54.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f12e00a1120 0x7f12e00a35e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f12f40090a0 tx=0x7f12f4095570 comp rx=0 tx=0).stop 2024-03-30T00:35:54.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 -- 172.21.15.99:0/763978889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc102750 msgr2=0x7f12fc0736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:54.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc102750 0x7f12fc0736e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12f400ef30 tx=0x7f12f400c450 comp rx=0 tx=0).stop 2024-03-30T00:35:54.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 -- 172.21.15.99:0/763978889 shutdown_connections 2024-03-30T00:35:54.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc102750 0x7f12fc0736e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f12e00a1120 0x7f12e00a35e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12fc100bc0 0x7f12fc074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 --2- 172.21.15.99:0/763978889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12fc101db0 0x7f12fc0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:54.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 -- 172.21.15.99:0/763978889 >> 172.21.15.99:0/763978889 conn(0x7f12fc0fc110 msgr2=0x7f12fc1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:54.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 -- 172.21.15.99:0/763978889 shutdown_connections 2024-03-30T00:35:54.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:54.694+0000 7f12fb200700 1 -- 172.21.15.99:0/763978889 wait complete. 2024-03-30T00:35:54.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T00:35:55.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.046+0000 7fa140200700 1 -- 172.21.15.99:0/2671777784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1381017e0 msgr2=0x7fa138103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:55.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.046+0000 7fa140200700 1 --2- 172.21.15.99:0/2671777784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1381017e0 0x7fa138103bd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009990 tx=0x7fa12c009ca0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.046+0000 7fa140200700 1 -- 172.21.15.99:0/2671777784 shutdown_connections 2024-03-30T00:35:55.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.046+0000 7fa140200700 1 --2- 172.21.15.99:0/2671777784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa138106a40 0x7fa138108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.046+0000 7fa140200700 1 --2- 172.21.15.99:0/2671777784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1381017e0 0x7fa138103bd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.046+0000 7fa140200700 1 --2- 172.21.15.99:0/2671777784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa138104110 0x7fa138106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.046+0000 7fa140200700 1 -- 172.21.15.99:0/2671777784 >> 172.21.15.99:0/2671777784 conn(0x7fa1380fb1d0 msgr2=0x7fa1380fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:55.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.046+0000 7fa140200700 1 -- 172.21.15.99:0/2671777784 shutdown_connections 2024-03-30T00:35:55.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.046+0000 7fa140200700 1 -- 172.21.15.99:0/2671777784 wait complete. 2024-03-30T00:35:55.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.048+0000 7fa140200700 1 Processor -- start 2024-03-30T00:35:55.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.048+0000 7fa140200700 1 -- start start 2024-03-30T00:35:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.049+0000 7fa140200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1381017e0 0x7fa138198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.049+0000 7fa140200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa138104110 0x7fa1381993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.049+0000 7fa140200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa138106a40 0x7fa1381a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.049+0000 7fa140200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1381a0990 con 0x7fa138104110 2024-03-30T00:35:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.049+0000 7fa140200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1381a0ad0 con 0x7fa1381017e0 2024-03-30T00:35:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.049+0000 7fa140200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1381a0c60 con 0x7fa138106a40 2024-03-30T00:35:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.049+0000 7fa13d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1381017e0 0x7fa138198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.049+0000 7fa13d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1381017e0 0x7fa138198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:46946/0 (socket says 172.21.15.99:46946) 2024-03-30T00:35:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.049+0000 7fa13d800700 1 -- 172.21.15.99:0/2562707 learned_addr learned my addr 172.21.15.99:0/2562707 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.049+0000 7fa13e200700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa138106a40 0x7fa1381a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.050+0000 7fa13d800700 1 -- 172.21.15.99:0/2562707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa138106a40 msgr2=0x7fa1381a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:55.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.050+0000 7fa13d800700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa138106a40 0x7fa1381a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.050+0000 7fa13d800700 1 -- 172.21.15.99:0/2562707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa138104110 msgr2=0x7fa1381993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:55.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.050+0000 7fa13d800700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa138104110 0x7fa1381993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.050+0000 7fa13d800700 1 -- 172.21.15.99:0/2562707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c0095f0 con 0x7fa1381017e0 2024-03-30T00:35:55.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.050+0000 7fa13d800700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1381017e0 0x7fa138198e60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa12c005950 tx=0x7fa12c003aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:55.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.050+0000 7fa136000700 1 -- 172.21.15.99:0/2562707 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c01d070 con 0x7fa1381017e0 2024-03-30T00:35:55.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.050+0000 7fa136000700 1 -- 172.21.15.99:0/2562707 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa12c004830 con 0x7fa1381017e0 2024-03-30T00:35:55.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.050+0000 7fa136000700 1 -- 172.21.15.99:0/2562707 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0053a0 con 0x7fa1381017e0 2024-03-30T00:35:55.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.050+0000 7fa140200700 1 -- 172.21.15.99:0/2562707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa138191de0 con 0x7fa1381017e0 2024-03-30T00:35:55.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.051+0000 7fa140200700 1 -- 172.21.15.99:0/2562707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1381921e0 con 0x7fa1381017e0 2024-03-30T00:35:55.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.055+0000 7fa136000700 1 -- 172.21.15.99:0/2562707 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa12c005500 con 0x7fa1381017e0 2024-03-30T00:35:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.057+0000 7fa136000700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa1200a11a0 0x7fa1200a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.057+0000 7fa153d36b80 1 Processor -- start 2024-03-30T00:35:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.058+0000 7fa136000700 1 -- 172.21.15.99:0/2562707 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fa12c0cdcb0 con 0x7fa1381017e0 2024-03-30T00:35:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.058+0000 7fa153d36b80 1 -- start start 2024-03-30T00:35:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.058+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 0x55f046359bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.058+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f04635a130 0x55f04635a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.058+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f04635b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.058+0000 7fa153d36b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0462b1990 con 0x55f04635afd0 2024-03-30T00:35:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.058+0000 7fa153d36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0462af350 con 0x55f04635a130 2024-03-30T00:35:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.058+0000 7fa153d36b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0462aac40 con 0x55f0463597d0 2024-03-30T00:35:55.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.059+0000 7fa13ce00700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa1200a11a0 0x7fa1200a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.059+0000 7fa13e200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f04635b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.059+0000 7fa13e200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f04635b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34236/0 (socket says 172.21.15.99:34236) 2024-03-30T00:35:55.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.059+0000 7fa13e200700 1 -- 172.21.15.99:0/731571850 learned_addr learned my addr 172.21.15.99:0/731571850 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:55.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.059+0000 7fa13d800700 1 --2- 172.21.15.99:0/731571850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 0x55f046359bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.059+0000 7fa13ce00700 1 --2- 172.21.15.99:0/731571850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f04635a130 0x55f04635a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.059+0000 7fa13e200700 1 -- 172.21.15.99:0/731571850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 msgr2=0x55f046359bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.059+0000 7fa13e200700 1 --2- 172.21.15.99:0/731571850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 0x55f046359bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.059+0000 7fa13e200700 1 -- 172.21.15.99:0/731571850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f04635a130 msgr2=0x55f04635a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.059+0000 7fa13e200700 1 --2- 172.21.15.99:0/731571850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f04635a130 0x55f04635a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.059+0000 7fa13e200700 1 -- 172.21.15.99:0/731571850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f04612c3b0 con 0x55f04635afd0 2024-03-30T00:35:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.060+0000 7fa13d800700 1 --2- 172.21.15.99:0/731571850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 0x55f046359bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.060+0000 7fa13e200700 1 --2- 172.21.15.99:0/731571850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f04635b450 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa130009440 tx=0x7fa130009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41129a1d546b4b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.060+0000 7fa13ce00700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa1200a11a0 0x7fa1200a3660 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa124005fd0 tx=0x7fa1240147a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.060+0000 7fa12aa00700 1 -- 172.21.15.99:0/731571850 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13000e070 con 0x55f04635afd0 2024-03-30T00:35:55.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.060+0000 7fa12aa00700 1 -- 172.21.15.99:0/731571850 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa13000f040 con 0x55f04635afd0 2024-03-30T00:35:55.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.060+0000 7fa12aa00700 1 -- 172.21.15.99:0/731571850 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130015a80 con 0x55f04635afd0 2024-03-30T00:35:55.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.061+0000 7fa153d36b80 1 -- 172.21.15.99:0/731571850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 msgr2=0x55f04635b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:55.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.061+0000 7fa153d36b80 1 --2- 172.21.15.99:0/731571850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f04635b450 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa130009440 tx=0x7fa130009920 comp rx=0 tx=0).stop 2024-03-30T00:35:55.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.062+0000 7fa153d36b80 1 -- 172.21.15.99:0/731571850 shutdown_connections 2024-03-30T00:35:55.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.062+0000 7fa153d36b80 1 --2- 172.21.15.99:0/731571850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f04635b450 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.062+0000 7fa153d36b80 1 --2- 172.21.15.99:0/731571850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 0x55f046359bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.062+0000 7fa153d36b80 1 --2- 172.21.15.99:0/731571850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f04635a130 0x55f04635a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.062+0000 7fa153d36b80 1 -- 172.21.15.99:0/731571850 >> 172.21.15.99:0/731571850 conn(0x55f0462b86a0 msgr2=0x55f0463549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:55.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.062+0000 7fa153d36b80 1 -- 172.21.15.99:0/731571850 shutdown_connections 2024-03-30T00:35:55.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.062+0000 7fa153d36b80 1 -- 172.21.15.99:0/731571850 wait complete. 2024-03-30T00:35:55.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.063+0000 7fa153d36b80 1 -- start start 2024-03-30T00:35:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.063+0000 7fa153d36b80 1 Processor -- start 2024-03-30T00:35:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.064+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 0x55f046363650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.064+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f04635a130 0x55f046363b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.064+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f04636abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.064+0000 7fa153d36b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f046246d80 con 0x55f04635afd0 2024-03-30T00:35:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.064+0000 7fa153d36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f046246a20 con 0x55f04635a130 2024-03-30T00:35:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.064+0000 7fa153d36b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04612e6a0 con 0x55f0463597d0 2024-03-30T00:35:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.064+0000 7fa153d36b80 10 client.0 _ll_register_callbacks cb 0x55f046250470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.064+0000 7fa13d800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 0x55f046363650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.064+0000 7fa13d800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 0x55f046363650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:38668/0 (socket says 172.21.15.99:38668) 2024-03-30T00:35:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.064+0000 7fa13d800700 1 -- 172.21.15.99:0/1824484951 learned_addr learned my addr 172.21.15.99:0/1824484951 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.064+0000 7fa13e200700 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f04636abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.065+0000 7fa13d800700 1 -- 172.21.15.99:0/1824484951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f04635a130 msgr2=0x55f046363b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:35:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.065+0000 7fa13d800700 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f04635a130 0x55f046363b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.065+0000 7fa13d800700 1 -- 172.21.15.99:0/1824484951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 msgr2=0x55f04636abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.065+0000 7fa13d800700 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f04636abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.065+0000 7fa13d800700 1 -- 172.21.15.99:0/1824484951 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1381921e0 con 0x55f0463597d0 2024-03-30T00:35:55.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.065+0000 7fa13e200700 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f04636abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:35:55.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.066+0000 7fa13d800700 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 0x55f046363650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00b2e0 tx=0x7fa12c0a1330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=395cb5b385c81e86 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:55.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.066+0000 7fa11f400700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:35:55.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.066+0000 7fa11f400700 1 -- 172.21.15.99:0/1824484951 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c09b550 con 0x55f0463597d0 2024-03-30T00:35:55.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.066+0000 7fa11f400700 1 -- 172.21.15.99:0/1824484951 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fa12c09fe40 con 0x55f0463597d0 2024-03-30T00:35:55.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.066+0000 7fa153d36b80 1 -- 172.21.15.99:0/1824484951 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f04626b380 con 0x55f0463597d0 2024-03-30T00:35:55.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.066+0000 7fa11f400700 1 -- 172.21.15.99:0/1824484951 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa12c028e30 con 0x55f0463597d0 2024-03-30T00:35:55.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.067+0000 7fa153d36b80 10 client.25513 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:55.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.067+0000 7fa153d36b80 1 -- 172.21.15.99:0/1824484951 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fa1300090f0 con 0x55f0463597d0 2024-03-30T00:35:55.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.067+0000 7fa11f400700 1 -- 172.21.15.99:0/1824484951 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f04626b380 con 0x55f0463597d0 2024-03-30T00:35:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.067+0000 7fa11f400700 1 -- 172.21.15.99:0/1824484951 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa12c09a5e0 con 0x55f0463597d0 2024-03-30T00:35:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.067+0000 7fa153d36b80 10 client.25513 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.067+0000 7fa153d36b80 10 client.25513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.067+0000 7fa153d36b80 20 client.25513 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.067+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f04636f090 0x55f0463714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.067+0000 7fa153d36b80 4 client.25513 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:35:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.067+0000 7fa153d36b80 1 -- 172.21.15.99:0/1824484951 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0462a3700 con 0x55f04636f090 2024-03-30T00:35:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.068+0000 7fa13ce00700 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f04636f090 0x55f0463714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.069+0000 7fa13ce00700 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f04636f090 0x55f0463714b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5741388398c7009c server_cookie=5e8960169fe2f338 in_seq=0 out_seq=0 2024-03-30T00:35:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.069+0000 7fa11f400700 10 client.25513 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.069+0000 7fa11f400700 1 -- 172.21.15.99:0/1824484951 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f0462a3700 con 0x55f04636f090 2024-03-30T00:35:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.069+0000 7fa11f400700 10 client.25513 handle_command_reply: tid=0 2024-03-30T00:35:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa153d36b80 1 client.25513 shutdown 2024-03-30T00:35:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa153d36b80 1 -- 172.21.15.99:0/1824484951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 msgr2=0x55f046363650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 0x55f046363650 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00b2e0 tx=0x7fa12c0a1330 comp rx=0 tx=0).stop 2024-03-30T00:35:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa153d36b80 1 -- 172.21.15.99:0/1824484951 shutdown_connections 2024-03-30T00:35:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f04636f090 0x55f0463714b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f04636abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0463597d0 0x55f046363650 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1824484951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f04635a130 0x55f046363b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa153d36b80 1 -- 172.21.15.99:0/1824484951 >> 172.21.15.99:0/1824484951 conn(0x55f0462b86a0 msgr2=0x55f046358170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa11f400700 0 client.25513 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa153d36b80 1 -- 172.21.15.99:0/1824484951 shutdown_connections 2024-03-30T00:35:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.070+0000 7fa153d36b80 1 -- 172.21.15.99:0/1824484951 wait complete. 2024-03-30T00:35:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.071+0000 7fa153d36b80 20 client.25513 trim_cache size 0 max 16384 2024-03-30T00:35:55.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.103+0000 7fa153d36b80 1 Processor -- start 2024-03-30T00:35:55.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.103+0000 7fa153d36b80 1 -- start start 2024-03-30T00:35:55.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.103+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 0x55f046362b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.103+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635a130 0x55f04636ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.103+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f0463614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.104+0000 7fa153d36b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04626b380 con 0x55f04635afd0 2024-03-30T00:35:55.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.104+0000 7fa153d36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0463554b0 con 0x55f0463597d0 2024-03-30T00:35:55.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.104+0000 7fa153d36b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f046359370 con 0x55f04635a130 2024-03-30T00:35:55.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.104+0000 7fa13ce00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635a130 0x55f04636ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.104+0000 7fa13ce00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635a130 0x55f04636ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:38682/0 (socket says 172.21.15.99:38682) 2024-03-30T00:35:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.104+0000 7fa13ce00700 1 -- 172.21.15.99:0/4189970723 learned_addr learned my addr 172.21.15.99:0/4189970723 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.104+0000 7fa13e200700 1 --2- 172.21.15.99:0/4189970723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f0463614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.104+0000 7fa13d800700 1 --2- 172.21.15.99:0/4189970723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 0x55f046362b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.104+0000 7fa13ce00700 1 -- 172.21.15.99:0/4189970723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 msgr2=0x55f046362b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.105+0000 7fa13ce00700 1 --2- 172.21.15.99:0/4189970723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 0x55f046362b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.105+0000 7fa13ce00700 1 -- 172.21.15.99:0/4189970723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 msgr2=0x55f0463614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:55.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.105+0000 7fa13ce00700 1 --2- 172.21.15.99:0/4189970723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f0463614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.105+0000 7fa13ce00700 1 -- 172.21.15.99:0/4189970723 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0460d8b60 con 0x55f04635a130 2024-03-30T00:35:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.105+0000 7fa13d800700 1 --2- 172.21.15.99:0/4189970723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 0x55f046362b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.105+0000 7fa13e200700 1 --2- 172.21.15.99:0/4189970723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f0463614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:35:55.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.105+0000 7fa13ce00700 1 --2- 172.21.15.99:0/4189970723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635a130 0x55f04636ab90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa124009970 tx=0x7fa124017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b551c7d5c2e90d0a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.106+0000 7fa12b400700 1 -- 172.21.15.99:0/4189970723 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124009240 con 0x55f04635a130 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.106+0000 7fa12b400700 1 -- 172.21.15.99:0/4189970723 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa124023de0 con 0x55f04635a130 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.106+0000 7fa12b400700 1 -- 172.21.15.99:0/4189970723 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124012da0 con 0x55f04635a130 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.107+0000 7fa153d36b80 1 -- 172.21.15.99:0/4189970723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635a130 msgr2=0x55f04636ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.107+0000 7fa153d36b80 1 --2- 172.21.15.99:0/4189970723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635a130 0x55f04636ab90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa124009970 tx=0x7fa124017000 comp rx=0 tx=0).stop 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.107+0000 7fa153d36b80 1 -- 172.21.15.99:0/4189970723 shutdown_connections 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.107+0000 7fa153d36b80 1 --2- 172.21.15.99:0/4189970723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635afd0 0x55f0463614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.107+0000 7fa153d36b80 1 --2- 172.21.15.99:0/4189970723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 0x55f046362b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.107+0000 7fa153d36b80 1 --2- 172.21.15.99:0/4189970723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635a130 0x55f04636ab90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.107+0000 7fa153d36b80 1 -- 172.21.15.99:0/4189970723 >> 172.21.15.99:0/4189970723 conn(0x55f0462b86a0 msgr2=0x55f04634d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.107+0000 7fa153d36b80 1 -- 172.21.15.99:0/4189970723 shutdown_connections 2024-03-30T00:35:55.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.107+0000 7fa153d36b80 1 -- 172.21.15.99:0/4189970723 wait complete. 2024-03-30T00:35:55.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.108+0000 7fa153d36b80 1 -- start start 2024-03-30T00:35:55.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.108+0000 7fa153d36b80 1 Processor -- start 2024-03-30T00:35:55.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.109+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 0x55f046368940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.109+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635a130 0x55f046368e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.109+0000 7fa153d36b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635afd0 0x55f046369d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.109+0000 7fa153d36b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04636a290 con 0x55f04635a130 2024-03-30T00:35:55.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.109+0000 7fa153d36b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04636a400 con 0x55f0463597d0 2024-03-30T00:35:55.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.109+0000 7fa153d36b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f04636a5c0 con 0x55f04635afd0 2024-03-30T00:35:55.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.110+0000 7fa153d36b80 10 client.0 _ll_register_callbacks cb 0x55f046250470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:35:55.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.110+0000 7fa13d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 0x55f046368940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.110+0000 7fa13ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635a130 0x55f046368e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.110+0000 7fa13d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 0x55f046368940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:46968/0 (socket says 172.21.15.99:46968) 2024-03-30T00:35:55.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.110+0000 7fa13d800700 1 -- 172.21.15.99:0/1731068285 learned_addr learned my addr 172.21.15.99:0/1731068285 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:35:55.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.110+0000 7fa13ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635a130 0x55f046368e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34270/0 (socket says 172.21.15.99:34270) 2024-03-30T00:35:55.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.110+0000 7fa13e200700 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635afd0 0x55f046369d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.110+0000 7fa13d800700 1 -- 172.21.15.99:0/1731068285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635afd0 msgr2=0x55f046369d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.110+0000 7fa13d800700 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635afd0 0x55f046369d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.111+0000 7fa13d800700 1 -- 172.21.15.99:0/1731068285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635a130 msgr2=0x55f046368e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.111+0000 7fa13d800700 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635a130 0x55f046368e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.111+0000 7fa13d800700 1 -- 172.21.15.99:0/1731068285 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12401a070 con 0x55f0463597d0 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.111+0000 7fa13d800700 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 0x55f046368940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0098e0 tx=0x7fa12c09aff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78792aeb53feae65 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.111+0000 7fa11f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.111+0000 7fa11f400700 1 -- 172.21.15.99:0/1731068285 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c09b550 con 0x55f0463597d0 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.111+0000 7fa11f400700 1 -- 172.21.15.99:0/1731068285 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fa12c09fe40 con 0x55f0463597d0 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.111+0000 7fa153d36b80 1 -- 172.21.15.99:0/1731068285 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa13000f040 con 0x55f0463597d0 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.112+0000 7fa11f400700 1 -- 172.21.15.99:0/1731068285 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa12c0b9220 con 0x55f0463597d0 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.112+0000 7fa153d36b80 10 client.25505 fetch_fsmap learned FSMap version 24 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.112+0000 7fa153d36b80 1 -- 172.21.15.99:0/1731068285 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f046270ea0 con 0x55f0463597d0 2024-03-30T00:35:55.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.112+0000 7fa11f400700 1 -- 172.21.15.99:0/1731068285 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa12c082030 con 0x55f0463597d0 2024-03-30T00:35:55.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.113+0000 7fa11f400700 1 -- 172.21.15.99:0/1731068285 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa12c0a3050 con 0x55f0463597d0 2024-03-30T00:35:55.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.113+0000 7fa153d36b80 10 client.25505 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:35:55.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.113+0000 7fa153d36b80 10 client.25505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:35:55.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.113+0000 7fa153d36b80 20 client.25505 populate_metadata read hostname 'smithi099' 2024-03-30T00:35:55.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.113+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f0463c3870 0x55f0463c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:35:55.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.113+0000 7fa153d36b80 4 client.25505 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:35:55.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.113+0000 7fa153d36b80 1 -- 172.21.15.99:0/1731068285 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f0460f9cf0 con 0x55f0463c3870 2024-03-30T00:35:55.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.113+0000 7fa13ce00700 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f0463c3870 0x55f0463c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:35:55.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.114+0000 7fa13ce00700 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f0463c3870 0x55f0463c5c60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ae35fc45fbd0299 server_cookie=4850f15252e4087a in_seq=0 out_seq=0 2024-03-30T00:35:55.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.114+0000 7fa11f400700 10 client.25505 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.114+0000 7fa11f400700 1 -- 172.21.15.99:0/1731068285 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f0460f9cf0 con 0x55f0463c3870 2024-03-30T00:35:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.114+0000 7fa11f400700 10 client.25505 handle_command_reply: tid=0 2024-03-30T00:35:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.114+0000 7fa153d36b80 1 client.25505 shutdown 2024-03-30T00:35:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.115+0000 7fa153d36b80 1 -- 172.21.15.99:0/1731068285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 msgr2=0x55f046368940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.115+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 0x55f046368940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0098e0 tx=0x7fa12c09aff0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.115+0000 7fa153d36b80 1 -- 172.21.15.99:0/1731068285 shutdown_connections 2024-03-30T00:35:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.115+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f0463c3870 0x55f0463c5c60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.115+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f04635afd0 0x55f046369d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.115+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0463597d0 0x55f046368940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.115+0000 7fa153d36b80 1 --2- 172.21.15.99:0/1731068285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f04635a130 0x55f046368e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.115+0000 7fa153d36b80 1 -- 172.21.15.99:0/1731068285 >> 172.21.15.99:0/1731068285 conn(0x55f0462b86a0 msgr2=0x55f046362b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:55.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.115+0000 7fa11f400700 0 client.25505 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:35:55.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.115+0000 7fa153d36b80 1 -- 172.21.15.99:0/1731068285 shutdown_connections 2024-03-30T00:35:55.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.115+0000 7fa153d36b80 1 -- 172.21.15.99:0/1731068285 wait complete. 2024-03-30T00:35:55.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.116+0000 7fa153d36b80 20 client.25505 trim_cache size 0 max 16384 2024-03-30T00:35:55.123 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:35:55.124 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:35:55.124 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": { 2024-03-30T00:35:55.124 INFO:teuthology.orchestra.run.smithi099.stdout: "84bceab3-f785-4e03-aa79-c4cb2c275237": { 2024-03-30T00:35:55.124 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/", 2024-03-30T00:35:55.124 INFO:teuthology.orchestra.run.smithi099.stdout: "tag": "84bceab3-f785-4e03-aa79-c4cb2c275237", 2024-03-30T00:35:55.124 INFO:teuthology.orchestra.run.smithi099.stdout: "options": "recursive,force" 2024-03-30T00:35:55.124 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:35:55.124 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:35:55.124 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.116+0000 7fa140200700 1 -- 172.21.15.99:0/2562707 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa1200a11a0 msgr2=0x7fa1200a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.116+0000 7fa140200700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa1200a11a0 0x7fa1200a3660 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa124005fd0 tx=0x7fa1240147a0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.116+0000 7fa140200700 1 -- 172.21.15.99:0/2562707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1381017e0 msgr2=0x7fa138198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.116+0000 7fa140200700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1381017e0 0x7fa138198e60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa12c005950 tx=0x7fa12c003aa0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.117+0000 7fa140200700 1 -- 172.21.15.99:0/2562707 shutdown_connections 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.117+0000 7fa140200700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa138106a40 0x7fa1381a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.117+0000 7fa140200700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa1200a11a0 0x7fa1200a3660 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.117+0000 7fa140200700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1381017e0 0x7fa138198e60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.117+0000 7fa140200700 1 --2- 172.21.15.99:0/2562707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa138104110 0x7fa1381993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.117+0000 7fa140200700 1 -- 172.21.15.99:0/2562707 >> 172.21.15.99:0/2562707 conn(0x7fa1380fb1d0 msgr2=0x7fa1380ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.117+0000 7fa140200700 1 -- 172.21.15.99:0/2562707 shutdown_connections 2024-03-30T00:35:55.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:35:55.117+0000 7fa140200700 1 -- 172.21.15.99:0/2562707 wait complete. 2024-03-30T00:35:55.152 INFO:tasks.cephfs.filesystem:scrub status for tag:84bceab3-f785-4e03-aa79-c4cb2c275237 - {'path': '/', 'tag': '84bceab3-f785-4e03-aa79-c4cb2c275237', 'options': 'recursive,force'} 2024-03-30T00:35:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:56 smithi161 ceph-mon[42221]: scrub queued for path: / 2024-03-30T00:35:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:56 smithi161 ceph-mon[42221]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:56 smithi161 ceph-mon[42221]: scrub summary: active paths [/] 2024-03-30T00:35:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:56 smithi161 ceph-mon[42221]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:56.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:56 smithi161 ceph-mon[42221]: pgmap v340: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:56.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:56 smithi099 ceph-mon[38079]: scrub queued for path: / 2024-03-30T00:35:56.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:56 smithi099 ceph-mon[38079]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:56.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:56 smithi099 ceph-mon[38079]: scrub summary: active paths [/] 2024-03-30T00:35:56.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:56 smithi099 ceph-mon[38079]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:56.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:56 smithi099 ceph-mon[38079]: pgmap v340: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:56.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:56 smithi142 ceph-mon[41779]: scrub queued for path: / 2024-03-30T00:35:56.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:56 smithi142 ceph-mon[41779]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:56.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:56 smithi142 ceph-mon[41779]: scrub summary: active paths [/] 2024-03-30T00:35:56.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:56 smithi142 ceph-mon[41779]: scrub summary: idle+waiting paths [/] 2024-03-30T00:35:56.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:56 smithi142 ceph-mon[41779]: pgmap v340: 97 pgs: 97 active+clean; 663 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:35:58.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:58 smithi161 ceph-mon[42221]: pgmap v341: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-30T00:35:58.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:58 smithi099 ceph-mon[38079]: pgmap v341: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-30T00:35:58.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:58 smithi142 ceph-mon[41779]: pgmap v341: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-30T00:35:59.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:35:59 smithi161 ceph-mon[42221]: scrub summary: idle 2024-03-30T00:35:59.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:35:59 smithi099 ceph-mon[38079]: scrub summary: idle 2024-03-30T00:35:59.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:35:59 smithi142 ceph-mon[41779]: scrub summary: idle 2024-03-30T00:36:00.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:00 smithi161 ceph-mon[42221]: pgmap v342: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:36:00.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:00 smithi099 ceph-mon[38079]: pgmap v342: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:36:00.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:00 smithi142 ceph-mon[41779]: pgmap v342: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:36:02.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:02 smithi161 ceph-mon[42221]: pgmap v343: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T00:36:02.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:02 smithi099 ceph-mon[38079]: pgmap v343: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T00:36:02.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:02 smithi142 ceph-mon[41779]: pgmap v343: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T00:36:04.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:04 smithi161 ceph-mon[42221]: pgmap v344: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:36:04.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:04 smithi099 ceph-mon[38079]: pgmap v344: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:36:04.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:04 smithi142 ceph-mon[41779]: pgmap v344: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-30T00:36:06.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:06 smithi161 ceph-mon[42221]: pgmap v345: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:36:06.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:06 smithi099 ceph-mon[38079]: pgmap v345: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:36:06.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:06 smithi142 ceph-mon[41779]: pgmap v345: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.040+0000 7fb722c00700 1 -- 172.21.15.99:0/724777303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c108bd0 msgr2=0x7fb71c10b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.040+0000 7fb722c00700 1 --2- 172.21.15.99:0/724777303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c108bd0 0x7fb71c10b060 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb71400b0a0 tx=0x7fb71400b3b0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.040+0000 7fb722c00700 1 -- 172.21.15.99:0/724777303 shutdown_connections 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.040+0000 7fb722c00700 1 --2- 172.21.15.99:0/724777303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c108bd0 0x7fb71c10b060 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.040+0000 7fb722c00700 1 --2- 172.21.15.99:0/724777303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb71c074df0 0x7fb71c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.040+0000 7fb722c00700 1 --2- 172.21.15.99:0/724777303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb71c073810 0x7fb71c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.040+0000 7fb722c00700 1 -- 172.21.15.99:0/724777303 >> 172.21.15.99:0/724777303 conn(0x7fb71c0fc280 msgr2=0x7fb71c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.040+0000 7fb722c00700 1 -- 172.21.15.99:0/724777303 shutdown_connections 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.040+0000 7fb722c00700 1 -- 172.21.15.99:0/724777303 wait complete. 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.041+0000 7fb722c00700 1 Processor -- start 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.041+0000 7fb722c00700 1 -- start start 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.041+0000 7fb722c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb71c073810 0x7fb71c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.041+0000 7fb722c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c074df0 0x7fb71c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.041+0000 7fb722c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb71c108bd0 0x7fb71c1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.041+0000 7fb722c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c1a0a00 con 0x7fb71c074df0 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.041+0000 7fb722c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c1a0b40 con 0x7fb71c108bd0 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.041+0000 7fb722c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c1a0cd0 con 0x7fb71c073810 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb71b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c074df0 0x7fb71c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb71b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c074df0 0x7fb71c199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52232/0 (socket says 172.21.15.99:52232) 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb71b400700 1 -- 172.21.15.99:0/4288329426 learned_addr learned my addr 172.21.15.99:0/4288329426 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb720c00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb71c108bd0 0x7fb71c1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb71be00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb71c073810 0x7fb71c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb71b400700 1 -- 172.21.15.99:0/4288329426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb71c073810 msgr2=0x7fb71c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb71b400700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb71c073810 0x7fb71c198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb71b400700 1 -- 172.21.15.99:0/4288329426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb71c108bd0 msgr2=0x7fb71c1a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb71b400700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb71c108bd0 0x7fb71c1a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb71b400700 1 -- 172.21.15.99:0/4288329426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb714009d00 con 0x7fb71c074df0 2024-03-30T00:36:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb71be00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb71c073810 0x7fb71c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.042+0000 7fb71b400700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c074df0 0x7fb71c199460 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb70800d980 tx=0x7fb70800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.043+0000 7fb718c00700 1 -- 172.21.15.99:0/4288329426 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708004020 con 0x7fb71c074df0 2024-03-30T00:36:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.043+0000 7fb718c00700 1 -- 172.21.15.99:0/4288329426 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb708009730 con 0x7fb71c074df0 2024-03-30T00:36:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.043+0000 7fb722c00700 1 -- 172.21.15.99:0/4288329426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71c1a0fb0 con 0x7fb71c074df0 2024-03-30T00:36:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.043+0000 7fb718c00700 1 -- 172.21.15.99:0/4288329426 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708003a00 con 0x7fb71c074df0 2024-03-30T00:36:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.043+0000 7fb722c00700 1 -- 172.21.15.99:0/4288329426 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71c1a1430 con 0x7fb71c074df0 2024-03-30T00:36:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.046+0000 7fb718c00700 1 -- 172.21.15.99:0/4288329426 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fb7080098a0 con 0x7fb71c074df0 2024-03-30T00:36:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.047+0000 7fb718c00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7000a1120 0x7fb7000a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.047+0000 7fb718c00700 1 -- 172.21.15.99:0/4288329426 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fb7080cc660 con 0x7fb71c074df0 2024-03-30T00:36:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.047+0000 7fb73678fb80 1 Processor -- start 2024-03-30T00:36:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.047+0000 7fb71be00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7000a1120 0x7fb7000a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.047+0000 7fb73678fb80 1 -- start start 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6200 0x55ab7d7a6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7a7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb73678fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d6fe930 con 0x55ab7d7a6200 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb73678fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d6fc2f0 con 0x55ab7d7a7a60 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb73678fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d6f7be0 con 0x55ab7d7a6b60 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb71be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6200 0x55ab7d7a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb71be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6200 0x55ab7d7a6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52248/0 (socket says 172.21.15.99:52248) 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb71be00700 1 -- 172.21.15.99:0/1996971917 learned_addr learned my addr 172.21.15.99:0/1996971917 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb71b400700 1 --2- 172.21.15.99:0/1996971917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb71be00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7000a1120 0x7fb7000a35e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb70c005950 tx=0x7fb70c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.048+0000 7fb720c00700 1 --2- 172.21.15.99:0/1996971917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7a7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.049+0000 7fb71be00700 1 -- 172.21.15.99:0/1996971917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6b60 msgr2=0x55ab7d7a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.049+0000 7fb71be00700 1 --2- 172.21.15.99:0/1996971917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.049+0000 7fb71be00700 1 -- 172.21.15.99:0/1996971917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 msgr2=0x55ab7d7a7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.049+0000 7fb71be00700 1 --2- 172.21.15.99:0/1996971917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7a7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.049+0000 7fb71be00700 1 -- 172.21.15.99:0/1996971917 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab7d579350 con 0x55ab7d7a6200 2024-03-30T00:36:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.049+0000 7fb720c00700 1 --2- 172.21.15.99:0/1996971917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7a7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.049+0000 7fb71be00700 1 --2- 172.21.15.99:0/1996971917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6200 0x55ab7d7a6620 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00ff00 tx=0x7fb70c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f9e096862bb852 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.049+0000 7fb711200700 1 -- 172.21.15.99:0/1996971917 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c011bb0 con 0x55ab7d7a6200 2024-03-30T00:36:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.049+0000 7fb711200700 1 -- 172.21.15.99:0/1996971917 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb70c011d10 con 0x55ab7d7a6200 2024-03-30T00:36:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.049+0000 7fb711200700 1 -- 172.21.15.99:0/1996971917 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c010630 con 0x55ab7d7a6200 2024-03-30T00:36:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.050+0000 7fb73678fb80 1 -- 172.21.15.99:0/1996971917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6200 msgr2=0x55ab7d7a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.050+0000 7fb73678fb80 1 --2- 172.21.15.99:0/1996971917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6200 0x55ab7d7a6620 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00ff00 tx=0x7fb70c012000 comp rx=0 tx=0).stop 2024-03-30T00:36:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.050+0000 7fb73678fb80 1 -- 172.21.15.99:0/1996971917 shutdown_connections 2024-03-30T00:36:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.050+0000 7fb73678fb80 1 --2- 172.21.15.99:0/1996971917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7a7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.050+0000 7fb73678fb80 1 --2- 172.21.15.99:0/1996971917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6200 0x55ab7d7a6620 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.050+0000 7fb73678fb80 1 --2- 172.21.15.99:0/1996971917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.050+0000 7fb73678fb80 1 -- 172.21.15.99:0/1996971917 >> 172.21.15.99:0/1996971917 conn(0x55ab7d705640 msgr2=0x55ab7d7a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.050+0000 7fb73678fb80 1 -- 172.21.15.99:0/1996971917 shutdown_connections 2024-03-30T00:36:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.050+0000 7fb73678fb80 1 -- 172.21.15.99:0/1996971917 wait complete. 2024-03-30T00:36:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.051+0000 7fb73678fb80 1 -- start start 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.051+0000 7fb73678fb80 1 Processor -- start 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.051+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 0x55ab7d7b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.051+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.051+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.051+0000 7fb73678fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d693d20 con 0x55ab7d7a6b60 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.051+0000 7fb73678fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d6939c0 con 0x55ab7d7a7a60 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.051+0000 7fb73678fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d57b640 con 0x55ab7d7a6200 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.051+0000 7fb73678fb80 10 client.0 _ll_register_callbacks cb 0x55ab7d69d410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 0x55ab7d7b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 0x55ab7d7b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:58522/0 (socket says 172.21.15.99:58522) 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71be00700 1 -- 172.21.15.99:0/3060461244 learned_addr learned my addr 172.21.15.99:0/3060461244 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71b400700 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52256/0 (socket says 172.21.15.99:52256) 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb720c00700 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71b400700 1 -- 172.21.15.99:0/3060461244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 msgr2=0x55ab7d7b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71b400700 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 0x55ab7d7b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71b400700 1 -- 172.21.15.99:0/3060461244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 msgr2=0x55ab7d7b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71b400700 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71b400700 1 -- 172.21.15.99:0/3060461244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c00fbe0 con 0x55ab7d7a6b60 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb720c00700 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71be00700 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 0x55ab7d7b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb71b400700 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b0690 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb7080996d0 tx=0x7fb708009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590d8cb0aceaa01c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.052+0000 7fb705600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb705600700 1 -- 172.21.15.99:0/3060461244 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7080b2370 con 0x55ab7d7a6b60 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb705600700 1 -- 172.21.15.99:0/3060461244 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fb70809d800 con 0x55ab7d7a6b60 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb73678fb80 1 -- 172.21.15.99:0/3060461244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab7d6b8320 con 0x55ab7d7a6b60 2024-03-30T00:36:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb705600700 1 -- 172.21.15.99:0/3060461244 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7080a1e50 con 0x55ab7d7a6b60 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb73678fb80 10 client.15810 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb73678fb80 1 -- 172.21.15.99:0/3060461244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ab7d525b00 con 0x55ab7d7a6b60 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb705600700 1 -- 172.21.15.99:0/3060461244 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ab7d6b8320 con 0x55ab7d7a6b60 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb705600700 1 -- 172.21.15.99:0/3060461244 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb7080a19c0 con 0x55ab7d7a6b60 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb73678fb80 10 client.15810 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb73678fb80 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb73678fb80 20 client.15810 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ab7d7bbf80 0x55ab7d7be3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb73678fb80 4 client.15810 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.053+0000 7fb73678fb80 1 -- 172.21.15.99:0/3060461244 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab7d6f06a0 con 0x55ab7d7bbf80 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.054+0000 7fb71be00700 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ab7d7bbf80 0x55ab7d7be3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.055+0000 7fb71be00700 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ab7d7bbf80 0x55ab7d7be3a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e7e29433fce74e server_cookie=e8717a8cc28712ad in_seq=0 out_seq=0 2024-03-30T00:36:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.055+0000 7fb705600700 10 client.15810 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.055+0000 7fb705600700 1 -- 172.21.15.99:0/3060461244 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ab7d6f06a0 con 0x55ab7d7bbf80 2024-03-30T00:36:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.055+0000 7fb705600700 10 client.15810 handle_command_reply: tid=0 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.055+0000 7fb73678fb80 1 client.15810 shutdown 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb73678fb80 1 -- 172.21.15.99:0/3060461244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 msgr2=0x55ab7d7b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b0690 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb7080996d0 tx=0x7fb708009c50 comp rx=0 tx=0).stop 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb73678fb80 1 -- 172.21.15.99:0/3060461244 shutdown_connections 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ab7d7bbf80 0x55ab7d7be3a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 0x55ab7d7b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb705600700 0 client.15810 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3060461244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b0690 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb73678fb80 1 -- 172.21.15.99:0/3060461244 >> 172.21.15.99:0/3060461244 conn(0x55ab7d705640 msgr2=0x55ab7d7a4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb73678fb80 1 -- 172.21.15.99:0/3060461244 shutdown_connections 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb73678fb80 1 -- 172.21.15.99:0/3060461244 wait complete. 2024-03-30T00:36:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.056+0000 7fb73678fb80 20 client.15810 trim_cache size 0 max 16384 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.071+0000 7fb73678fb80 1 Processor -- start 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.071+0000 7fb73678fb80 1 -- start start 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.071+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 0x55ab7d7af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.071+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.071+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.071+0000 7fb73678fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d76c780 con 0x55ab7d7a7a60 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.071+0000 7fb73678fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d7b9460 con 0x55ab7d7a6b60 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.071+0000 7fb73678fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d7b12d0 con 0x55ab7d7a6200 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.072+0000 7fb720c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.072+0000 7fb720c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52258/0 (socket says 172.21.15.99:52258) 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.072+0000 7fb720c00700 1 -- 172.21.15.99:0/1679287077 learned_addr learned my addr 172.21.15.99:0/1679287077 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.072+0000 7fb71b400700 1 --2- 172.21.15.99:0/1679287077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.072+0000 7fb71be00700 1 --2- 172.21.15.99:0/1679287077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 0x55ab7d7af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.072+0000 7fb720c00700 1 -- 172.21.15.99:0/1679287077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 msgr2=0x55ab7d7af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.072+0000 7fb720c00700 1 --2- 172.21.15.99:0/1679287077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 0x55ab7d7af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.072+0000 7fb720c00700 1 -- 172.21.15.99:0/1679287077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a6b60 msgr2=0x55ab7d7b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.072+0000 7fb720c00700 1 --2- 172.21.15.99:0/1679287077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.072+0000 7fb720c00700 1 -- 172.21.15.99:0/1679287077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab7d6bde40 con 0x55ab7d7a7a60 2024-03-30T00:36:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.073+0000 7fb720c00700 1 --2- 172.21.15.99:0/1679287077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b7d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb71400b070 tx=0x7fb714007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21a97595725f460 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.073+0000 7fb711c00700 1 -- 172.21.15.99:0/1679287077 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400e040 con 0x55ab7d7a7a60 2024-03-30T00:36:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.073+0000 7fb711c00700 1 -- 172.21.15.99:0/1679287077 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb714020070 con 0x55ab7d7a7a60 2024-03-30T00:36:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.073+0000 7fb711c00700 1 -- 172.21.15.99:0/1679287077 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714005440 con 0x55ab7d7a7a60 2024-03-30T00:36:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.074+0000 7fb73678fb80 1 -- 172.21.15.99:0/1679287077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a7a60 msgr2=0x55ab7d7b7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.074+0000 7fb73678fb80 1 --2- 172.21.15.99:0/1679287077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b7d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb71400b070 tx=0x7fb714007590 comp rx=0 tx=0).stop 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.074+0000 7fb73678fb80 1 -- 172.21.15.99:0/1679287077 shutdown_connections 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.074+0000 7fb73678fb80 1 --2- 172.21.15.99:0/1679287077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b7d80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.074+0000 7fb73678fb80 1 --2- 172.21.15.99:0/1679287077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a6200 0x55ab7d7af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.074+0000 7fb73678fb80 1 --2- 172.21.15.99:0/1679287077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.074+0000 7fb73678fb80 1 -- 172.21.15.99:0/1679287077 >> 172.21.15.99:0/1679287077 conn(0x55ab7d705640 msgr2=0x55ab7d7c2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.074+0000 7fb73678fb80 1 -- 172.21.15.99:0/1679287077 shutdown_connections 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.074+0000 7fb73678fb80 1 -- 172.21.15.99:0/1679287077 wait complete. 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.074+0000 7fb73678fb80 1 -- start start 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.074+0000 7fb73678fb80 1 Processor -- start 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a6200 0x55ab7d7b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb73678fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb73678fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d7b6f80 con 0x55ab7d7a6b60 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb73678fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d7b70f0 con 0x55ab7d7a6200 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb73678fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7d7b72b0 con 0x55ab7d7a7a60 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb73678fb80 10 client.0 _ll_register_callbacks cb 0x55ab7d69d410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb71b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb71b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52268/0 (socket says 172.21.15.99:52268) 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb71b400700 1 -- 172.21.15.99:0/3404412952 learned_addr learned my addr 172.21.15.99:0/3404412952 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb71be00700 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a6200 0x55ab7d7b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.075+0000 7fb720c00700 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb71b400700 1 -- 172.21.15.99:0/3404412952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a7a60 msgr2=0x55ab7d7b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb71b400700 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb71b400700 1 -- 172.21.15.99:0/3404412952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a6200 msgr2=0x55ab7d7b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb71b400700 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a6200 0x55ab7d7b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb71b400700 1 -- 172.21.15.99:0/3404412952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71c10bc80 con 0x55ab7d7a6b60 2024-03-30T00:36:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb71b400700 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b5a80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb7080a1cf0 tx=0x7fb7080a1bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c7faf8f53dba16 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb705600700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb705600700 1 -- 172.21.15.99:0/3404412952 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708065410 con 0x55ab7d7a6b60 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb705600700 1 -- 172.21.15.99:0/3404412952 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fb70809d800 con 0x55ab7d7a6b60 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb73678fb80 1 -- 172.21.15.99:0/3404412952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb70c011d10 con 0x55ab7d7a6b60 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb705600700 1 -- 172.21.15.99:0/3404412952 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb708095d10 con 0x55ab7d7a6b60 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb73678fb80 10 client.15816 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.076+0000 7fb73678fb80 1 -- 172.21.15.99:0/3404412952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ab7d57dc70 con 0x55ab7d7a6b60 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.077+0000 7fb705600700 1 -- 172.21.15.99:0/3404412952 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb70c011d10 con 0x55ab7d7a6b60 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.077+0000 7fb705600700 1 -- 172.21.15.99:0/3404412952 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb708003b60 con 0x55ab7d7a6b60 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.077+0000 7fb73678fb80 10 client.15816 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.077+0000 7fb73678fb80 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.077+0000 7fb73678fb80 20 client.15816 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.077+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ab7d810960 0x55ab7d812d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.077+0000 7fb73678fb80 4 client.15816 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:36:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.077+0000 7fb73678fb80 1 -- 172.21.15.99:0/3404412952 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ab7d546c90 con 0x55ab7d810960 2024-03-30T00:36:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.077+0000 7fb71be00700 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ab7d810960 0x55ab7d812d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.078+0000 7fb705600700 10 client.15816 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.078+0000 7fb71be00700 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ab7d810960 0x55ab7d812d80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6d0debf9a72143 server_cookie=a2072dec8295ff41 in_seq=0 out_seq=0 2024-03-30T00:36:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.082+0000 7fb705600700 1 -- 172.21.15.99:0/3404412952 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ab7d546c90 con 0x55ab7d810960 2024-03-30T00:36:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.082+0000 7fb705600700 10 client.15816 handle_command_reply: tid=0 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.082+0000 7fb73678fb80 1 client.15816 shutdown 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb73678fb80 1 -- 172.21.15.99:0/3404412952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 msgr2=0x55ab7d7b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b5a80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb7080a1cf0 tx=0x7fb7080a1bd0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb73678fb80 1 -- 172.21.15.99:0/3404412952 shutdown_connections 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ab7d810960 0x55ab7d812d80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ab7d7a7a60 0x55ab7d7b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7d7a6200 0x55ab7d7b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb705600700 0 client.15816 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb73678fb80 1 --2- 172.21.15.99:0/3404412952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab7d7a6b60 0x55ab7d7b5a80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb73678fb80 1 -- 172.21.15.99:0/3404412952 >> 172.21.15.99:0/3404412952 conn(0x55ab7d705640 msgr2=0x55ab7d7b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb73678fb80 1 -- 172.21.15.99:0/3404412952 shutdown_connections 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb73678fb80 1 -- 172.21.15.99:0/3404412952 wait complete. 2024-03-30T00:36:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.083+0000 7fb73678fb80 20 client.15816 trim_cache size 0 max 16384 2024-03-30T00:36:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:36:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:36:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:36:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 -- 172.21.15.99:0/4288329426 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7000a1120 msgr2=0x7fb7000a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7000a1120 0x7fb7000a35e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb70c005950 tx=0x7fb70c0058e0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 -- 172.21.15.99:0/4288329426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c074df0 msgr2=0x7fb71c199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c074df0 0x7fb71c199460 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb70800d980 tx=0x7fb70800dc90 comp rx=0 tx=0).stop 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 -- 172.21.15.99:0/4288329426 shutdown_connections 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb7000a1120 0x7fb7000a35e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb71c108bd0 0x7fb71c1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb71c073810 0x7fb71c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 --2- 172.21.15.99:0/4288329426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb71c074df0 0x7fb71c199460 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 -- 172.21.15.99:0/4288329426 >> 172.21.15.99:0/4288329426 conn(0x7fb71c0fc280 msgr2=0x7fb71c104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 -- 172.21.15.99:0/4288329426 shutdown_connections 2024-03-30T00:36:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:07.084+0000 7fb722c00700 1 -- 172.21.15.99:0/4288329426 wait complete. 2024-03-30T00:36:08.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:08 smithi161 ceph-mon[42221]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T00:36:08.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:08 smithi099 ceph-mon[38079]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T00:36:08.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:08 smithi142 ceph-mon[41779]: pgmap v346: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T00:36:10.169 INFO:tasks.workunit.client.0.smithi099.stderr:Updating files: 64% (7863/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-30T00:36:10.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:10 smithi161 ceph-mon[42221]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-30T00:36:10.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:10 smithi099 ceph-mon[38079]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-30T00:36:10.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:10 smithi142 ceph-mon[41779]: pgmap v347: 97 pgs: 97 active+clean; 664 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-30T00:36:10.940 INFO:tasks.workunit.client.0.smithi099.stderr:Note: switching to '470100a4ccbcf7769052bf09a5413881ccf1b97f'. 2024-03-30T00:36:10.940 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-30T00:36:10.940 INFO:tasks.workunit.client.0.smithi099.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-30T00:36:10.940 INFO:tasks.workunit.client.0.smithi099.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-30T00:36:10.940 INFO:tasks.workunit.client.0.smithi099.stderr:state without impacting any branches by switching back to a branch. 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr: git switch -c 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr:Or undo this operation with: 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr: git switch - 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-30T00:36:10.941 INFO:tasks.workunit.client.0.smithi099.stderr:HEAD is now at 470100a4ccb DNM: trigger builds again 2024-03-30T00:36:10.955 DEBUG:teuthology.orchestra.run.smithi099:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-30T00:36:11.007 INFO:tasks.workunit.client.0.smithi099.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-30T00:36:11.012 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-30T00:36:11.012 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-30T00:36:11.619 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-30T00:36:11.739 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-30T00:36:11.803 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-30T00:36:11.806 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-30T00:36:11.806 INFO:tasks.workunit.client.0.smithi099.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-30T00:36:11.869 INFO:tasks.workunit.client.0.smithi099.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-30T00:36:11.876 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:36:11.876 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-30T00:36:11.924 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-03-30T00:36:11.925 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-03-30T00:36:11.925 DEBUG:teuthology.orchestra.run.smithi099:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=470100a4ccbcf7769052bf09a5413881ccf1b97f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-03-30T00:36:12.039 INFO:tasks.workunit.client.0.smithi099.stderr:+ set -e 2024-03-30T00:36:12.039 INFO:tasks.workunit.client.0.smithi099.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-03-30T00:36:12.084 INFO:tasks.workunit.client.0.smithi099.stderr:--2024-03-30 00:36:12-- http://download.ceph.com/qa/fsync-tester.c 2024-03-30T00:36:12.085 INFO:tasks.workunit.client.0.smithi099.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-30T00:36:12.107 INFO:tasks.workunit.client.0.smithi099.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-30T00:36:12.129 INFO:tasks.workunit.client.0.smithi099.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-30T00:36:12.130 INFO:tasks.workunit.client.0.smithi099.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-03-30T00:36:12.133 INFO:tasks.workunit.client.0.smithi099.stderr:Saving to: ‘fsync-tester.c’ 2024-03-30T00:36:12.137 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-30T00:36:12.137 INFO:tasks.workunit.client.0.smithi099.stderr: 0K .. 100% 185M=0s 2024-03-30T00:36:12.137 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-30T00:36:12.138 INFO:tasks.workunit.client.0.smithi099.stderr:2024-03-30 00:36:12 (185 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-03-30T00:36:12.138 INFO:tasks.workunit.client.0.smithi099.stderr: 2024-03-30T00:36:12.138 INFO:tasks.workunit.client.0.smithi099.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-03-30T00:36:12.292 INFO:tasks.workunit.client.0.smithi099.stderr:+ ./fsync-tester 2024-03-30T00:36:12.488 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:12 smithi099 ceph-mon[38079]: pgmap v348: 97 pgs: 97 active+clean; 665 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:36:12.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:12 smithi142 ceph-mon[41779]: pgmap v348: 97 pgs: 97 active+clean; 665 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:36:12.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:12 smithi161 ceph-mon[42221]: pgmap v348: 97 pgs: 97 active+clean; 665 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-30T00:36:14.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:14 smithi099 ceph-mon[38079]: pgmap v349: 97 pgs: 97 active+clean; 665 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:36:14.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:14 smithi142 ceph-mon[41779]: pgmap v349: 97 pgs: 97 active+clean; 665 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:36:14.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:14 smithi161 ceph-mon[42221]: pgmap v349: 97 pgs: 97 active+clean; 665 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-30T00:36:16.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:16 smithi099 ceph-mon[38079]: pgmap v350: 97 pgs: 97 active+clean; 8.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 685 KiB/s wr, 7 op/s 2024-03-30T00:36:16.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:16 smithi142 ceph-mon[41779]: pgmap v350: 97 pgs: 97 active+clean; 8.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 685 KiB/s wr, 7 op/s 2024-03-30T00:36:16.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:16 smithi161 ceph-mon[42221]: pgmap v350: 97 pgs: 97 active+clean; 8.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 685 KiB/s wr, 7 op/s 2024-03-30T00:36:18.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:18 smithi099 ceph-mon[38079]: pgmap v351: 97 pgs: 97 active+clean; 21 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 MiB/s wr, 9 op/s 2024-03-30T00:36:18.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:18 smithi142 ceph-mon[41779]: pgmap v351: 97 pgs: 97 active+clean; 21 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 MiB/s wr, 9 op/s 2024-03-30T00:36:18.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:18 smithi161 ceph-mon[42221]: pgmap v351: 97 pgs: 97 active+clean; 21 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 MiB/s wr, 9 op/s 2024-03-30T00:36:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.237+0000 7f333f200700 1 -- 172.21.15.99:0/3511186037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33401015d0 msgr2=0x7f33401039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.237+0000 7f333f200700 1 --2- 172.21.15.99:0/3511186037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33401015d0 0x7f33401039c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3330009990 tx=0x7f3330009ca0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.237+0000 7f333f200700 1 -- 172.21.15.99:0/3511186037 shutdown_connections 2024-03-30T00:36:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.237+0000 7f333f200700 1 --2- 172.21.15.99:0/3511186037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3340106830 0x7f3340108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.237+0000 7f333f200700 1 --2- 172.21.15.99:0/3511186037 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33401015d0 0x7f33401039c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.237+0000 7f333f200700 1 --2- 172.21.15.99:0/3511186037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3340103f00 0x7f33401062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.237+0000 7f333f200700 1 -- 172.21.15.99:0/3511186037 >> 172.21.15.99:0/3511186037 conn(0x7f33400fafc0 msgr2=0x7f33400fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.237+0000 7f333f200700 1 -- 172.21.15.99:0/3511186037 shutdown_connections 2024-03-30T00:36:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.237+0000 7f333f200700 1 -- 172.21.15.99:0/3511186037 wait complete. 2024-03-30T00:36:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.238+0000 7f333f200700 1 Processor -- start 2024-03-30T00:36:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333f200700 1 -- start start 2024-03-30T00:36:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333f200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33401015d0 0x7f3340196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333f200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3340103f00 0x7f3340196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333f200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3340106830 0x7f334019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333f200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334019e530 con 0x7f33401015d0 2024-03-30T00:36:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333f200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334019e670 con 0x7f3340103f00 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333f200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334019e800 con 0x7f3340106830 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33401015d0 0x7f3340196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33401015d0 0x7f3340196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53300/0 (socket says 172.21.15.99:53300) 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333de00700 1 -- 172.21.15.99:0/4254517346 learned_addr learned my addr 172.21.15.99:0/4254517346 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333d400700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3340103f00 0x7f3340196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.239+0000 7f333e800700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3340106830 0x7f334019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.240+0000 7f333de00700 1 -- 172.21.15.99:0/4254517346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3340106830 msgr2=0x7f334019df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.240+0000 7f333de00700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3340106830 0x7f334019df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.240+0000 7f333de00700 1 -- 172.21.15.99:0/4254517346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3340103f00 msgr2=0x7f3340196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.240+0000 7f333de00700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3340103f00 0x7f3340196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.240+0000 7f333de00700 1 -- 172.21.15.99:0/4254517346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33300095f0 con 0x7f33401015d0 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.240+0000 7f333de00700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33401015d0 0x7f3340196a00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f333000b2e0 tx=0x7f3330004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.240+0000 7f3336a00700 1 -- 172.21.15.99:0/4254517346 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333001d070 con 0x7f33401015d0 2024-03-30T00:36:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.240+0000 7f3336a00700 1 -- 172.21.15.99:0/4254517346 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f333000beb0 con 0x7f33401015d0 2024-03-30T00:36:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.240+0000 7f333f200700 1 -- 172.21.15.99:0/4254517346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33400ff500 con 0x7f33401015d0 2024-03-30T00:36:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.240+0000 7f3336a00700 1 -- 172.21.15.99:0/4254517346 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330021410 con 0x7f33401015d0 2024-03-30T00:36:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.240+0000 7f333f200700 1 -- 172.21.15.99:0/4254517346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33400ffa00 con 0x7f33401015d0 2024-03-30T00:36:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.243+0000 7f3336a00700 1 -- 172.21.15.99:0/4254517346 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f333000f640 con 0x7f33401015d0 2024-03-30T00:36:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.245+0000 7f3336a00700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f33240a11a0 0x7f33240a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.245+0000 7f3336a00700 1 -- 172.21.15.99:0/4254517346 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f33300cdca0 con 0x7f33401015d0 2024-03-30T00:36:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.245+0000 7f333d400700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f33240a11a0 0x7f33240a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.245+0000 7f3358966b80 1 Processor -- start 2024-03-30T00:36:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.245+0000 7f3358966b80 1 -- start start 2024-03-30T00:36:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.246+0000 7f3358966b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 0x55f11849b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.246+0000 7f3358966b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849be40 0x55f11849c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.246+0000 7f3358966b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849cc90 0x55f11849d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.246+0000 7f3358966b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1183f3930 con 0x55f11849cc90 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.246+0000 7f3358966b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1183f12f0 con 0x55f11849be40 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.246+0000 7f3358966b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1183ecbe0 con 0x55f11849b4e0 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.246+0000 7f333d400700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f33240a11a0 0x7f33240a3660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f332c009c60 tx=0x7f332c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.246+0000 7f333de00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 0x55f11849b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.246+0000 7f333e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849cc90 0x55f11849d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.246+0000 7f333de00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 0x55f11849b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:45374/0 (socket says 172.21.15.99:45374) 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.246+0000 7f333de00700 1 -- 172.21.15.99:0/2105097116 learned_addr learned my addr 172.21.15.99:0/2105097116 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.247+0000 7f333d400700 1 --2- 172.21.15.99:0/2105097116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849be40 0x55f11849c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.247+0000 7f333de00700 1 -- 172.21.15.99:0/2105097116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849be40 msgr2=0x55f11849c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.247+0000 7f333de00700 1 --2- 172.21.15.99:0/2105097116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849be40 0x55f11849c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.247+0000 7f333de00700 1 -- 172.21.15.99:0/2105097116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849cc90 msgr2=0x55f11849d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.247+0000 7f333de00700 1 --2- 172.21.15.99:0/2105097116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849cc90 0x55f11849d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.247+0000 7f333de00700 1 -- 172.21.15.99:0/2105097116 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f11826e350 con 0x55f11849b4e0 2024-03-30T00:36:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.247+0000 7f333e800700 1 --2- 172.21.15.99:0/2105097116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849cc90 0x55f11849d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.247+0000 7f333de00700 1 --2- 172.21.15.99:0/2105097116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 0x55f11849b900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f33300975a0 tx=0x7f3330004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=284271cda3727f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.247+0000 7f332b400700 1 -- 172.21.15.99:0/2105097116 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333000f460 con 0x55f11849b4e0 2024-03-30T00:36:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.247+0000 7f332b400700 1 -- 172.21.15.99:0/2105097116 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f333009b7a0 con 0x55f11849b4e0 2024-03-30T00:36:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.247+0000 7f332b400700 1 -- 172.21.15.99:0/2105097116 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300cb8a0 con 0x55f11849b4e0 2024-03-30T00:36:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.248+0000 7f3358966b80 1 -- 172.21.15.99:0/2105097116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 msgr2=0x55f11849b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.248+0000 7f3358966b80 1 --2- 172.21.15.99:0/2105097116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 0x55f11849b900 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f33300975a0 tx=0x7f3330004320 comp rx=0 tx=0).stop 2024-03-30T00:36:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.248+0000 7f3358966b80 1 -- 172.21.15.99:0/2105097116 shutdown_connections 2024-03-30T00:36:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.248+0000 7f3358966b80 1 --2- 172.21.15.99:0/2105097116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849cc90 0x55f11849d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.248+0000 7f3358966b80 1 --2- 172.21.15.99:0/2105097116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 0x55f11849b900 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.248+0000 7f3358966b80 1 --2- 172.21.15.99:0/2105097116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849be40 0x55f11849c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.248+0000 7f3358966b80 1 -- 172.21.15.99:0/2105097116 >> 172.21.15.99:0/2105097116 conn(0x55f1183fa640 msgr2=0x55f118496360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.248+0000 7f3358966b80 1 -- 172.21.15.99:0/2105097116 shutdown_connections 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.248+0000 7f3358966b80 1 -- 172.21.15.99:0/2105097116 wait complete. 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.249+0000 7f3358966b80 1 -- start start 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.249+0000 7f3358966b80 1 Processor -- start 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f3358966b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184a5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f3358966b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849be40 0x55f1184a5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f3358966b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f3358966b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f118388d20 con 0x55f11849b4e0 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f3358966b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1183889c0 con 0x55f11849cc90 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f3358966b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f118270640 con 0x55f11849be40 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f3358966b80 10 client.0 _ll_register_callbacks cb 0x55f118392410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f333de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f333d400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849be40 0x55f1184a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f333de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184a5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53330/0 (socket says 172.21.15.99:53330) 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f333de00700 1 -- 172.21.15.99:0/18341070 learned_addr learned my addr 172.21.15.99:0/18341070 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f333e800700 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.250+0000 7f333de00700 1 -- 172.21.15.99:0/18341070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849be40 msgr2=0x55f1184a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f333de00700 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849be40 0x55f1184a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f333de00700 1 -- 172.21.15.99:0/18341070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 msgr2=0x55f1184ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f333de00700 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f333de00700 1 -- 172.21.15.99:0/18341070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3338009900 con 0x55f11849b4e0 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f333e800700 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f333de00700 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184a5020 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f33300095c0 tx=0x7f333009ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c84bae5ea399c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f331fe00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f331fe00700 1 -- 172.21.15.99:0/18341070 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333009eed0 con 0x55f11849b4e0 2024-03-30T00:36:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f331fe00700 1 -- 172.21.15.99:0/18341070 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f33300cb840 con 0x55f11849b4e0 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f3358966b80 1 -- 172.21.15.99:0/18341070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1183ad320 con 0x55f11849b4e0 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f331fe00700 1 -- 172.21.15.99:0/18341070 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33300b9250 con 0x55f11849b4e0 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f3358966b80 10 client.15828 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.251+0000 7f3358966b80 1 -- 172.21.15.99:0/18341070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f33400ffa00 con 0x55f11849b4e0 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.252+0000 7f331fe00700 1 -- 172.21.15.99:0/18341070 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f1183ad320 con 0x55f11849b4e0 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.252+0000 7f331fe00700 1 -- 172.21.15.99:0/18341070 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f33300b3bc0 con 0x55f11849b4e0 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.252+0000 7f3358966b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.252+0000 7f3358966b80 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.252+0000 7f3358966b80 20 client.15828 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.252+0000 7f3358966b80 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f1184b08b0 0x55f1184b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.252+0000 7f3358966b80 4 client.15828 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.252+0000 7f3358966b80 1 -- 172.21.15.99:0/18341070 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1183e56a0 con 0x55f1184b08b0 2024-03-30T00:36:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.252+0000 7f333d400700 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f1184b08b0 0x55f1184b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.253+0000 7f333d400700 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f1184b08b0 0x55f1184b2cd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=274c0618c318f830 server_cookie=ec4cea5aaa2bd22e in_seq=0 out_seq=0 2024-03-30T00:36:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.253+0000 7f331fe00700 10 client.15828 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f331fe00700 1 -- 172.21.15.99:0/18341070 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f1183e56a0 con 0x55f1184b08b0 2024-03-30T00:36:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f331fe00700 10 client.15828 handle_command_reply: tid=0 2024-03-30T00:36:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 1 client.15828 shutdown 2024-03-30T00:36:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 1 -- 172.21.15.99:0/18341070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 msgr2=0x55f1184a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184a5020 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f33300095c0 tx=0x7f333009ab30 comp rx=0 tx=0).stop 2024-03-30T00:36:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 1 -- 172.21.15.99:0/18341070 shutdown_connections 2024-03-30T00:36:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f1184b08b0 0x55f1184b2cd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184a5020 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 1 --2- 172.21.15.99:0/18341070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849be40 0x55f1184a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 1 -- 172.21.15.99:0/18341070 >> 172.21.15.99:0/18341070 conn(0x55f1183fa640 msgr2=0x55f11849a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f331fe00700 0 client.15828 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 1 -- 172.21.15.99:0/18341070 shutdown_connections 2024-03-30T00:36:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 1 -- 172.21.15.99:0/18341070 wait complete. 2024-03-30T00:36:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.254+0000 7f3358966b80 20 client.15828 trim_cache size 0 max 16384 2024-03-30T00:36:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.274+0000 7f3358966b80 1 Processor -- start 2024-03-30T00:36:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.274+0000 7f3358966b80 1 -- start start 2024-03-30T00:36:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.274+0000 7f3358966b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 0x55f11849d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.274+0000 7f3358966b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849be40 0x55f1184ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.274+0000 7f3358966b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184a4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.274+0000 7f3358966b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11849b120 con 0x55f11849be40 2024-03-30T00:36:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.274+0000 7f3358966b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f118492230 con 0x55f11849cc90 2024-03-30T00:36:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.274+0000 7f3358966b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f118490ac0 con 0x55f11849b4e0 2024-03-30T00:36:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849be40 0x55f1184ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849be40 0x55f1184ac560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53342/0 (socket says 172.21.15.99:53342) 2024-03-30T00:36:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333d400700 1 -- 172.21.15.99:0/2425687457 learned_addr learned my addr 172.21.15.99:0/2425687457 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333e800700 1 --2- 172.21.15.99:0/2425687457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333de00700 1 --2- 172.21.15.99:0/2425687457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 0x55f11849d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333d400700 1 -- 172.21.15.99:0/2425687457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 msgr2=0x55f11849d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333d400700 1 --2- 172.21.15.99:0/2425687457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 0x55f11849d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333d400700 1 -- 172.21.15.99:0/2425687457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 msgr2=0x55f1184a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333d400700 1 --2- 172.21.15.99:0/2425687457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333d400700 1 -- 172.21.15.99:0/2425687457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f11821ab00 con 0x55f11849be40 2024-03-30T00:36:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333de00700 1 --2- 172.21.15.99:0/2425687457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 0x55f11849d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f333d400700 1 --2- 172.21.15.99:0/2425687457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849be40 0x55f1184ac560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f332c01e140 tx=0x7f332c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ca36f97a974354 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.275+0000 7f332be00700 1 -- 172.21.15.99:0/2425687457 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c023040 con 0x55f11849be40 2024-03-30T00:36:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.276+0000 7f332be00700 1 -- 172.21.15.99:0/2425687457 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f332c004590 con 0x55f11849be40 2024-03-30T00:36:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.276+0000 7f332be00700 1 -- 172.21.15.99:0/2425687457 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c003c80 con 0x55f11849be40 2024-03-30T00:36:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.276+0000 7f3358966b80 1 -- 172.21.15.99:0/2425687457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849be40 msgr2=0x55f1184ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.276+0000 7f3358966b80 1 --2- 172.21.15.99:0/2425687457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849be40 0x55f1184ac560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f332c01e140 tx=0x7f332c014000 comp rx=0 tx=0).stop 2024-03-30T00:36:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.276+0000 7f3358966b80 1 -- 172.21.15.99:0/2425687457 shutdown_connections 2024-03-30T00:36:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.276+0000 7f3358966b80 1 --2- 172.21.15.99:0/2425687457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.276+0000 7f3358966b80 1 --2- 172.21.15.99:0/2425687457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849b4e0 0x55f11849d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.276+0000 7f3358966b80 1 --2- 172.21.15.99:0/2425687457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849be40 0x55f1184ac560 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.276+0000 7f3358966b80 1 -- 172.21.15.99:0/2425687457 >> 172.21.15.99:0/2425687457 conn(0x55f1183fa640 msgr2=0x55f1184b7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.277+0000 7f3358966b80 1 -- 172.21.15.99:0/2425687457 shutdown_connections 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.277+0000 7f3358966b80 1 -- 172.21.15.99:0/2425687457 wait complete. 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.277+0000 7f3358966b80 1 -- start start 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.277+0000 7f3358966b80 1 Processor -- start 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f3358966b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f3358966b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849be40 0x55f1184aab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f3358966b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f3358966b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1184abf20 con 0x55f11849b4e0 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f3358966b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1184ac090 con 0x55f11849cc90 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f3358966b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1184ac250 con 0x55f11849be40 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f3358966b80 10 client.0 _ll_register_callbacks cb 0x55f118392410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f333de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f333de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184aa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53354/0 (socket says 172.21.15.99:53354) 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f333de00700 1 -- 172.21.15.99:0/4155128714 learned_addr learned my addr 172.21.15.99:0/4155128714 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f333e800700 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.278+0000 7f333d400700 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849be40 0x55f1184aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f333de00700 1 -- 172.21.15.99:0/4155128714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849be40 msgr2=0x55f1184aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f333de00700 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849be40 0x55f1184aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f333de00700 1 -- 172.21.15.99:0/4155128714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 msgr2=0x55f1184ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f333de00700 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f333de00700 1 -- 172.21.15.99:0/4155128714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c017070 con 0x55f11849b4e0 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f333d400700 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849be40 0x55f1184aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f333de00700 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184aa5d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f33300975a0 tx=0x7f3330099650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc665dfbd7b1ed49 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f331fe00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f331fe00700 1 -- 172.21.15.99:0/4155128714 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300cbcf0 con 0x55f11849b4e0 2024-03-30T00:36:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f331fe00700 1 -- 172.21.15.99:0/4155128714 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f333009eed0 con 0x55f11849b4e0 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f3358966b80 1 -- 172.21.15.99:0/4155128714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f11823bc90 con 0x55f11849b4e0 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f331fe00700 1 -- 172.21.15.99:0/4155128714 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33300a3490 con 0x55f11849b4e0 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f3358966b80 10 client.15834 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.279+0000 7f3358966b80 1 -- 172.21.15.99:0/4155128714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f1183b2e40 con 0x55f11849b4e0 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.280+0000 7f331fe00700 1 -- 172.21.15.99:0/4155128714 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f11823bc90 con 0x55f11849b4e0 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.280+0000 7f331fe00700 1 -- 172.21.15.99:0/4155128714 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f33300a3090 con 0x55f11849b4e0 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.280+0000 7f3358966b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.280+0000 7f3358966b80 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.280+0000 7f3358966b80 20 client.15834 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.280+0000 7f3358966b80 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f1185050c0 0x55f1185074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.280+0000 7f3358966b80 4 client.15834 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.280+0000 7f3358966b80 1 -- 172.21.15.99:0/4155128714 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f118270c80 con 0x55f1185050c0 2024-03-30T00:36:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.280+0000 7f333d400700 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f1185050c0 0x55f1185074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.281+0000 7f331fe00700 10 client.15834 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.281+0000 7f333d400700 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f1185050c0 0x55f1185074b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ea841a836477b6 server_cookie=f7d7f030a4585187 in_seq=0 out_seq=0 2024-03-30T00:36:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.293+0000 7f331fe00700 1 -- 172.21.15.99:0/4155128714 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f118270c80 con 0x55f1185050c0 2024-03-30T00:36:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.293+0000 7f331fe00700 10 client.15834 handle_command_reply: tid=0 2024-03-30T00:36:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.293+0000 7f3358966b80 1 client.15834 shutdown 2024-03-30T00:36:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.293+0000 7f3358966b80 1 -- 172.21.15.99:0/4155128714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 msgr2=0x55f1184aa5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.293+0000 7f3358966b80 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184aa5d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f33300975a0 tx=0x7f3330099650 comp rx=0 tx=0).stop 2024-03-30T00:36:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.293+0000 7f3358966b80 1 -- 172.21.15.99:0/4155128714 shutdown_connections 2024-03-30T00:36:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.293+0000 7f3358966b80 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55f1185050c0 0x55f1185074b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.293+0000 7f3358966b80 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f11849cc90 0x55f1184ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.293+0000 7f3358966b80 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f11849b4e0 0x55f1184aa5d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.293+0000 7f3358966b80 1 --2- 172.21.15.99:0/4155128714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f11849be40 0x55f1184aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.293+0000 7f3358966b80 1 -- 172.21.15.99:0/4155128714 >> 172.21.15.99:0/4155128714 conn(0x55f1183fa640 msgr2=0x55f118496230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.294+0000 7f331fe00700 0 client.15834 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.294+0000 7f3358966b80 1 -- 172.21.15.99:0/4155128714 shutdown_connections 2024-03-30T00:36:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.294+0000 7f3358966b80 1 -- 172.21.15.99:0/4155128714 wait complete. 2024-03-30T00:36:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.294+0000 7f3358966b80 20 client.15834 trim_cache size 0 max 16384 2024-03-30T00:36:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:36:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:36:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:36:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:36:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 -- 172.21.15.99:0/4254517346 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f33240a11a0 msgr2=0x7f33240a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f33240a11a0 0x7f33240a3660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f332c009c60 tx=0x7f332c009260 comp rx=0 tx=0).stop 2024-03-30T00:36:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 -- 172.21.15.99:0/4254517346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33401015d0 msgr2=0x7f3340196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33401015d0 0x7f3340196a00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f333000b2e0 tx=0x7f3330004730 comp rx=0 tx=0).stop 2024-03-30T00:36:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 -- 172.21.15.99:0/4254517346 shutdown_connections 2024-03-30T00:36:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3340106830 0x7f334019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33401015d0 0x7f3340196a00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f33240a11a0 0x7f33240a3660 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 --2- 172.21.15.99:0/4254517346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3340103f00 0x7f3340196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 -- 172.21.15.99:0/4254517346 >> 172.21.15.99:0/4254517346 conn(0x7f33400fafc0 msgr2=0x7f3340103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 -- 172.21.15.99:0/4254517346 shutdown_connections 2024-03-30T00:36:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:20.295+0000 7f333f200700 1 -- 172.21.15.99:0/4254517346 wait complete. 2024-03-30T00:36:20.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:20 smithi099 ceph-mon[38079]: pgmap v352: 97 pgs: 97 active+clean; 69 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.7 MiB/s wr, 8 op/s 2024-03-30T00:36:20.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:20 smithi142 ceph-mon[41779]: pgmap v352: 97 pgs: 97 active+clean; 69 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.7 MiB/s wr, 8 op/s 2024-03-30T00:36:20.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:20 smithi161 ceph-mon[42221]: pgmap v352: 97 pgs: 97 active+clean; 69 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.7 MiB/s wr, 8 op/s 2024-03-30T00:36:21.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:21 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:36:21.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:21 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:36:21.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:21 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:36:22.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:22 smithi142 ceph-mon[41779]: pgmap v353: 97 pgs: 97 active+clean; 93 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.7 MiB/s wr, 11 op/s 2024-03-30T00:36:22.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:36:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:36:22.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:36:22.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:22 smithi161 ceph-mon[42221]: pgmap v353: 97 pgs: 97 active+clean; 93 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.7 MiB/s wr, 11 op/s 2024-03-30T00:36:22.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:36:22.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:36:22.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:36:22.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:22 smithi099 ceph-mon[38079]: pgmap v353: 97 pgs: 97 active+clean; 93 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.7 MiB/s wr, 11 op/s 2024-03-30T00:36:22.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:36:22.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:36:22.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:36:24.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:24 smithi142 ceph-mon[41779]: pgmap v354: 97 pgs: 97 active+clean; 133 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-03-30T00:36:24.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:24 smithi161 ceph-mon[42221]: pgmap v354: 97 pgs: 97 active+clean; 133 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-03-30T00:36:24.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:24 smithi099 ceph-mon[38079]: pgmap v354: 97 pgs: 97 active+clean; 133 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 11 MiB/s wr, 12 op/s 2024-03-30T00:36:25.152 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T00:36:25.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.362+0000 7f72e8a00700 1 -- 172.21.15.99:0/3950816747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e4102750 msgr2=0x7f72e4109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.362+0000 7f72e8a00700 1 --2- 172.21.15.99:0/3950816747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e4102750 0x7f72e4109000 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00b0d0 tx=0x7f72dc00b3e0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.362+0000 7f72e8a00700 1 -- 172.21.15.99:0/3950816747 shutdown_connections 2024-03-30T00:36:25.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.362+0000 7f72e8a00700 1 --2- 172.21.15.99:0/3950816747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e4102750 0x7f72e4109000 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.362+0000 7f72e8a00700 1 --2- 172.21.15.99:0/3950816747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e4100bc0 0x7f72e4100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.362+0000 7f72e8a00700 1 --2- 172.21.15.99:0/3950816747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e4101db0 0x7f72e4102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.362+0000 7f72e8a00700 1 -- 172.21.15.99:0/3950816747 >> 172.21.15.99:0/3950816747 conn(0x7f72e40fc110 msgr2=0x7f72e40fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:25.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.362+0000 7f72e8a00700 1 -- 172.21.15.99:0/3950816747 shutdown_connections 2024-03-30T00:36:25.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.362+0000 7f72e8a00700 1 -- 172.21.15.99:0/3950816747 wait complete. 2024-03-30T00:36:25.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.363+0000 7f72e8a00700 1 Processor -- start 2024-03-30T00:36:25.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.363+0000 7f72e8a00700 1 -- start start 2024-03-30T00:36:25.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e8a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e4100bc0 0x7f72e4198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e8a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e4101db0 0x7f72e41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e8a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e4102750 0x7f72e41a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e8a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e41a08e0 con 0x7f72e4101db0 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e8a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e41a0a20 con 0x7f72e4100bc0 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e8a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e41a0bb0 con 0x7f72e4102750 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e4101db0 0x7f72e41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e3200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e4102750 0x7f72e41a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e3200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e4102750 0x7f72e41a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:59272/0 (socket says 172.21.15.99:59272) 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e4101db0 0x7f72e41992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59296/0 (socket says 172.21.15.99:59296) 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e3200700 1 -- 172.21.15.99:0/1682880092 learned_addr learned my addr 172.21.15.99:0/1682880092 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e2800700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e4100bc0 0x7f72e4198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e1e00700 1 -- 172.21.15.99:0/1682880092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e4102750 msgr2=0x7f72e41a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.364+0000 7f72e1e00700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e4102750 0x7f72e41a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.365+0000 7f72e1e00700 1 -- 172.21.15.99:0/1682880092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e4100bc0 msgr2=0x7f72e4198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.365+0000 7f72e1e00700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e4100bc0 0x7f72e4198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.365+0000 7f72e1e00700 1 -- 172.21.15.99:0/1682880092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc009d00 con 0x7f72e4101db0 2024-03-30T00:36:25.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.365+0000 7f72e3200700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e4102750 0x7f72e41a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:25.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.365+0000 7f72e1e00700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e4101db0 0x7f72e41992f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f72d000d950 tx=0x7f72d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:25.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.365+0000 7f72db400700 1 -- 172.21.15.99:0/1682880092 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000f890 con 0x7f72e4101db0 2024-03-30T00:36:25.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.365+0000 7f72db400700 1 -- 172.21.15.99:0/1682880092 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72d000fe80 con 0x7f72e4101db0 2024-03-30T00:36:25.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.365+0000 7f72db400700 1 -- 172.21.15.99:0/1682880092 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000e3b0 con 0x7f72e4101db0 2024-03-30T00:36:25.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.365+0000 7f72e8a00700 1 -- 172.21.15.99:0/1682880092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e41a0e90 con 0x7f72e4101db0 2024-03-30T00:36:25.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.365+0000 7f72e8a00700 1 -- 172.21.15.99:0/1682880092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e41a1260 con 0x7f72e4101db0 2024-03-30T00:36:25.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.369+0000 7f72db400700 1 -- 172.21.15.99:0/1682880092 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f72d000f9f0 con 0x7f72e4101db0 2024-03-30T00:36:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.370+0000 7f72fcffcb80 1 Processor -- start 2024-03-30T00:36:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.370+0000 7f72fcffcb80 1 -- start start 2024-03-30T00:36:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.370+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.370+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3a9340 0x55e7db3a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.370+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3aa190 0x55e7db3aa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.370+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db300990 con 0x55e7db3a89e0 2024-03-30T00:36:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db2fe350 con 0x55e7db3aa190 2024-03-30T00:36:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db2f9c40 con 0x55e7db3a9340 2024-03-30T00:36:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72e3200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72e3200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3a8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59298/0 (socket says 172.21.15.99:59298) 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72e3200700 1 -- 172.21.15.99:0/2258095604 learned_addr learned my addr 172.21.15.99:0/2258095604 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72e1e00700 1 --2- 172.21.15.99:0/2258095604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3aa190 0x55e7db3aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72db400700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72c80a1120 0x7f72c80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72e2800700 1 --2- 172.21.15.99:0/2258095604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3a9340 0x55e7db3a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72e3200700 1 -- 172.21.15.99:0/2258095604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3a9340 msgr2=0x55e7db3a9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72db400700 1 -- 172.21.15.99:0/1682880092 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f72d00cc630 con 0x7f72e4101db0 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72e3200700 1 --2- 172.21.15.99:0/2258095604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3a9340 0x55e7db3a9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72e3200700 1 -- 172.21.15.99:0/2258095604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3aa190 msgr2=0x55e7db3aa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72e3200700 1 --2- 172.21.15.99:0/2258095604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3aa190 0x55e7db3aa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.371+0000 7f72e3200700 1 -- 172.21.15.99:0/2258095604 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7db17b3b0 con 0x55e7db3a89e0 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.372+0000 7f72e3200700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72c80a1120 0x7f72c80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.372+0000 7f72e3200700 1 --2- 172.21.15.99:0/2258095604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3a8e00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00cda0 tx=0x7f72dc00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95d57d7f8f24b7a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.372+0000 7f72cf400700 1 -- 172.21.15.99:0/2258095604 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00e040 con 0x55e7db3a89e0 2024-03-30T00:36:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.372+0000 7f72cf400700 1 -- 172.21.15.99:0/2258095604 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72dc009290 con 0x55e7db3a89e0 2024-03-30T00:36:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.372+0000 7f72cf400700 1 -- 172.21.15.99:0/2258095604 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc02a6a0 con 0x55e7db3a89e0 2024-03-30T00:36:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.372+0000 7f72e3200700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72c80a1120 0x7f72c80a35e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f72dc02a000 tx=0x7f72dc027000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.373+0000 7f72fcffcb80 1 -- 172.21.15.99:0/2258095604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 msgr2=0x55e7db3a8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.373+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/2258095604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3a8e00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00cda0 tx=0x7f72dc00cdd0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.373+0000 7f72fcffcb80 1 -- 172.21.15.99:0/2258095604 shutdown_connections 2024-03-30T00:36:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.373+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/2258095604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3aa190 0x55e7db3aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.373+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/2258095604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3a8e00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.373+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/2258095604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3a9340 0x55e7db3a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.373+0000 7f72fcffcb80 1 -- 172.21.15.99:0/2258095604 >> 172.21.15.99:0/2258095604 conn(0x55e7db3076a0 msgr2=0x55e7db3a3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.373+0000 7f72fcffcb80 1 -- 172.21.15.99:0/2258095604 shutdown_connections 2024-03-30T00:36:25.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.373+0000 7f72fcffcb80 1 -- 172.21.15.99:0/2258095604 wait complete. 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.374+0000 7f72fcffcb80 1 -- start start 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.374+0000 7f72fcffcb80 1 Processor -- start 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3a89e0 0x55e7db3b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3aa190 0x55e7db3b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db295d80 con 0x55e7db3a9340 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db295a20 con 0x55e7db3aa190 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db17d6a0 con 0x55e7db3a89e0 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72fcffcb80 10 client.0 _ll_register_callbacks cb 0x55e7db29f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59314/0 (socket says 172.21.15.99:59314) 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72e1e00700 1 -- 172.21.15.99:0/1828700312 learned_addr learned my addr 172.21.15.99:0/1828700312 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72e2800700 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3a89e0 0x55e7db3b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.375+0000 7f72e3200700 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3aa190 0x55e7db3b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.376+0000 7f72e1e00700 1 -- 172.21.15.99:0/1828700312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3a89e0 msgr2=0x55e7db3b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.376+0000 7f72e1e00700 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3a89e0 0x55e7db3b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.376+0000 7f72e1e00700 1 -- 172.21.15.99:0/1828700312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3aa190 msgr2=0x55e7db3b9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.376+0000 7f72e1e00700 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3aa190 0x55e7db3b9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.376+0000 7f72e1e00700 1 -- 172.21.15.99:0/1828700312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e4109950 con 0x55e7db3a9340 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.376+0000 7f72e3200700 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3aa190 0x55e7db3b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.376+0000 7f72e1e00700 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b2d90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f72d00ca0b0 tx=0x7f72d0099830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3a221c3c443b76 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.376+0000 7f72c3e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.376+0000 7f72c3e00700 1 -- 172.21.15.99:0/1828700312 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d00106d0 con 0x55e7db3a9340 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.376+0000 7f72c3e00700 1 -- 172.21.15.99:0/1828700312 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f72d009a100 con 0x55e7db3a9340 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.376+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1828700312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7db2f2700 con 0x55e7db3a9340 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.377+0000 7f72c3e00700 1 -- 172.21.15.99:0/1828700312 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72d000fca0 con 0x55e7db3a9340 2024-03-30T00:36:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.377+0000 7f72fcffcb80 10 client.15846 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.377+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1828700312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e7db127b60 con 0x55e7db3a9340 2024-03-30T00:36:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.377+0000 7f72c3e00700 1 -- 172.21.15.99:0/1828700312 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e7db2f2700 con 0x55e7db3a9340 2024-03-30T00:36:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.377+0000 7f72c3e00700 1 -- 172.21.15.99:0/1828700312 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f72d0099c60 con 0x55e7db3a9340 2024-03-30T00:36:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.377+0000 7f72fcffcb80 10 client.15846 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.377+0000 7f72fcffcb80 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.377+0000 7f72fcffcb80 20 client.15846 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.377+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7db3be600 0x55e7db3c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.377+0000 7f72fcffcb80 4 client.15846 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.377+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1828700312 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7db148cf0 con 0x55e7db3be600 2024-03-30T00:36:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.378+0000 7f72e2800700 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7db3be600 0x55e7db3c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.378+0000 7f72c3e00700 10 client.15846 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.378+0000 7f72e2800700 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7db3be600 0x55e7db3c0a20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c45bf451d09dadf6 server_cookie=4daa55c21d206657 in_seq=0 out_seq=0 2024-03-30T00:36:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72c3e00700 1 -- 172.21.15.99:0/1828700312 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e7db148cf0 con 0x55e7db3be600 2024-03-30T00:36:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72c3e00700 10 client.15846 handle_command_reply: tid=0 2024-03-30T00:36:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72fcffcb80 1 client.15846 shutdown 2024-03-30T00:36:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1828700312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 msgr2=0x55e7db3b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b2d90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f72d00ca0b0 tx=0x7f72d0099830 comp rx=0 tx=0).stop 2024-03-30T00:36:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1828700312 shutdown_connections 2024-03-30T00:36:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7db3be600 0x55e7db3c0a20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3aa190 0x55e7db3b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3a89e0 0x55e7db3b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72c3e00700 0 client.15846 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:25.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1828700312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b2d90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1828700312 >> 172.21.15.99:0/1828700312 conn(0x55e7db3076a0 msgr2=0x55e7db3a73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:25.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1828700312 shutdown_connections 2024-03-30T00:36:25.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.379+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1828700312 wait complete. 2024-03-30T00:36:25.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.380+0000 7f72fcffcb80 20 client.15846 trim_cache size 0 max 16384 2024-03-30T00:36:25.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.408+0000 7f72fcffcb80 1 Processor -- start 2024-03-30T00:36:25.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.408+0000 7f72fcffcb80 1 -- start start 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.408+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.408+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a9340 0x55e7db3b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.408+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 0x55e7db3aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.408+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db3aa000 con 0x55e7db3a89e0 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.408+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db3b23c0 con 0x55e7db3a9340 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.408+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db39fa90 con 0x55e7db3aa190 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.408+0000 7f72e2800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.408+0000 7f72e2800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59318/0 (socket says 172.21.15.99:59318) 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.408+0000 7f72e2800700 1 -- 172.21.15.99:0/176970915 learned_addr learned my addr 172.21.15.99:0/176970915 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72e1e00700 1 --2- 172.21.15.99:0/176970915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a9340 0x55e7db3b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72e2800700 1 -- 172.21.15.99:0/176970915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 msgr2=0x55e7db3aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72e3200700 1 --2- 172.21.15.99:0/176970915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 0x55e7db3aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72e2800700 1 --2- 172.21.15.99:0/176970915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 0x55e7db3aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72e2800700 1 -- 172.21.15.99:0/176970915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a9340 msgr2=0x55e7db3b9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:25.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72e2800700 1 --2- 172.21.15.99:0/176970915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a9340 0x55e7db3b9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72e2800700 1 -- 172.21.15.99:0/176970915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7db2bfea0 con 0x55e7db3a89e0 2024-03-30T00:36:25.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72e1e00700 1 --2- 172.21.15.99:0/176970915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a9340 0x55e7db3b9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:25.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72e3200700 1 --2- 172.21.15.99:0/176970915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 0x55e7db3aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:25.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72e2800700 1 --2- 172.21.15.99:0/176970915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3b1d60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55e7db3a5050 tx=0x7f72d400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ee2df3465e3c80 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:25.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72cfe00700 1 -- 172.21.15.99:0/176970915 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d4016030 con 0x55e7db3a89e0 2024-03-30T00:36:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.409+0000 7f72cfe00700 1 -- 172.21.15.99:0/176970915 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72d4017040 con 0x55e7db3a89e0 2024-03-30T00:36:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.410+0000 7f72cfe00700 1 -- 172.21.15.99:0/176970915 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d4005280 con 0x55e7db3a89e0 2024-03-30T00:36:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.410+0000 7f72fcffcb80 1 -- 172.21.15.99:0/176970915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 msgr2=0x55e7db3b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.410+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/176970915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3b1d60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55e7db3a5050 tx=0x7f72d400b410 comp rx=0 tx=0).stop 2024-03-30T00:36:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.410+0000 7f72fcffcb80 1 -- 172.21.15.99:0/176970915 shutdown_connections 2024-03-30T00:36:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.410+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/176970915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 0x55e7db3aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.410+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/176970915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a89e0 0x55e7db3b1d60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.410+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/176970915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a9340 0x55e7db3b9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.410+0000 7f72fcffcb80 1 -- 172.21.15.99:0/176970915 >> 172.21.15.99:0/176970915 conn(0x55e7db3076a0 msgr2=0x55e7db39c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.410+0000 7f72fcffcb80 1 -- 172.21.15.99:0/176970915 shutdown_connections 2024-03-30T00:36:25.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.410+0000 7f72fcffcb80 1 -- 172.21.15.99:0/176970915 wait complete. 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.411+0000 7f72fcffcb80 1 -- start start 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.411+0000 7f72fcffcb80 1 Processor -- start 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.411+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a89e0 0x55e7db3b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.411+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.411+0000 7f72fcffcb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 0x55e7db3b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.411+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db412420 con 0x55e7db3a9340 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.411+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db412560 con 0x55e7db3a89e0 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.411+0000 7f72fcffcb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7db4126a0 con 0x55e7db3aa190 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72fcffcb80 10 client.0 _ll_register_callbacks cb 0x55e7db29f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59332/0 (socket says 172.21.15.99:59332) 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e1e00700 1 -- 172.21.15.99:0/1008669272 learned_addr learned my addr 172.21.15.99:0/1008669272 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e2800700 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a89e0 0x55e7db3b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e1e00700 1 -- 172.21.15.99:0/1008669272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 msgr2=0x55e7db3b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e3200700 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 0x55e7db3b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e1e00700 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 0x55e7db3b9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e1e00700 1 -- 172.21.15.99:0/1008669272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a89e0 msgr2=0x55e7db3b86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e1e00700 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a89e0 0x55e7db3b86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e1e00700 1 -- 172.21.15.99:0/1008669272 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d401fa60 con 0x55e7db3a9340 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e2800700 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a89e0 0x55e7db3b86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e3200700 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 0x55e7db3b9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72e1e00700 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b8bf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f72d0010a80 tx=0x7f72d0097fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead68821a9b04134 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.412+0000 7f72c3e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72c3e00700 1 -- 172.21.15.99:0/1008669272 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0014040 con 0x55e7db3a9340 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72c3e00700 1 -- 172.21.15.99:0/1008669272 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f72d00106d0 con 0x55e7db3a9340 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1008669272 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7db2ba380 con 0x55e7db3a9340 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72c3e00700 1 -- 172.21.15.99:0/1008669272 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72d00990f0 con 0x55e7db3a9340 2024-03-30T00:36:25.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72fcffcb80 10 client.15852 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:25.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1008669272 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e7db17fcd0 con 0x55e7db3a9340 2024-03-30T00:36:25.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72c3e00700 1 -- 172.21.15.99:0/1008669272 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e7db2ba380 con 0x55e7db3a9340 2024-03-30T00:36:25.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72c3e00700 1 -- 172.21.15.99:0/1008669272 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f72d00b2200 con 0x55e7db3a9340 2024-03-30T00:36:25.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72fcffcb80 10 client.15852 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:25.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72fcffcb80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:25.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72fcffcb80 20 client.15852 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:25.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7db413bf0 0x55e7db416010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72fcffcb80 4 client.15852 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:36:25.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.413+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1008669272 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e7db17dce0 con 0x55e7db413bf0 2024-03-30T00:36:25.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.414+0000 7f72e2800700 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7db413bf0 0x55e7db416010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72e2800700 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7db413bf0 0x55e7db416010 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=110f21e9acba9d25 server_cookie=d0cd95d80f1e3f1b in_seq=0 out_seq=0 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72c3e00700 10 client.15852 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72c3e00700 1 -- 172.21.15.99:0/1008669272 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e7db17dce0 con 0x55e7db413bf0 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72c3e00700 10 client.15852 handle_command_reply: tid=0 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72fcffcb80 1 client.15852 shutdown 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1008669272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 msgr2=0x55e7db3b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b8bf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f72d0010a80 tx=0x7f72d0097fb0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1008669272 shutdown_connections 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55e7db413bf0 0x55e7db416010 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e7db3aa190 0x55e7db3b9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e7db3a89e0 0x55e7db3b86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72fcffcb80 1 --2- 172.21.15.99:0/1008669272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7db3a9340 0x55e7db3b8bf0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72c3e00700 0 client.15852 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:25.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1008669272 >> 172.21.15.99:0/1008669272 conn(0x55e7db3076a0 msgr2=0x55e7db39c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:25.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1008669272 shutdown_connections 2024-03-30T00:36:25.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.415+0000 7f72fcffcb80 1 -- 172.21.15.99:0/1008669272 wait complete. 2024-03-30T00:36:25.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.416+0000 7f72fcffcb80 20 client.15852 trim_cache size 0 max 16384 2024-03-30T00:36:25.428 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:36:25.428 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "no active scrubs running", 2024-03-30T00:36:25.428 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": {} 2024-03-30T00:36:25.428 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:36:25.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.416+0000 7f72e8a00700 1 -- 172.21.15.99:0/1682880092 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72c80a1120 msgr2=0x7f72c80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.416+0000 7f72e8a00700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72c80a1120 0x7f72c80a35e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f72dc02a000 tx=0x7f72dc027000 comp rx=0 tx=0).stop 2024-03-30T00:36:25.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.416+0000 7f72e8a00700 1 -- 172.21.15.99:0/1682880092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e4101db0 msgr2=0x7f72e41992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.416+0000 7f72e8a00700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e4101db0 0x7f72e41992f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f72d000d950 tx=0x7f72d000dc60 comp rx=0 tx=0).stop 2024-03-30T00:36:25.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.416+0000 7f72e8a00700 1 -- 172.21.15.99:0/1682880092 shutdown_connections 2024-03-30T00:36:25.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.416+0000 7f72e8a00700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72e4102750 0x7f72e41a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.416+0000 7f72e8a00700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f72c80a1120 0x7f72c80a35e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.416+0000 7f72e8a00700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72e4100bc0 0x7f72e4198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.416+0000 7f72e8a00700 1 --2- 172.21.15.99:0/1682880092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e4101db0 0x7f72e41992f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.416+0000 7f72e8a00700 1 -- 172.21.15.99:0/1682880092 >> 172.21.15.99:0/1682880092 conn(0x7f72e40fc110 msgr2=0x7f72e4072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:25.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.417+0000 7f72e8a00700 1 -- 172.21.15.99:0/1682880092 shutdown_connections 2024-03-30T00:36:25.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.417+0000 7f72e8a00700 1 -- 172.21.15.99:0/1682880092 wait complete. 2024-03-30T00:36:25.442 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:36:25.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:36:25.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.652+0000 7fb87da00700 1 -- 172.21.15.99:0/2802821876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb878101e70 msgr2=0x7fb8781022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.652+0000 7fb87da00700 1 --2- 172.21.15.99:0/2802821876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb878101e70 0x7fb8781022d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb8640099c0 tx=0x7fb864009cd0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.652+0000 7fb87da00700 1 -- 172.21.15.99:0/2802821876 shutdown_connections 2024-03-30T00:36:25.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.652+0000 7fb87da00700 1 --2- 172.21.15.99:0/2802821876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb878102810 0x7fb8781090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.652+0000 7fb87da00700 1 --2- 172.21.15.99:0/2802821876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb878100c80 0x7fb8781010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.652+0000 7fb87da00700 1 --2- 172.21.15.99:0/2802821876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb878101e70 0x7fb8781022d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.652+0000 7fb87da00700 1 -- 172.21.15.99:0/2802821876 >> 172.21.15.99:0/2802821876 conn(0x7fb8780fc210 msgr2=0x7fb8780fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:25.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.652+0000 7fb87da00700 1 -- 172.21.15.99:0/2802821876 shutdown_connections 2024-03-30T00:36:25.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.653+0000 7fb87da00700 1 -- 172.21.15.99:0/2802821876 wait complete. 2024-03-30T00:36:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.654+0000 7fb87da00700 1 Processor -- start 2024-03-30T00:36:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.654+0000 7fb87da00700 1 -- start start 2024-03-30T00:36:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.654+0000 7fb87da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb878100c80 0x7fb878198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb87da00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb878101e70 0x7fb8781992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb87da00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb878102810 0x7fb8781a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb87da00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781a0850 con 0x7fb878101e70 2024-03-30T00:36:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb87da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781a09c0 con 0x7fb878100c80 2024-03-30T00:36:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb87da00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781a0b80 con 0x7fb878102810 2024-03-30T00:36:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb876000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb878101e70 0x7fb8781992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb876000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb878101e70 0x7fb8781992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59348/0 (socket says 172.21.15.99:59348) 2024-03-30T00:36:25.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb876000700 1 -- 172.21.15.99:0/2440906226 learned_addr learned my addr 172.21.15.99:0/2440906226 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:25.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb876a00700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb878100c80 0x7fb878198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb877400700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb878102810 0x7fb8781a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb876000700 1 -- 172.21.15.99:0/2440906226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb878102810 msgr2=0x7fb8781a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb876000700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb878102810 0x7fb8781a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb876000700 1 -- 172.21.15.99:0/2440906226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb878100c80 msgr2=0x7fb878198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb876000700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb878100c80 0x7fb878198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb876000700 1 -- 172.21.15.99:0/2440906226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8640095f0 con 0x7fb878101e70 2024-03-30T00:36:25.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.655+0000 7fb877400700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb878102810 0x7fb8781a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:25.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.656+0000 7fb876000700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb878101e70 0x7fb8781992a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb864009990 tx=0x7fb86400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:25.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.656+0000 7fb86f400700 1 -- 172.21.15.99:0/2440906226 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86401d070 con 0x7fb878101e70 2024-03-30T00:36:25.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.656+0000 7fb87da00700 1 -- 172.21.15.99:0/2440906226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8781a0e00 con 0x7fb878101e70 2024-03-30T00:36:25.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.656+0000 7fb86f400700 1 -- 172.21.15.99:0/2440906226 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb86400bd10 con 0x7fb878101e70 2024-03-30T00:36:25.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.656+0000 7fb86f400700 1 -- 172.21.15.99:0/2440906226 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864017650 con 0x7fb878101e70 2024-03-30T00:36:25.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.656+0000 7fb87da00700 1 -- 172.21.15.99:0/2440906226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8781a11d0 con 0x7fb878101e70 2024-03-30T00:36:25.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.657+0000 7fb87da00700 1 -- 172.21.15.99:0/2440906226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb878067180 con 0x7fb878101e70 2024-03-30T00:36:25.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.659+0000 7fb86f400700 1 -- 172.21.15.99:0/2440906226 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fb864022470 con 0x7fb878101e70 2024-03-30T00:36:25.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.661+0000 7fb86f400700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb85c0a1120 0x7fb85c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:25.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.661+0000 7fb86f400700 1 -- 172.21.15.99:0/2440906226 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fb8640ce0a0 con 0x7fb878101e70 2024-03-30T00:36:25.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.661+0000 7fb876a00700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb85c0a1120 0x7fb85c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:25.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.662+0000 7fb876a00700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb85c0a1120 0x7fb85c0a35e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb87819a1e0 tx=0x7fb86800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:25.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.664+0000 7fb86f400700 1 -- 172.21.15.99:0/2440906226 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb8640977e0 con 0x7fb878101e70 2024-03-30T00:36:25.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.963+0000 7fb87da00700 1 -- 172.21.15.99:0/2440906226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb87810c390 con 0x7fb878101e70 2024-03-30T00:36:25.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.964+0000 7fb86f400700 1 -- 172.21.15.99:0/2440906226 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10860 (secure 0 0 0) 0x7fb86409ae00 con 0x7fb878101e70 2024-03-30T00:36:25.972 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:36:25.972 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:34:39.366251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:36:25.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.967+0000 7fb87da00700 1 -- 172.21.15.99:0/2440906226 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb85c0a1120 msgr2=0x7fb85c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.967+0000 7fb87da00700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb85c0a1120 0x7fb85c0a35e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb87819a1e0 tx=0x7fb86800b410 comp rx=0 tx=0).stop 2024-03-30T00:36:25.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.967+0000 7fb87da00700 1 -- 172.21.15.99:0/2440906226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb878101e70 msgr2=0x7fb8781992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:25.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.967+0000 7fb87da00700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb878101e70 0x7fb8781992a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb864009990 tx=0x7fb86400fa00 comp rx=0 tx=0).stop 2024-03-30T00:36:25.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.967+0000 7fb87da00700 1 -- 172.21.15.99:0/2440906226 shutdown_connections 2024-03-30T00:36:25.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.967+0000 7fb87da00700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb878102810 0x7fb8781a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.967+0000 7fb87da00700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fb85c0a1120 0x7fb85c0a35e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.967+0000 7fb87da00700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb878100c80 0x7fb878198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.967+0000 7fb87da00700 1 --2- 172.21.15.99:0/2440906226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb878101e70 0x7fb8781992a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:25.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.967+0000 7fb87da00700 1 -- 172.21.15.99:0/2440906226 >> 172.21.15.99:0/2440906226 conn(0x7fb8780fc210 msgr2=0x7fb878107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:25.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.968+0000 7fb87da00700 1 -- 172.21.15.99:0/2440906226 shutdown_connections 2024-03-30T00:36:25.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:25.968+0000 7fb87da00700 1 -- 172.21.15.99:0/2440906226 wait complete. 2024-03-30T00:36:25.976 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 24 2024-03-30T00:36:25.994 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T00:36:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.200+0000 7f4235e00700 1 -- 172.21.15.99:0/2717885709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4230104180 msgr2=0x7f42301045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.200+0000 7f4235e00700 1 --2- 172.21.15.99:0/2717885709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4230104180 0x7f42301045e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f421c0099c0 tx=0x7f421c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.200+0000 7f4235e00700 1 -- 172.21.15.99:0/2717885709 shutdown_connections 2024-03-30T00:36:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.200+0000 7f4235e00700 1 --2- 172.21.15.99:0/2717885709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230104b20 0x7f423010b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.200+0000 7f4235e00700 1 --2- 172.21.15.99:0/2717885709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4230102f90 0x7f42301033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.200+0000 7f4235e00700 1 --2- 172.21.15.99:0/2717885709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4230104180 0x7f42301045e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.200+0000 7f4235e00700 1 -- 172.21.15.99:0/2717885709 >> 172.21.15.99:0/2717885709 conn(0x7f42300fe500 msgr2=0x7f4230100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.200+0000 7f4235e00700 1 -- 172.21.15.99:0/2717885709 shutdown_connections 2024-03-30T00:36:26.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.200+0000 7f4235e00700 1 -- 172.21.15.99:0/2717885709 wait complete. 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.201+0000 7f4235e00700 1 Processor -- start 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.201+0000 7f4235e00700 1 -- start start 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f4235e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4230102f90 0x7f4230198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f4235e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230104180 0x7f4230199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f4235e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4230104b20 0x7f42301a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f4235e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42301a0970 con 0x7f4230102f90 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f4235e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42301a0ab0 con 0x7f4230104b20 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f4235e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42301a0c40 con 0x7f4230104180 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f422ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4230102f90 0x7f4230198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f422ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4230102f90 0x7f4230198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59374/0 (socket says 172.21.15.99:59374) 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f422ea00700 1 -- 172.21.15.99:0/1655618528 learned_addr learned my addr 172.21.15.99:0/1655618528 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f422e000700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230104180 0x7f4230199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f422f400700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4230104b20 0x7f42301a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f422ea00700 1 -- 172.21.15.99:0/1655618528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230104180 msgr2=0x7f4230199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f422ea00700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230104180 0x7f4230199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f422ea00700 1 -- 172.21.15.99:0/1655618528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4230104b20 msgr2=0x7f42301a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f422ea00700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4230104b20 0x7f42301a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.202+0000 7f422ea00700 1 -- 172.21.15.99:0/1655618528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c0095f0 con 0x7f4230102f90 2024-03-30T00:36:26.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.203+0000 7f422ea00700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4230102f90 0x7f4230198e40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f422000d980 tx=0x7f422000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.203+0000 7f4234a00700 1 -- 172.21.15.99:0/1655618528 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422000bb60 con 0x7f4230102f90 2024-03-30T00:36:26.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.203+0000 7f4234a00700 1 -- 172.21.15.99:0/1655618528 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4220007620 con 0x7f4230102f90 2024-03-30T00:36:26.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.203+0000 7f4235e00700 1 -- 172.21.15.99:0/1655618528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42301a0f20 con 0x7f4230102f90 2024-03-30T00:36:26.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.203+0000 7f4234a00700 1 -- 172.21.15.99:0/1655618528 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422000f3c0 con 0x7f4230102f90 2024-03-30T00:36:26.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.203+0000 7f4235e00700 1 -- 172.21.15.99:0/1655618528 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42301a1350 con 0x7f4230102f90 2024-03-30T00:36:26.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.206+0000 7f4234a00700 1 -- 172.21.15.99:0/1655618528 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f4220010460 con 0x7f4230102f90 2024-03-30T00:36:26.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.208+0000 7f4234a00700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f42140a11a0 0x7f42140a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.208+0000 7f4234a00700 1 -- 172.21.15.99:0/1655618528 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f42200ccc50 con 0x7f4230102f90 2024-03-30T00:36:26.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.208+0000 7f422e000700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f42140a11a0 0x7f42140a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.209+0000 7f422e000700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f42140a11a0 0x7f42140a3660 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f421c00b580 tx=0x7f421c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.209+0000 7f42499c6b80 1 Processor -- start 2024-03-30T00:36:26.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.209+0000 7f42499c6b80 1 -- start start 2024-03-30T00:36:26.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.209+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cbf7d0 0x55cac4cbfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.209+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cc0130 0x55cac4cc09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.209+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0fd0 0x55cac4cc1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.209+0000 7f42499c6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4c17990 con 0x55cac4cbf7d0 2024-03-30T00:36:26.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.209+0000 7f42499c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4c15350 con 0x55cac4cc0130 2024-03-30T00:36:26.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.209+0000 7f42499c6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4c10c40 con 0x55cac4cc0fd0 2024-03-30T00:36:26.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.209+0000 7f422ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cbf7d0 0x55cac4cbfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.209+0000 7f422ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cbf7d0 0x55cac4cbfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59380/0 (socket says 172.21.15.99:59380) 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.210+0000 7f422ea00700 1 -- 172.21.15.99:0/2250615716 learned_addr learned my addr 172.21.15.99:0/2250615716 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.210+0000 7f422f400700 1 --2- 172.21.15.99:0/2250615716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0fd0 0x55cac4cc1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.210+0000 7f422e000700 1 --2- 172.21.15.99:0/2250615716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cc0130 0x55cac4cc09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.210+0000 7f422ea00700 1 -- 172.21.15.99:0/2250615716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0fd0 msgr2=0x55cac4cc1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.210+0000 7f422ea00700 1 --2- 172.21.15.99:0/2250615716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0fd0 0x55cac4cc1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.210+0000 7f422ea00700 1 -- 172.21.15.99:0/2250615716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cc0130 msgr2=0x55cac4cc09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.210+0000 7f422ea00700 1 --2- 172.21.15.99:0/2250615716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cc0130 0x55cac4cc09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.210+0000 7f422ea00700 1 -- 172.21.15.99:0/2250615716 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cac4a923b0 con 0x55cac4cbf7d0 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.210+0000 7f422e000700 1 --2- 172.21.15.99:0/2250615716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cc0130 0x55cac4cc09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.210+0000 7f422ea00700 1 --2- 172.21.15.99:0/2250615716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cbf7d0 0x55cac4cbfbf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f42200964c0 tx=0x7f42200999f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e9a956933602d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.210+0000 7f421be00700 1 -- 172.21.15.99:0/2250615716 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422001ae90 con 0x55cac4cbf7d0 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.211+0000 7f421be00700 1 -- 172.21.15.99:0/2250615716 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42200ade90 con 0x55cac4cbf7d0 2024-03-30T00:36:26.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.211+0000 7f421be00700 1 -- 172.21.15.99:0/2250615716 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220098360 con 0x55cac4cbf7d0 2024-03-30T00:36:26.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.211+0000 7f42499c6b80 1 -- 172.21.15.99:0/2250615716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cbf7d0 msgr2=0x55cac4cbfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.211+0000 7f42499c6b80 1 --2- 172.21.15.99:0/2250615716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cbf7d0 0x55cac4cbfbf0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f42200964c0 tx=0x7f42200999f0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.211+0000 7f42499c6b80 1 -- 172.21.15.99:0/2250615716 shutdown_connections 2024-03-30T00:36:26.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.211+0000 7f42499c6b80 1 --2- 172.21.15.99:0/2250615716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0fd0 0x55cac4cc1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.211+0000 7f42499c6b80 1 --2- 172.21.15.99:0/2250615716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cbf7d0 0x55cac4cbfbf0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.212+0000 7f42499c6b80 1 --2- 172.21.15.99:0/2250615716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cc0130 0x55cac4cc09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.212+0000 7f42499c6b80 1 -- 172.21.15.99:0/2250615716 >> 172.21.15.99:0/2250615716 conn(0x55cac4c1e6a0 msgr2=0x55cac4cba9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.212+0000 7f42499c6b80 1 -- 172.21.15.99:0/2250615716 shutdown_connections 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.212+0000 7f42499c6b80 1 -- 172.21.15.99:0/2250615716 wait complete. 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.212+0000 7f42499c6b80 1 -- start start 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.212+0000 7f42499c6b80 1 Processor -- start 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cbf7d0 0x55cac4cc9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0130 0x55cac4cc9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0fd0 0x55cac4cd0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f42499c6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4bacd80 con 0x55cac4cc0fd0 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f42499c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4baca20 con 0x55cac4cbf7d0 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f42499c6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4a946a0 con 0x55cac4cc0130 2024-03-30T00:36:26.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f42499c6b80 10 client.0 _ll_register_callbacks cb 0x55cac4bb6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f422f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0fd0 0x55cac4cd0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f422f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0fd0 0x55cac4cd0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59394/0 (socket says 172.21.15.99:59394) 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f422f400700 1 -- 172.21.15.99:0/3498877082 learned_addr learned my addr 172.21.15.99:0/3498877082 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f422e000700 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0130 0x55cac4cc9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.213+0000 7f422ea00700 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cbf7d0 0x55cac4cc9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f422f400700 1 -- 172.21.15.99:0/3498877082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0130 msgr2=0x55cac4cc9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f422f400700 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0130 0x55cac4cc9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f422f400700 1 -- 172.21.15.99:0/3498877082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cbf7d0 msgr2=0x55cac4cc9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f422f400700 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cbf7d0 0x55cac4cc9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f422f400700 1 -- 172.21.15.99:0/3498877082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42301a1350 con 0x55cac4cc0fd0 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f422f400700 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0fd0 0x55cac4cd0c00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4228007b60 tx=0x7f4228009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f001eff461b8d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f4218c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f4218c00700 1 -- 172.21.15.99:0/3498877082 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422800e030 con 0x55cac4cc0fd0 2024-03-30T00:36:26.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f4218c00700 1 -- 172.21.15.99:0/3498877082 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f4228015860 con 0x55cac4cc0fd0 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f42499c6b80 1 -- 172.21.15.99:0/3498877082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cac4bd1380 con 0x55cac4cc0fd0 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f4218c00700 1 -- 172.21.15.99:0/3498877082 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4228015a20 con 0x55cac4cc0fd0 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f42499c6b80 10 client.15870 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.214+0000 7f42499c6b80 1 -- 172.21.15.99:0/3498877082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cac4a3eb60 con 0x55cac4cc0fd0 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.215+0000 7f4218c00700 1 -- 172.21.15.99:0/3498877082 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cac4bd1380 con 0x55cac4cc0fd0 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.215+0000 7f4218c00700 1 -- 172.21.15.99:0/3498877082 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4228016cf0 con 0x55cac4cc0fd0 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.215+0000 7f42499c6b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.215+0000 7f42499c6b80 10 client.15870 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.215+0000 7f42499c6b80 20 client.15870 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.215+0000 7f42499c6b80 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55cac4cd53b0 0x55cac4cd77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.215+0000 7f42499c6b80 4 client.15870 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.215+0000 7f42499c6b80 1 -- 172.21.15.99:0/3498877082 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cac4c09700 con 0x55cac4cd53b0 2024-03-30T00:36:26.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.216+0000 7f422ea00700 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55cac4cd53b0 0x55cac4cd77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.216+0000 7f4218c00700 10 client.15870 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.216+0000 7f422ea00700 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55cac4cd53b0 0x55cac4cd77d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=654d4c73d679eab server_cookie=4b28220343043b8 in_seq=0 out_seq=0 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f4218c00700 1 -- 172.21.15.99:0/3498877082 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cac4c09700 con 0x55cac4cd53b0 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f4218c00700 10 client.15870 handle_command_reply: tid=0 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f42499c6b80 1 client.15870 shutdown 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f42499c6b80 1 -- 172.21.15.99:0/3498877082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0fd0 msgr2=0x55cac4cd0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f42499c6b80 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0fd0 0x55cac4cd0c00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4228007b60 tx=0x7f4228009510 comp rx=0 tx=0).stop 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f42499c6b80 1 -- 172.21.15.99:0/3498877082 shutdown_connections 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f42499c6b80 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55cac4cd53b0 0x55cac4cd77d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f42499c6b80 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0fd0 0x55cac4cd0c00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f42499c6b80 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cbf7d0 0x55cac4cc9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f4218c00700 0 client.15870 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f42499c6b80 1 --2- 172.21.15.99:0/3498877082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0130 0x55cac4cc9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.217+0000 7f42499c6b80 1 -- 172.21.15.99:0/3498877082 >> 172.21.15.99:0/3498877082 conn(0x55cac4c1e6a0 msgr2=0x55cac4cbe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.218+0000 7f42499c6b80 1 -- 172.21.15.99:0/3498877082 shutdown_connections 2024-03-30T00:36:26.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.218+0000 7f42499c6b80 1 -- 172.21.15.99:0/3498877082 wait complete. 2024-03-30T00:36:26.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.218+0000 7f42499c6b80 20 client.15870 trim_cache size 0 max 16384 2024-03-30T00:36:26.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.231+0000 7f42499c6b80 1 Processor -- start 2024-03-30T00:36:26.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.231+0000 7f42499c6b80 1 -- start start 2024-03-30T00:36:26.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cbf7d0 0x55cac4cc8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4cd0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cc0fd0 0x55cac4cc13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f42499c6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4cbb8f0 con 0x55cac4cc0130 2024-03-30T00:36:26.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f42499c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4cc91f0 con 0x55cac4cc0fd0 2024-03-30T00:36:26.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f42499c6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4cd1d80 con 0x55cac4cbf7d0 2024-03-30T00:36:26.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f422e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4cd0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f422e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4cd0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59404/0 (socket says 172.21.15.99:59404) 2024-03-30T00:36:26.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f422e000700 1 -- 172.21.15.99:0/760675325 learned_addr learned my addr 172.21.15.99:0/760675325 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f422f400700 1 --2- 172.21.15.99:0/760675325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cc0fd0 0x55cac4cc13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f422ea00700 1 --2- 172.21.15.99:0/760675325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cbf7d0 0x55cac4cc8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f422e000700 1 -- 172.21.15.99:0/760675325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cbf7d0 msgr2=0x55cac4cc8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f422e000700 1 --2- 172.21.15.99:0/760675325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cbf7d0 0x55cac4cc8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f422e000700 1 -- 172.21.15.99:0/760675325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cc0fd0 msgr2=0x55cac4cc13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f422e000700 1 --2- 172.21.15.99:0/760675325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cc0fd0 0x55cac4cc13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.232+0000 7f422e000700 1 -- 172.21.15.99:0/760675325 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cac4bd6ea0 con 0x55cac4cc0130 2024-03-30T00:36:26.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.233+0000 7f422e000700 1 --2- 172.21.15.99:0/760675325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4cd0b20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f421c009fd0 tx=0x7f421c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910783a0c489cfd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.233+0000 7f4225200700 1 -- 172.21.15.99:0/760675325 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c028070 con 0x55cac4cc0130 2024-03-30T00:36:26.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.233+0000 7f4225200700 1 -- 172.21.15.99:0/760675325 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f421c029040 con 0x55cac4cc0130 2024-03-30T00:36:26.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.233+0000 7f4225200700 1 -- 172.21.15.99:0/760675325 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c01d040 con 0x55cac4cc0130 2024-03-30T00:36:26.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.234+0000 7f42499c6b80 1 -- 172.21.15.99:0/760675325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 msgr2=0x55cac4cd0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.234+0000 7f42499c6b80 1 --2- 172.21.15.99:0/760675325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4cd0b20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f421c009fd0 tx=0x7f421c01e000 comp rx=0 tx=0).stop 2024-03-30T00:36:26.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.234+0000 7f42499c6b80 1 -- 172.21.15.99:0/760675325 shutdown_connections 2024-03-30T00:36:26.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.234+0000 7f42499c6b80 1 --2- 172.21.15.99:0/760675325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cc0fd0 0x55cac4cc13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.234+0000 7f42499c6b80 1 --2- 172.21.15.99:0/760675325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cbf7d0 0x55cac4cc8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.234+0000 7f42499c6b80 1 --2- 172.21.15.99:0/760675325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4cd0b20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.234+0000 7f42499c6b80 1 -- 172.21.15.99:0/760675325 >> 172.21.15.99:0/760675325 conn(0x55cac4c1e6a0 msgr2=0x55cac4cc7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.234+0000 7f42499c6b80 1 -- 172.21.15.99:0/760675325 shutdown_connections 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.234+0000 7f42499c6b80 1 -- 172.21.15.99:0/760675325 wait complete. 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.235+0000 7f42499c6b80 1 -- start start 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.235+0000 7f42499c6b80 1 Processor -- start 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.235+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cbf7d0 0x55cac4cd07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.235+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4ccee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.235+0000 7f42499c6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0fd0 0x55cac4ccf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.235+0000 7f42499c6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4ccf8c0 con 0x55cac4cc0130 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.235+0000 7f42499c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4ccfa30 con 0x55cac4cbf7d0 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.235+0000 7f42499c6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cac4ccfbf0 con 0x55cac4cc0fd0 2024-03-30T00:36:26.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.235+0000 7f42499c6b80 10 client.0 _ll_register_callbacks cb 0x55cac4bb6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4ccee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4ccee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59420/0 (socket says 172.21.15.99:59420) 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422e000700 1 -- 172.21.15.99:0/1071174897 learned_addr learned my addr 172.21.15.99:0/1071174897 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422ea00700 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cbf7d0 0x55cac4cd07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422e000700 1 -- 172.21.15.99:0/1071174897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0fd0 msgr2=0x55cac4ccf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422f400700 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0fd0 0x55cac4ccf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422e000700 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0fd0 0x55cac4ccf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422e000700 1 -- 172.21.15.99:0/1071174897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cbf7d0 msgr2=0x55cac4cd07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422e000700 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cbf7d0 0x55cac4cd07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422e000700 1 -- 172.21.15.99:0/1071174897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423010bd20 con 0x55cac4cc0130 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422f400700 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0fd0 0x55cac4ccf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.236+0000 7f422ea00700 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cbf7d0 0x55cac4cd07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.237+0000 7f422e000700 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4ccee40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f421c009990 tx=0x7f421c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c1fd57e6baab16 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.237+0000 7f4218c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.237+0000 7f4218c00700 1 -- 172.21.15.99:0/1071174897 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c028050 con 0x55cac4cc0130 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.237+0000 7f42499c6b80 1 -- 172.21.15.99:0/1071174897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42200ade90 con 0x55cac4cc0130 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.237+0000 7f4218c00700 1 -- 172.21.15.99:0/1071174897 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f421c02f440 con 0x55cac4cc0130 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.237+0000 7f4218c00700 1 -- 172.21.15.99:0/1071174897 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f421c01d040 con 0x55cac4cc0130 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.237+0000 7f42499c6b80 10 client.15876 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.237+0000 7f42499c6b80 1 -- 172.21.15.99:0/1071174897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cac4a96cd0 con 0x55cac4cc0130 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.237+0000 7f4218c00700 1 -- 172.21.15.99:0/1071174897 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42200ade90 con 0x55cac4cc0130 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.238+0000 7f4218c00700 1 -- 172.21.15.99:0/1071174897 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f421c004890 con 0x55cac4cc0130 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.238+0000 7f42499c6b80 10 client.15876 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.238+0000 7f42499c6b80 10 client.15876 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.238+0000 7f42499c6b80 20 client.15876 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.238+0000 7f42499c6b80 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55cac4d2a4d0 0x55cac4d2c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.238+0000 7f42499c6b80 4 client.15876 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:36:26.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.238+0000 7f42499c6b80 1 -- 172.21.15.99:0/1071174897 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cac4a5fcf0 con 0x55cac4d2a4d0 2024-03-30T00:36:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.238+0000 7f422ea00700 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55cac4d2a4d0 0x55cac4d2c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.239+0000 7f422ea00700 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55cac4d2a4d0 0x55cac4d2c8c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bdefd7092260d544 server_cookie=e38642ab78f0c398 in_seq=0 out_seq=0 2024-03-30T00:36:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.239+0000 7f4218c00700 10 client.15876 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:36:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.239+0000 7f4218c00700 1 -- 172.21.15.99:0/1071174897 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cac4a5fcf0 con 0x55cac4d2a4d0 2024-03-30T00:36:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.239+0000 7f4218c00700 10 client.15876 handle_command_reply: tid=0 2024-03-30T00:36:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.239+0000 7f42499c6b80 1 client.15876 shutdown 2024-03-30T00:36:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.239+0000 7f42499c6b80 1 -- 172.21.15.99:0/1071174897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 msgr2=0x55cac4ccee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.239+0000 7f42499c6b80 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4ccee40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f421c009990 tx=0x7f421c00be00 comp rx=0 tx=0).stop 2024-03-30T00:36:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f42499c6b80 1 -- 172.21.15.99:0/1071174897 shutdown_connections 2024-03-30T00:36:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f42499c6b80 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55cac4d2a4d0 0x55cac4d2c8c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f42499c6b80 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cac4cc0fd0 0x55cac4ccf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f42499c6b80 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cac4cbf7d0 0x55cac4cd07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f42499c6b80 1 --2- 172.21.15.99:0/1071174897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac4cc0130 0x55cac4ccee40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f4218c00700 0 client.15876 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:36:26.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f42499c6b80 1 -- 172.21.15.99:0/1071174897 >> 172.21.15.99:0/1071174897 conn(0x55cac4c1e6a0 msgr2=0x55cac4cc7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f42499c6b80 1 -- 172.21.15.99:0/1071174897 shutdown_connections 2024-03-30T00:36:26.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f42499c6b80 1 -- 172.21.15.99:0/1071174897 wait complete. 2024-03-30T00:36:26.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f42499c6b80 20 client.15876 trim_cache size 0 max 16384 2024-03-30T00:36:26.251 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:36:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f4235e00700 1 -- 172.21.15.99:0/1655618528 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f42140a11a0 msgr2=0x7f42140a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f4235e00700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f42140a11a0 0x7f42140a3660 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f421c00b580 tx=0x7f421c01a040 comp rx=0 tx=0).stop 2024-03-30T00:36:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f4235e00700 1 -- 172.21.15.99:0/1655618528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4230102f90 msgr2=0x7f4230198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.240+0000 7f4235e00700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4230102f90 0x7f4230198e40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f422000d980 tx=0x7f422000dd40 comp rx=0 tx=0).stop 2024-03-30T00:36:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.241+0000 7f4235e00700 1 -- 172.21.15.99:0/1655618528 shutdown_connections 2024-03-30T00:36:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.241+0000 7f4235e00700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f42140a11a0 0x7f42140a3660 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.241+0000 7f4235e00700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4230104b20 0x7f42301a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.241+0000 7f4235e00700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4230102f90 0x7f4230198e40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.241+0000 7f4235e00700 1 --2- 172.21.15.99:0/1655618528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4230104180 0x7f4230199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.241+0000 7f4235e00700 1 -- 172.21.15.99:0/1655618528 >> 172.21.15.99:0/1655618528 conn(0x7f42300fe500 msgr2=0x7f42301074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.241+0000 7f4235e00700 1 -- 172.21.15.99:0/1655618528 shutdown_connections 2024-03-30T00:36:26.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.241+0000 7f4235e00700 1 -- 172.21.15.99:0/1655618528 wait complete. 2024-03-30T00:36:26.253 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:26 smithi099 ceph-mon[38079]: pgmap v355: 97 pgs: 97 active+clean; 177 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-03-30T00:36:26.253 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:26 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2440906226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:36:26.270 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T00:36:26.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.478+0000 7f8098e00700 1 -- 172.21.15.99:0/3454100815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094073810 msgr2=0x7f8094073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.478+0000 7f8098e00700 1 --2- 172.21.15.99:0/3454100815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094073810 0x7f8094073c90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8080009990 tx=0x7f8080009ca0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.478+0000 7f8098e00700 1 -- 172.21.15.99:0/3454100815 shutdown_connections 2024-03-30T00:36:26.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.478+0000 7f8098e00700 1 --2- 172.21.15.99:0/3454100815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8094108c40 0x7f809410b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.478+0000 7f8098e00700 1 --2- 172.21.15.99:0/3454100815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8094074df0 0x7f8094073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.478+0000 7f8098e00700 1 --2- 172.21.15.99:0/3454100815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094073810 0x7f8094073c90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.478+0000 7f8098e00700 1 -- 172.21.15.99:0/3454100815 >> 172.21.15.99:0/3454100815 conn(0x7f80940fc2f0 msgr2=0x7f80940fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.478+0000 7f8098e00700 1 -- 172.21.15.99:0/3454100815 shutdown_connections 2024-03-30T00:36:26.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.479+0000 7f8098e00700 1 -- 172.21.15.99:0/3454100815 wait complete. 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.480+0000 7f8098e00700 1 Processor -- start 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.480+0000 7f8098e00700 1 -- start start 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.480+0000 7f8098e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8094073810 0x7f8094198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8098e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094074df0 0x7f80941993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8098e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8094108c40 0x7f80941a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8098e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80941a09e0 con 0x7f8094074df0 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8098e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80941a0b20 con 0x7f8094108c40 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8098e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80941a0cb0 con 0x7f8094073810 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8091400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094074df0 0x7f80941993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8091e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8094073810 0x7f8094198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8091400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094074df0 0x7f80941993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59442/0 (socket says 172.21.15.99:59442) 2024-03-30T00:36:26.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8091400700 1 -- 172.21.15.99:0/311251974 learned_addr learned my addr 172.21.15.99:0/311251974 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8092800700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8094108c40 0x7f80941a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8091e00700 1 -- 172.21.15.99:0/311251974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8094108c40 msgr2=0x7f80941a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8091e00700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8094108c40 0x7f80941a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8091e00700 1 -- 172.21.15.99:0/311251974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094074df0 msgr2=0x7f80941993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8091e00700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094074df0 0x7f80941993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8091e00700 1 -- 172.21.15.99:0/311251974 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800095f0 con 0x7f8094073810 2024-03-30T00:36:26.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8092800700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8094108c40 0x7f80941a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:26.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.481+0000 7f8091400700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094074df0 0x7f80941993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:26.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.482+0000 7f8091e00700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8094073810 0x7f8094198eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f808400e9f0 tx=0x7f808400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.482+0000 7f808aa00700 1 -- 172.21.15.99:0/311251974 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808400eee0 con 0x7f8094073810 2024-03-30T00:36:26.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.482+0000 7f808aa00700 1 -- 172.21.15.99:0/311251974 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f808400cbd0 con 0x7f8094073810 2024-03-30T00:36:26.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.482+0000 7f8098e00700 1 -- 172.21.15.99:0/311251974 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80941a0f90 con 0x7f8094073810 2024-03-30T00:36:26.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.482+0000 7f808aa00700 1 -- 172.21.15.99:0/311251974 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084018680 con 0x7f8094073810 2024-03-30T00:36:26.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.482+0000 7f8098e00700 1 -- 172.21.15.99:0/311251974 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80941a13b0 con 0x7f8094073810 2024-03-30T00:36:26.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.485+0000 7f808aa00700 1 -- 172.21.15.99:0/311251974 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f808400cd40 con 0x7f8094073810 2024-03-30T00:36:26.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.487+0000 7f80ac8f0b80 1 Processor -- start 2024-03-30T00:36:26.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.487+0000 7f80ac8f0b80 1 -- start start 2024-03-30T00:36:26.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.487+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613cf7d0 0x5617613cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.487+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0130 0x5617613d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.487+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613d0fd0 0x5617613d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.487+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x561761327990 con 0x5617613d0130 2024-03-30T00:36:26.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.487+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561761325350 con 0x5617613cf7d0 2024-03-30T00:36:26.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.487+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561761320c40 con 0x5617613d0fd0 2024-03-30T00:36:26.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f808aa00700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f80780a1120 0x7f80780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8091e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0130 0x5617613d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8091e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0130 0x5617613d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59446/0 (socket says 172.21.15.99:59446) 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8091e00700 1 -- 172.21.15.99:0/1737076511 learned_addr learned my addr 172.21.15.99:0/1737076511 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8092800700 1 --2- 172.21.15.99:0/1737076511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613cf7d0 0x5617613cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8091400700 1 --2- 172.21.15.99:0/1737076511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613d0fd0 0x5617613d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8092800700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f80780a1120 0x7f80780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f808aa00700 1 -- 172.21.15.99:0/311251974 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f8084014070 con 0x7f8094073810 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8091e00700 1 -- 172.21.15.99:0/1737076511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613d0fd0 msgr2=0x5617613d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8091e00700 1 --2- 172.21.15.99:0/1737076511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613d0fd0 0x5617613d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8091e00700 1 -- 172.21.15.99:0/1737076511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613cf7d0 msgr2=0x5617613cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8091e00700 1 --2- 172.21.15.99:0/1737076511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613cf7d0 0x5617613cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8091e00700 1 -- 172.21.15.99:0/1737076511 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617611a23b0 con 0x5617613d0130 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.488+0000 7f8091e00700 1 --2- 172.21.15.99:0/1737076511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0130 0x5617613d09f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f80840ca530 tx=0x7f8084009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f846f43a166a13 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.489+0000 7f807f400700 1 -- 172.21.15.99:0/1737076511 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80840a2390 con 0x5617613d0130 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.489+0000 7f8092800700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f80780a1120 0x7f80780a35e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f808c0060b0 tx=0x7f808c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.489+0000 7f807f400700 1 -- 172.21.15.99:0/1737076511 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8084099f10 con 0x5617613d0130 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.489+0000 7f807f400700 1 -- 172.21.15.99:0/1737076511 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80840a1f00 con 0x5617613d0130 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.490+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/1737076511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0130 msgr2=0x5617613d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.490+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/1737076511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0130 0x5617613d09f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f80840ca530 tx=0x7f8084009790 comp rx=0 tx=0).stop 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.490+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/1737076511 shutdown_connections 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.490+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/1737076511 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613d0fd0 0x5617613d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.490+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/1737076511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613cf7d0 0x5617613cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.490+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/1737076511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0130 0x5617613d09f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.490+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/1737076511 >> 172.21.15.99:0/1737076511 conn(0x56176132e6a0 msgr2=0x5617613ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.490+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/1737076511 shutdown_connections 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.490+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/1737076511 wait complete. 2024-03-30T00:36:26.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.490+0000 7f80ac8f0b80 1 -- start start 2024-03-30T00:36:26.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.491+0000 7f80ac8f0b80 1 Processor -- start 2024-03-30T00:36:26.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.491+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613d9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.491+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613d9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.491+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0fd0 0x5617613e0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.491+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617612bcd80 con 0x5617613d0fd0 2024-03-30T00:36:26.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.491+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617612bca20 con 0x5617613d0130 2024-03-30T00:36:26.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.491+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617611a46a0 con 0x5617613cf7d0 2024-03-30T00:36:26.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.492+0000 7f80ac8f0b80 10 client.0 _ll_register_callbacks cb 0x5617612c6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:26.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.492+0000 7f8091e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613d9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.492+0000 7f8091e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613d9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:59418/0 (socket says 172.21.15.99:59418) 2024-03-30T00:36:26.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.492+0000 7f8091e00700 1 -- 172.21.15.99:0/4281121556 learned_addr learned my addr 172.21.15.99:0/4281121556 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.492+0000 7f8092800700 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0fd0 0x5617613e0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.492+0000 7f8091400700 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613d9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.492+0000 7f8091e00700 1 -- 172.21.15.99:0/4281121556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 msgr2=0x5617613d9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.492+0000 7f8091e00700 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613d9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.492+0000 7f8091e00700 1 -- 172.21.15.99:0/4281121556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0fd0 msgr2=0x5617613e0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.492+0000 7f8091e00700 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0fd0 0x5617613e0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.492+0000 7f8091e00700 1 -- 172.21.15.99:0/4281121556 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80941a13b0 con 0x5617613cf7d0 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.493+0000 7f8091e00700 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613d9610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f80840ce040 tx=0x7f80840a2130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a68238bb44aecf0a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.493+0000 7f8073e00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.493+0000 7f8073e00700 1 -- 172.21.15.99:0/4281121556 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084099ce0 con 0x5617613cf7d0 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.493+0000 7f8073e00700 1 -- 172.21.15.99:0/4281121556 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f8084098050 con 0x5617613cf7d0 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.493+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/4281121556 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617612e1380 con 0x5617613cf7d0 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.493+0000 7f8073e00700 1 -- 172.21.15.99:0/4281121556 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80840a2390 con 0x5617613cf7d0 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.493+0000 7f80ac8f0b80 10 client.25597 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.493+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/4281121556 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56176114eb60 con 0x5617613cf7d0 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.493+0000 7f8073e00700 1 -- 172.21.15.99:0/4281121556 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8084063020 con 0x5617613cf7d0 2024-03-30T00:36:26.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.494+0000 7f8073e00700 1 -- 172.21.15.99:0/4281121556 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8084098e70 con 0x5617613cf7d0 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.494+0000 7f80ac8f0b80 10 client.25597 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.494+0000 7f80ac8f0b80 10 client.25597 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.494+0000 7f80ac8f0b80 20 client.25597 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.494+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5617613e5120 0x5617613e7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.494+0000 7f80ac8f0b80 4 client.25597 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.494+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/4281121556 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561761319700 con 0x5617613e5120 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.494+0000 7f8091400700 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5617613e5120 0x5617613e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.494+0000 7f8091400700 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5617613e5120 0x5617613e7540 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11bfb6013b8db4cb server_cookie=5e33b733b1b3f842 in_seq=0 out_seq=0 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.494+0000 7f8073e00700 10 client.25597 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.495+0000 7f8073e00700 1 -- 172.21.15.99:0/4281121556 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561761319700 con 0x5617613e5120 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.495+0000 7f8073e00700 10 client.25597 handle_command_reply: tid=0 2024-03-30T00:36:26.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.495+0000 7f80ac8f0b80 1 client.25597 shutdown 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.495+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/4281121556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 msgr2=0x5617613d9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.495+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613d9610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f80840ce040 tx=0x7f80840a2130 comp rx=0 tx=0).stop 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.496+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/4281121556 shutdown_connections 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.496+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5617613e5120 0x5617613e7540 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.496+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0fd0 0x5617613e0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.496+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613d9610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.496+0000 7f8073e00700 0 client.25597 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.496+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/4281121556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613d9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.496+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/4281121556 >> 172.21.15.99:0/4281121556 conn(0x56176132e6a0 msgr2=0x5617613ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.496+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/4281121556 shutdown_connections 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.496+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/4281121556 wait complete. 2024-03-30T00:36:26.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.496+0000 7f80ac8f0b80 20 client.25597 trim_cache size 0 max 16384 2024-03-30T00:36:26.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f80ac8f0b80 1 Processor -- start 2024-03-30T00:36:26.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f80ac8f0b80 1 -- start start 2024-03-30T00:36:26.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613cf7d0 0x5617613e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613e0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613d0fd0 0x5617613c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617613da650 con 0x5617613cf7d0 2024-03-30T00:36:26.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617613da800 con 0x5617613d0130 2024-03-30T00:36:26.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617613e7ba0 con 0x5617613d0fd0 2024-03-30T00:36:26.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f8091e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613cf7d0 0x5617613e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f8091e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613cf7d0 0x5617613e1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59466/0 (socket says 172.21.15.99:59466) 2024-03-30T00:36:26.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f8091e00700 1 -- 172.21.15.99:0/737450520 learned_addr learned my addr 172.21.15.99:0/737450520 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f8092800700 1 --2- 172.21.15.99:0/737450520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613d0fd0 0x5617613c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.517+0000 7f8091400700 1 --2- 172.21.15.99:0/737450520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613e0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.518+0000 7f8091e00700 1 -- 172.21.15.99:0/737450520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613d0fd0 msgr2=0x5617613c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.518+0000 7f8091e00700 1 --2- 172.21.15.99:0/737450520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613d0fd0 0x5617613c35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.518+0000 7f8091e00700 1 -- 172.21.15.99:0/737450520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 msgr2=0x5617613e0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.518+0000 7f8091e00700 1 --2- 172.21.15.99:0/737450520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613e0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.518+0000 7f8091e00700 1 -- 172.21.15.99:0/737450520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617612e6ea0 con 0x5617613cf7d0 2024-03-30T00:36:26.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.518+0000 7f8091400700 1 --2- 172.21.15.99:0/737450520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613e0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.518+0000 7f8091e00700 1 --2- 172.21.15.99:0/737450520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613cf7d0 0x5617613e1240 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f808400c5b0 tx=0x7f808400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671774994d73b137 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.518+0000 7f807fe00700 1 -- 172.21.15.99:0/737450520 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80840a21a0 con 0x5617613cf7d0 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.518+0000 7f807fe00700 1 -- 172.21.15.99:0/737450520 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8084063070 con 0x5617613cf7d0 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.518+0000 7f807fe00700 1 -- 172.21.15.99:0/737450520 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084098e70 con 0x5617613cf7d0 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.519+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/737450520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613cf7d0 msgr2=0x5617613e1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.519+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/737450520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613cf7d0 0x5617613e1240 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f808400c5b0 tx=0x7f808400e940 comp rx=0 tx=0).stop 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.519+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/737450520 shutdown_connections 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.519+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/737450520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613d0fd0 0x5617613c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.519+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/737450520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613cf7d0 0x5617613e1240 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.519+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/737450520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613e0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.519+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/737450520 >> 172.21.15.99:0/737450520 conn(0x56176132e6a0 msgr2=0x5617613d8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.519+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/737450520 shutdown_connections 2024-03-30T00:36:26.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.519+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/737450520 wait complete. 2024-03-30T00:36:26.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.519+0000 7f80ac8f0b80 1 -- start start 2024-03-30T00:36:26.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.520+0000 7f80ac8f0b80 1 Processor -- start 2024-03-30T00:36:26.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.520+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613dea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.520+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.520+0000 7f80ac8f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0fd0 0x5617613dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.520+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617613e0400 con 0x5617613d0fd0 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.520+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617613e0570 con 0x5617613d0130 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.520+0000 7f80ac8f0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617613e0730 con 0x5617613cf7d0 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f80ac8f0b80 10 client.0 _ll_register_callbacks cb 0x5617612c6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8091e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613dea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8092800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0fd0 0x5617613dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8091e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613dea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:59426/0 (socket says 172.21.15.99:59426) 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8091e00700 1 -- 172.21.15.99:0/405917974 learned_addr learned my addr 172.21.15.99:0/405917974 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8091400700 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8091e00700 1 -- 172.21.15.99:0/405917974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 msgr2=0x5617613def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8091e00700 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613def50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8091e00700 1 -- 172.21.15.99:0/405917974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0fd0 msgr2=0x5617613dfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8091e00700 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0fd0 0x5617613dfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8091e00700 1 -- 172.21.15.99:0/405917974 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56176114eb60 con 0x5617613cf7d0 2024-03-30T00:36:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8091e00700 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613dea10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8084099800 tx=0x7f80840092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee74e41dbe88cfc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.521+0000 7f8073e00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:36:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f8073e00700 1 -- 172.21.15.99:0/405917974 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084099ce0 con 0x5617613cf7d0 2024-03-30T00:36:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f8073e00700 1 -- 172.21.15.99:0/405917974 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f8084064080 con 0x5617613cf7d0 2024-03-30T00:36:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/405917974 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8084099f10 con 0x5617613cf7d0 2024-03-30T00:36:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f8073e00700 1 -- 172.21.15.99:0/405917974 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80840a2390 con 0x5617613cf7d0 2024-03-30T00:36:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f80ac8f0b80 10 client.25603 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/405917974 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5617611a6cd0 con 0x5617613cf7d0 2024-03-30T00:36:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f8073e00700 1 -- 172.21.15.99:0/405917974 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8084099f10 con 0x5617613cf7d0 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f8073e00700 1 -- 172.21.15.99:0/405917974 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f80840cab10 con 0x5617613cf7d0 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f80ac8f0b80 10 client.25603 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f80ac8f0b80 10 client.25603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f80ac8f0b80 20 client.25603 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5617614399e0 0x56176143bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f80ac8f0b80 4 client.25603 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.522+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/405917974 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56176116fcf0 con 0x5617614399e0 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.523+0000 7f8091400700 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5617614399e0 0x56176143bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.523+0000 7f8073e00700 10 client.25603 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.523+0000 7f8091400700 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5617614399e0 0x56176143bdd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2a1e03b188c5a30 server_cookie=69c0b397ad9c2503 in_seq=0 out_seq=0 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.523+0000 7f8073e00700 1 -- 172.21.15.99:0/405917974 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56176116fcf0 con 0x5617614399e0 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.523+0000 7f8073e00700 10 client.25603 handle_command_reply: tid=0 2024-03-30T00:36:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.523+0000 7f80ac8f0b80 1 client.25603 shutdown 2024-03-30T00:36:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/405917974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 msgr2=0x5617613dea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613dea10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8084099800 tx=0x7f80840092d0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/405917974 shutdown_connections 2024-03-30T00:36:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5617614399e0 0x56176143bdd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617613d0fd0 0x5617613dfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f8073e00700 0 client.25603 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:36:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617613cf7d0 0x5617613dea10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f80ac8f0b80 1 --2- 172.21.15.99:0/405917974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5617613d0130 0x5617613def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/405917974 >> 172.21.15.99:0/405917974 conn(0x56176132e6a0 msgr2=0x5617613c35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/405917974 shutdown_connections 2024-03-30T00:36:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f80ac8f0b80 1 -- 172.21.15.99:0/405917974 wait complete. 2024-03-30T00:36:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.524+0000 7f80ac8f0b80 20 client.25603 trim_cache size 0 max 16384 2024-03-30T00:36:26.537 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:36:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 -- 172.21.15.99:0/311251974 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f80780a1120 msgr2=0x7f80780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f80780a1120 0x7f80780a35e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f808c0060b0 tx=0x7f808c00b040 comp rx=0 tx=0).stop 2024-03-30T00:36:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 -- 172.21.15.99:0/311251974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8094073810 msgr2=0x7f8094198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8094073810 0x7f8094198eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f808400e9f0 tx=0x7f808400ed00 comp rx=0 tx=0).stop 2024-03-30T00:36:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 -- 172.21.15.99:0/311251974 shutdown_connections 2024-03-30T00:36:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f80780a1120 0x7f80780a35e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8094108c40 0x7f80941a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8094073810 0x7f8094198eb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 --2- 172.21.15.99:0/311251974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8094074df0 0x7f80941993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 -- 172.21.15.99:0/311251974 >> 172.21.15.99:0/311251974 conn(0x7f80940fc2f0 msgr2=0x7f8094104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 -- 172.21.15.99:0/311251974 shutdown_connections 2024-03-30T00:36:26.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.525+0000 7f8098e00700 1 -- 172.21.15.99:0/311251974 wait complete. 2024-03-30T00:36:26.553 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T00:36:26.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:26 smithi142 ceph-mon[41779]: pgmap v355: 97 pgs: 97 active+clean; 177 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-03-30T00:36:26.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:26 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2440906226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:36:26.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:26 smithi161 ceph-mon[42221]: pgmap v355: 97 pgs: 97 active+clean; 177 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 MiB/s wr, 14 op/s 2024-03-30T00:36:26.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:26 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2440906226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:36:26.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.773+0000 7fc183200700 1 -- 172.21.15.99:0/3446290217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184103fe0 msgr2=0x7fc184104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.773+0000 7fc183200700 1 --2- 172.21.15.99:0/3446290217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184103fe0 0x7fc184104440 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc1700099c0 tx=0x7fc170009cd0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.774+0000 7fc183200700 1 -- 172.21.15.99:0/3446290217 shutdown_connections 2024-03-30T00:36:26.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.774+0000 7fc183200700 1 --2- 172.21.15.99:0/3446290217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc184104980 0x7fc18410b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.774+0000 7fc183200700 1 --2- 172.21.15.99:0/3446290217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184102df0 0x7fc184103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.774+0000 7fc183200700 1 --2- 172.21.15.99:0/3446290217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184103fe0 0x7fc184104440 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.774+0000 7fc183200700 1 -- 172.21.15.99:0/3446290217 >> 172.21.15.99:0/3446290217 conn(0x7fc1840fe380 msgr2=0x7fc1841007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.774+0000 7fc183200700 1 -- 172.21.15.99:0/3446290217 shutdown_connections 2024-03-30T00:36:26.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.774+0000 7fc183200700 1 -- 172.21.15.99:0/3446290217 wait complete. 2024-03-30T00:36:26.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.775+0000 7fc183200700 1 Processor -- start 2024-03-30T00:36:26.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.775+0000 7fc183200700 1 -- start start 2024-03-30T00:36:26.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.775+0000 7fc183200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc184102df0 0x7fc184198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.775+0000 7fc183200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184103fe0 0x7fc1841990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.775+0000 7fc183200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184104980 0x7fc1841a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.775+0000 7fc183200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1841a06d0 con 0x7fc184104980 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.775+0000 7fc183200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1841a0810 con 0x7fc184102df0 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.775+0000 7fc183200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1841a09a0 con 0x7fc184103fe0 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc181e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc184102df0 0x7fc184198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc182800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184104980 0x7fc1841a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc182800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184104980 0x7fc1841a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59476/0 (socket says 172.21.15.99:59476) 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc181e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc184102df0 0x7fc184198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:56048/0 (socket says 172.21.15.99:56048) 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc182800700 1 -- 172.21.15.99:0/2082065466 learned_addr learned my addr 172.21.15.99:0/2082065466 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc181400700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184103fe0 0x7fc1841990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc182800700 1 -- 172.21.15.99:0/2082065466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184103fe0 msgr2=0x7fc1841990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc182800700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184103fe0 0x7fc1841990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc182800700 1 -- 172.21.15.99:0/2082065466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc184102df0 msgr2=0x7fc184198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc182800700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc184102df0 0x7fc184198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc182800700 1 -- 172.21.15.99:0/2082065466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1700095f0 con 0x7fc184104980 2024-03-30T00:36:26.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc181400700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184103fe0 0x7fc1841990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:26.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.776+0000 7fc182800700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184104980 0x7fc1841a0100 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc17c007ee0 tx=0x7fc17c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.777+0000 7fc17aa00700 1 -- 172.21.15.99:0/2082065466 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c017070 con 0x7fc184104980 2024-03-30T00:36:26.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.777+0000 7fc17aa00700 1 -- 172.21.15.99:0/2082065466 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc17c00f040 con 0x7fc184104980 2024-03-30T00:36:26.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.777+0000 7fc17aa00700 1 -- 172.21.15.99:0/2082065466 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c005710 con 0x7fc184104980 2024-03-30T00:36:26.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.777+0000 7fc183200700 1 -- 172.21.15.99:0/2082065466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1841a0c80 con 0x7fc184104980 2024-03-30T00:36:26.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.777+0000 7fc183200700 1 -- 172.21.15.99:0/2082065466 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1841a1140 con 0x7fc184104980 2024-03-30T00:36:26.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.781+0000 7fc17aa00700 1 -- 172.21.15.99:0/2082065466 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc17c005870 con 0x7fc184104980 2024-03-30T00:36:26.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.782+0000 7fc17aa00700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1680a11a0 0x7fc1680a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.782+0000 7fc17aa00700 1 -- 172.21.15.99:0/2082065466 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fc17c0cc0f0 con 0x7fc184104980 2024-03-30T00:36:26.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.782+0000 7fc19ccddb80 1 Processor -- start 2024-03-30T00:36:26.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.782+0000 7fc181e00700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1680a11a0 0x7fc1680a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.782+0000 7fc19ccddb80 1 -- start start 2024-03-30T00:36:26.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.783+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db49e0 0x556e44db4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.783+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 0x556e44db5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.783+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 0x556e44db6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.783+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44d0c990 con 0x556e44db6190 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.783+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44d0a350 con 0x556e44db5340 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.783+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44d05c40 con 0x556e44db49e0 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.783+0000 7fc181e00700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1680a11a0 0x7fc1680a3660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc184103e40 tx=0x7fc17400a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.783+0000 7fc181400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 0x556e44db5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.783+0000 7fc181400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 0x556e44db5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:56060/0 (socket says 172.21.15.99:56060) 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.783+0000 7fc181400700 1 -- 172.21.15.99:0/1732959680 learned_addr learned my addr 172.21.15.99:0/1732959680 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.783+0000 7fc181e00700 1 --2- 172.21.15.99:0/1732959680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db49e0 0x556e44db4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.784+0000 7fc182800700 1 --2- 172.21.15.99:0/1732959680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 0x556e44db6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.784+0000 7fc181400700 1 -- 172.21.15.99:0/1732959680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db49e0 msgr2=0x556e44db4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.784+0000 7fc181400700 1 --2- 172.21.15.99:0/1732959680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db49e0 0x556e44db4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.784+0000 7fc181400700 1 -- 172.21.15.99:0/1732959680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 msgr2=0x556e44db6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.784+0000 7fc181400700 1 --2- 172.21.15.99:0/1732959680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 0x556e44db6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.784+0000 7fc181400700 1 -- 172.21.15.99:0/1732959680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e44b873b0 con 0x556e44db5340 2024-03-30T00:36:26.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.784+0000 7fc181400700 1 --2- 172.21.15.99:0/1732959680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 0x556e44db5bb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc1700099c0 tx=0x7fc170005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30eda9c0c2918a47 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.785+0000 7fc16f400700 1 -- 172.21.15.99:0/1732959680 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17000b990 con 0x556e44db5340 2024-03-30T00:36:26.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.785+0000 7fc16f400700 1 -- 172.21.15.99:0/1732959680 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc17001e040 con 0x556e44db5340 2024-03-30T00:36:26.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.785+0000 7fc16f400700 1 -- 172.21.15.99:0/1732959680 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1700048e0 con 0x556e44db5340 2024-03-30T00:36:26.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.785+0000 7fc19ccddb80 1 -- 172.21.15.99:0/1732959680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 msgr2=0x556e44db5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.785+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/1732959680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 0x556e44db5bb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc1700099c0 tx=0x7fc170005f00 comp rx=0 tx=0).stop 2024-03-30T00:36:26.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.786+0000 7fc19ccddb80 1 -- 172.21.15.99:0/1732959680 shutdown_connections 2024-03-30T00:36:26.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.786+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/1732959680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 0x556e44db6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.786+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/1732959680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db49e0 0x556e44db4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.786+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/1732959680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 0x556e44db5bb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.786+0000 7fc19ccddb80 1 -- 172.21.15.99:0/1732959680 >> 172.21.15.99:0/1732959680 conn(0x556e44d136a0 msgr2=0x556e44dafad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.786+0000 7fc19ccddb80 1 -- 172.21.15.99:0/1732959680 shutdown_connections 2024-03-30T00:36:26.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.786+0000 7fc19ccddb80 1 -- 172.21.15.99:0/1732959680 wait complete. 2024-03-30T00:36:26.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.786+0000 7fc19ccddb80 1 -- start start 2024-03-30T00:36:26.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc19ccddb80 1 Processor -- start 2024-03-30T00:36:26.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 0x556e44dbe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 0x556e44dbed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db6190 0x556e44dc5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44ca1d80 con 0x556e44db5340 2024-03-30T00:36:26.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44ca1a20 con 0x556e44db49e0 2024-03-30T00:36:26.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44b896a0 con 0x556e44db6190 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc19ccddb80 10 client.0 _ll_register_callbacks cb 0x556e44cab470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc181e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 0x556e44dbe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc181400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 0x556e44dbed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc181e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 0x556e44dbe7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:56076/0 (socket says 172.21.15.99:56076) 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.787+0000 7fc181e00700 1 -- 172.21.15.99:0/98232635 learned_addr learned my addr 172.21.15.99:0/98232635 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.788+0000 7fc181400700 1 -- 172.21.15.99:0/98232635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db6190 msgr2=0x556e44dc5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.788+0000 7fc181400700 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db6190 0x556e44dc5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.788+0000 7fc181400700 1 -- 172.21.15.99:0/98232635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 msgr2=0x556e44dbe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.788+0000 7fc181400700 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 0x556e44dbe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.788+0000 7fc181400700 1 -- 172.21.15.99:0/98232635 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18410bb80 con 0x556e44db5340 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.788+0000 7fc181e00700 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 0x556e44dbe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.788+0000 7fc181400700 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 0x556e44dbed20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc17000bef0 tx=0x7fc170005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a26bd4ec2143d71 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.788+0000 7fc163e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.788+0000 7fc163e00700 1 -- 172.21.15.99:0/98232635 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170003d30 con 0x556e44db5340 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc163e00700 1 -- 172.21.15.99:0/98232635 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fc170003e90 con 0x556e44db5340 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc19ccddb80 1 -- 172.21.15.99:0/98232635 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e44cc6380 con 0x556e44db5340 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc163e00700 1 -- 172.21.15.99:0/98232635 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc170004050 con 0x556e44db5340 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc19ccddb80 10 client.15906 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc19ccddb80 1 -- 172.21.15.99:0/98232635 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556e44b33b60 con 0x556e44db5340 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc163e00700 1 -- 172.21.15.99:0/98232635 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556e44cc6380 con 0x556e44db5340 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc163e00700 1 -- 172.21.15.99:0/98232635 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc170021070 con 0x556e44db5340 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc19ccddb80 10 client.15906 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc19ccddb80 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc19ccddb80 20 client.15906 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556e44dca050 0x556e44dcc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc19ccddb80 4 client.15906 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.789+0000 7fc19ccddb80 1 -- 172.21.15.99:0/98232635 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e44cfe700 con 0x556e44dca050 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.790+0000 7fc181e00700 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556e44dca050 0x556e44dcc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.791+0000 7fc181e00700 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556e44dca050 0x556e44dcc440 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ad5844be46d7bec server_cookie=7db5549aa0b78eff in_seq=0 out_seq=0 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.791+0000 7fc163e00700 10 client.15906 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.791+0000 7fc163e00700 1 -- 172.21.15.99:0/98232635 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556e44cfe700 con 0x556e44dca050 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.791+0000 7fc163e00700 10 client.15906 handle_command_reply: tid=0 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.791+0000 7fc19ccddb80 1 client.15906 shutdown 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc19ccddb80 1 -- 172.21.15.99:0/98232635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 msgr2=0x556e44dbed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 0x556e44dbed20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc17000bef0 tx=0x7fc170005740 comp rx=0 tx=0).stop 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc19ccddb80 1 -- 172.21.15.99:0/98232635 shutdown_connections 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556e44dca050 0x556e44dcc440 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db6190 0x556e44dc5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 0x556e44dbe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc163e00700 0 client.15906 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/98232635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 0x556e44dbed20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc19ccddb80 1 -- 172.21.15.99:0/98232635 >> 172.21.15.99:0/98232635 conn(0x556e44d136a0 msgr2=0x556e44db33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc19ccddb80 1 -- 172.21.15.99:0/98232635 shutdown_connections 2024-03-30T00:36:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc19ccddb80 1 -- 172.21.15.99:0/98232635 wait complete. 2024-03-30T00:36:26.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.792+0000 7fc19ccddb80 20 client.15906 trim_cache size 0 max 16384 2024-03-30T00:36:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc19ccddb80 1 Processor -- start 2024-03-30T00:36:26.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc19ccddb80 1 -- start start 2024-03-30T00:36:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db49e0 0x556e44dbdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 0x556e44dc5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 0x556e44db6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44dbe350 con 0x556e44db6190 2024-03-30T00:36:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44db0ed0 con 0x556e44db5340 2024-03-30T00:36:26.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44db5830 con 0x556e44db49e0 2024-03-30T00:36:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc182800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 0x556e44db6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc182800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 0x556e44db6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59512/0 (socket says 172.21.15.99:59512) 2024-03-30T00:36:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc182800700 1 -- 172.21.15.99:0/136658949 learned_addr learned my addr 172.21.15.99:0/136658949 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.813+0000 7fc181e00700 1 --2- 172.21.15.99:0/136658949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db49e0 0x556e44dbdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.814+0000 7fc181400700 1 --2- 172.21.15.99:0/136658949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 0x556e44dc5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.814+0000 7fc182800700 1 -- 172.21.15.99:0/136658949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db49e0 msgr2=0x556e44dbdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.814+0000 7fc182800700 1 --2- 172.21.15.99:0/136658949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db49e0 0x556e44dbdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.814+0000 7fc182800700 1 -- 172.21.15.99:0/136658949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 msgr2=0x556e44dc5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.814+0000 7fc182800700 1 --2- 172.21.15.99:0/136658949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 0x556e44dc5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.814+0000 7fc182800700 1 -- 172.21.15.99:0/136658949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e44ccbea0 con 0x556e44db6190 2024-03-30T00:36:26.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.814+0000 7fc182800700 1 --2- 172.21.15.99:0/136658949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 0x556e44db6570 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00d7b0 tx=0x7fc17c084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbeb7bc9126d56d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.814+0000 7fc16fe00700 1 -- 172.21.15.99:0/136658949 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c098e20 con 0x556e44db6190 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.814+0000 7fc16fe00700 1 -- 172.21.15.99:0/136658949 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc17c00dc60 con 0x556e44db6190 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.814+0000 7fc16fe00700 1 -- 172.21.15.99:0/136658949 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0a1940 con 0x556e44db6190 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.815+0000 7fc19ccddb80 1 -- 172.21.15.99:0/136658949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 msgr2=0x556e44db6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.815+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/136658949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 0x556e44db6570 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00d7b0 tx=0x7fc17c084040 comp rx=0 tx=0).stop 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.815+0000 7fc19ccddb80 1 -- 172.21.15.99:0/136658949 shutdown_connections 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.815+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/136658949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db6190 0x556e44db6570 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.815+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/136658949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db49e0 0x556e44dbdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.815+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/136658949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db5340 0x556e44dc5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.815+0000 7fc19ccddb80 1 -- 172.21.15.99:0/136658949 >> 172.21.15.99:0/136658949 conn(0x556e44d136a0 msgr2=0x556e44da87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.815+0000 7fc19ccddb80 1 -- 172.21.15.99:0/136658949 shutdown_connections 2024-03-30T00:36:26.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.815+0000 7fc19ccddb80 1 -- 172.21.15.99:0/136658949 wait complete. 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.816+0000 7fc19ccddb80 1 -- start start 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.816+0000 7fc19ccddb80 1 Processor -- start 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.816+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 0x556e44dc3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.816+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 0x556e44dc4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.816+0000 7fc19ccddb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db6190 0x556e44dc5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.816+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44dc5790 con 0x556e44db5340 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.816+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44dc5900 con 0x556e44db49e0 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.816+0000 7fc19ccddb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e44dc5ac0 con 0x556e44db6190 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.816+0000 7fc19ccddb80 10 client.0 _ll_register_callbacks cb 0x556e44cab470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc181400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 0x556e44dc4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc181400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 0x556e44dc4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59518/0 (socket says 172.21.15.99:59518) 2024-03-30T00:36:26.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc181400700 1 -- 172.21.15.99:0/3460680863 learned_addr learned my addr 172.21.15.99:0/3460680863 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:26.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc181e00700 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 0x556e44dc3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc182800700 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db6190 0x556e44dc5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc181400700 1 -- 172.21.15.99:0/3460680863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db6190 msgr2=0x556e44dc5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc181400700 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db6190 0x556e44dc5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc181400700 1 -- 172.21.15.99:0/3460680863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 msgr2=0x556e44dc3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:26.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc181400700 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 0x556e44dc3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc181400700 1 -- 172.21.15.99:0/3460680863 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1841a1140 con 0x556e44db5340 2024-03-30T00:36:26.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc181400700 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 0x556e44dc4240 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc170003780 tx=0x7fc17000fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a79b659efba5da server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:26.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.817+0000 7fc163e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.818+0000 7fc163e00700 1 -- 172.21.15.99:0/3460680863 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170003d30 con 0x556e44db5340 2024-03-30T00:36:26.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.818+0000 7fc163e00700 1 -- 172.21.15.99:0/3460680863 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fc170000e40 con 0x556e44db5340 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.818+0000 7fc19ccddb80 1 -- 172.21.15.99:0/3460680863 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc17001e040 con 0x556e44db5340 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.818+0000 7fc163e00700 1 -- 172.21.15.99:0/3460680863 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc170005a00 con 0x556e44db5340 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.818+0000 7fc19ccddb80 10 client.15912 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.818+0000 7fc19ccddb80 1 -- 172.21.15.99:0/3460680863 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556e44b8bcd0 con 0x556e44db5340 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.818+0000 7fc163e00700 1 -- 172.21.15.99:0/3460680863 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc17004b080 con 0x556e44db5340 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.819+0000 7fc163e00700 1 -- 172.21.15.99:0/3460680863 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc1700099f0 con 0x556e44db5340 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.819+0000 7fc19ccddb80 10 client.15912 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.819+0000 7fc19ccddb80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.819+0000 7fc19ccddb80 20 client.15912 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.819+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556e44e1ec30 0x556e44e21050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.819+0000 7fc19ccddb80 4 client.15912 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:36:26.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.819+0000 7fc19ccddb80 1 -- 172.21.15.99:0/3460680863 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556e44b54cf0 con 0x556e44e1ec30 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.820+0000 7fc181e00700 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556e44e1ec30 0x556e44e21050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.821+0000 7fc181e00700 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556e44e1ec30 0x556e44e21050 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40e9020d7f68c995 server_cookie=4f216df57aefc13b in_seq=0 out_seq=0 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.821+0000 7fc163e00700 10 client.15912 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.821+0000 7fc163e00700 1 -- 172.21.15.99:0/3460680863 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556e44b54cf0 con 0x556e44e1ec30 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.821+0000 7fc163e00700 10 client.15912 handle_command_reply: tid=0 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.821+0000 7fc19ccddb80 1 client.15912 shutdown 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.821+0000 7fc19ccddb80 1 -- 172.21.15.99:0/3460680863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 msgr2=0x556e44dc4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.821+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 0x556e44dc4240 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc170003780 tx=0x7fc17000fba0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.821+0000 7fc19ccddb80 1 -- 172.21.15.99:0/3460680863 shutdown_connections 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.821+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x556e44e1ec30 0x556e44e21050 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e44db6190 0x556e44dc5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e44db49e0 0x556e44dc3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc19ccddb80 1 --2- 172.21.15.99:0/3460680863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e44db5340 0x556e44dc4240 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc19ccddb80 1 -- 172.21.15.99:0/3460680863 >> 172.21.15.99:0/3460680863 conn(0x556e44d136a0 msgr2=0x556e44da9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc163e00700 0 client.15912 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:26.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc19ccddb80 1 -- 172.21.15.99:0/3460680863 shutdown_connections 2024-03-30T00:36:26.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc19ccddb80 1 -- 172.21.15.99:0/3460680863 wait complete. 2024-03-30T00:36:26.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc19ccddb80 20 client.15912 trim_cache size 0 max 16384 2024-03-30T00:36:26.832 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:36:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc183200700 1 -- 172.21.15.99:0/2082065466 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1680a11a0 msgr2=0x7fc1680a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc183200700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1680a11a0 0x7fc1680a3660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc184103e40 tx=0x7fc17400a3b0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc183200700 1 -- 172.21.15.99:0/2082065466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184104980 msgr2=0x7fc1841a0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.822+0000 7fc183200700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184104980 0x7fc1841a0100 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc17c007ee0 tx=0x7fc17c00d3b0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.823+0000 7fc183200700 1 -- 172.21.15.99:0/2082065466 shutdown_connections 2024-03-30T00:36:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.823+0000 7fc183200700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc184104980 0x7fc1841a0100 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.823+0000 7fc183200700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc1680a11a0 0x7fc1680a3660 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.823+0000 7fc183200700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc184102df0 0x7fc184198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.823+0000 7fc183200700 1 --2- 172.21.15.99:0/2082065466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc184103fe0 0x7fc1841990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.823+0000 7fc183200700 1 -- 172.21.15.99:0/2082065466 >> 172.21.15.99:0/2082065466 conn(0x7fc1840fe380 msgr2=0x7fc184107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.823+0000 7fc183200700 1 -- 172.21.15.99:0/2082065466 shutdown_connections 2024-03-30T00:36:26.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:26.823+0000 7fc183200700 1 -- 172.21.15.99:0/2082065466 wait complete. 2024-03-30T00:36:27.852 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T00:36:27.852 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T00:36:28.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.061+0000 7f7818c00700 1 -- 172.21.15.99:0/136905136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7814102910 msgr2=0x7f78141091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.061+0000 7f7818c00700 1 --2- 172.21.15.99:0/136905136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7814102910 0x7f78141091c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f780c00b0d0 tx=0x7f780c00b3e0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.061+0000 7f7818c00700 1 -- 172.21.15.99:0/136905136 shutdown_connections 2024-03-30T00:36:28.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.062+0000 7f7818c00700 1 --2- 172.21.15.99:0/136905136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7814102910 0x7f78141091c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.062+0000 7f7818c00700 1 --2- 172.21.15.99:0/136905136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7814100d80 0x7f78141011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.062+0000 7f7818c00700 1 --2- 172.21.15.99:0/136905136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7814101f70 0x7f78141023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.062+0000 7f7818c00700 1 -- 172.21.15.99:0/136905136 >> 172.21.15.99:0/136905136 conn(0x7f78140fc2f0 msgr2=0x7f78140fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.062+0000 7f7818c00700 1 -- 172.21.15.99:0/136905136 shutdown_connections 2024-03-30T00:36:28.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.062+0000 7f7818c00700 1 -- 172.21.15.99:0/136905136 wait complete. 2024-03-30T00:36:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.063+0000 7f7818c00700 1 Processor -- start 2024-03-30T00:36:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.063+0000 7f7818c00700 1 -- start start 2024-03-30T00:36:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.063+0000 7f7818c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7814100d80 0x7f7814074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.063+0000 7f7818c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7814101f70 0x7f78140731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.063+0000 7f7818c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7814102910 0x7f78140736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.063+0000 7f7818c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7814073e80 con 0x7f7814102910 2024-03-30T00:36:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.063+0000 7f7818c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7814073ff0 con 0x7f7814101f70 2024-03-30T00:36:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.063+0000 7f7818c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78141a0d30 con 0x7f7814100d80 2024-03-30T00:36:28.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.063+0000 7f7812800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7814102910 0x7f78140736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.064+0000 7f7812800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7814102910 0x7f78140736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59536/0 (socket says 172.21.15.99:59536) 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.064+0000 7f7812800700 1 -- 172.21.15.99:0/1758703927 learned_addr learned my addr 172.21.15.99:0/1758703927 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.064+0000 7f7811e00700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7814100d80 0x7f7814074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.064+0000 7f7811400700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7814101f70 0x7f78140731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.064+0000 7f7812800700 1 -- 172.21.15.99:0/1758703927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7814100d80 msgr2=0x7f7814074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.064+0000 7f7812800700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7814100d80 0x7f7814074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.064+0000 7f7812800700 1 -- 172.21.15.99:0/1758703927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7814101f70 msgr2=0x7f78140731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.064+0000 7f7812800700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7814101f70 0x7f78140731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.064+0000 7f7812800700 1 -- 172.21.15.99:0/1758703927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c009d00 con 0x7f7814102910 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.064+0000 7f7812800700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7814102910 0x7f78140736e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f780c007b60 tx=0x7f780c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.064+0000 7f780aa00700 1 -- 172.21.15.99:0/1758703927 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c00e040 con 0x7f7814102910 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.065+0000 7f780aa00700 1 -- 172.21.15.99:0/1758703927 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f780c01dd20 con 0x7f7814102910 2024-03-30T00:36:28.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.065+0000 7f780aa00700 1 -- 172.21.15.99:0/1758703927 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c012dd0 con 0x7f7814102910 2024-03-30T00:36:28.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.065+0000 7f7818c00700 1 -- 172.21.15.99:0/1758703927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78141a0f60 con 0x7f7814102910 2024-03-30T00:36:28.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.065+0000 7f7818c00700 1 -- 172.21.15.99:0/1758703927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78141a12d0 con 0x7f7814102910 2024-03-30T00:36:28.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.068+0000 7f780aa00700 1 -- 172.21.15.99:0/1758703927 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f780c01d890 con 0x7f7814102910 2024-03-30T00:36:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.069+0000 7f782c64eb80 1 Processor -- start 2024-03-30T00:36:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.069+0000 7f780aa00700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77f80a11a0 0x7f77f80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.069+0000 7f782c64eb80 1 -- start start 2024-03-30T00:36:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.069+0000 7f780aa00700 1 -- 172.21.15.99:0/1758703927 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f780c0ce790 con 0x7f7814102910 2024-03-30T00:36:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c71ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7811400700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77f80a11a0 0x7f77f80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71b1b0 0x563d5c71ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71c050 0x563d5c71c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f782c64eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c672a10 con 0x563d5c71b1b0 2024-03-30T00:36:28.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f782c64eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c6703d0 con 0x563d5c71a850 2024-03-30T00:36:28.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f782c64eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c66bcc0 con 0x563d5c71c050 2024-03-30T00:36:28.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7812800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71c050 0x563d5c71c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7812800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71c050 0x563d5c71c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:59504/0 (socket says 172.21.15.99:59504) 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7812800700 1 -- 172.21.15.99:0/2516917967 learned_addr learned my addr 172.21.15.99:0/2516917967 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7811400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71b1b0 0x563d5c71ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7811e00700 1 --2- 172.21.15.99:0/2516917967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c71ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7812800700 1 -- 172.21.15.99:0/2516917967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 msgr2=0x563d5c71ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7812800700 1 --2- 172.21.15.99:0/2516917967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c71ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7812800700 1 -- 172.21.15.99:0/2516917967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71b1b0 msgr2=0x563d5c71ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7812800700 1 --2- 172.21.15.99:0/2516917967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71b1b0 0x563d5c71ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7812800700 1 -- 172.21.15.99:0/2516917967 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d5c4ed430 con 0x563d5c71c050 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.070+0000 7f7811e00700 1 --2- 172.21.15.99:0/2516917967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c71ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.071+0000 7f7811400700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77f80a11a0 0x7f77f80a3660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f780000b6c0 tx=0x7f7800012890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.071+0000 7f7812800700 1 --2- 172.21.15.99:0/2516917967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71c050 0x563d5c71c4d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f780c09bb80 tx=0x7f780c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1b21d5a82848f1f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.071+0000 7f77ff000700 1 -- 172.21.15.99:0/2516917967 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c09b550 con 0x563d5c71c050 2024-03-30T00:36:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.071+0000 7f77ff000700 1 -- 172.21.15.99:0/2516917967 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f780c01de90 con 0x563d5c71c050 2024-03-30T00:36:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.071+0000 7f77ff000700 1 -- 172.21.15.99:0/2516917967 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c09bc60 con 0x563d5c71c050 2024-03-30T00:36:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.072+0000 7f782c64eb80 1 -- 172.21.15.99:0/2516917967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71c050 msgr2=0x563d5c71c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.072+0000 7f782c64eb80 1 --2- 172.21.15.99:0/2516917967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71c050 0x563d5c71c4d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f780c09bb80 tx=0x7f780c0099e0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.072+0000 7f782c64eb80 1 -- 172.21.15.99:0/2516917967 shutdown_connections 2024-03-30T00:36:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.072+0000 7f782c64eb80 1 --2- 172.21.15.99:0/2516917967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71c050 0x563d5c71c4d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.072+0000 7f782c64eb80 1 --2- 172.21.15.99:0/2516917967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c71ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.072+0000 7f782c64eb80 1 --2- 172.21.15.99:0/2516917967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71b1b0 0x563d5c71ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.072+0000 7f782c64eb80 1 -- 172.21.15.99:0/2516917967 >> 172.21.15.99:0/2516917967 conn(0x563d5c679720 msgr2=0x563d5c715a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.072+0000 7f782c64eb80 1 -- 172.21.15.99:0/2516917967 shutdown_connections 2024-03-30T00:36:28.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.072+0000 7f782c64eb80 1 -- 172.21.15.99:0/2516917967 wait complete. 2024-03-30T00:36:28.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.073+0000 7f782c64eb80 1 -- start start 2024-03-30T00:36:28.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.073+0000 7f782c64eb80 1 Processor -- start 2024-03-30T00:36:28.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c7244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c7249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 0x563d5c72ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f782c64eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c607e00 con 0x563d5c71c050 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f782c64eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c607aa0 con 0x563d5c71a850 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f782c64eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c4ef720 con 0x563d5c71b1b0 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f782c64eb80 10 client.0 _ll_register_callbacks cb 0x563d5c6114f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f7811e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c7244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f7812800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 0x563d5c72ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f7811e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c7244a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:56138/0 (socket says 172.21.15.99:56138) 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f7811e00700 1 -- 172.21.15.99:0/223909012 learned_addr learned my addr 172.21.15.99:0/223909012 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.074+0000 7f7811400700 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c7249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.075+0000 7f7811e00700 1 -- 172.21.15.99:0/223909012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 msgr2=0x563d5c7249e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.075+0000 7f7811e00700 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c7249e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.075+0000 7f7811e00700 1 -- 172.21.15.99:0/223909012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 msgr2=0x563d5c72ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.075+0000 7f7811e00700 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 0x563d5c72ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.075+0000 7f7811e00700 1 -- 172.21.15.99:0/223909012 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78141a12d0 con 0x563d5c71a850 2024-03-30T00:36:28.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.075+0000 7f7812800700 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 0x563d5c72ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:28.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.075+0000 7f7811e00700 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c7244a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78040128b0 tx=0x7f7804012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1dd06f2c475c300 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.075+0000 7f77f3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:36:28.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.075+0000 7f77f3400700 1 -- 172.21.15.99:0/223909012 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804004460 con 0x563d5c71a850 2024-03-30T00:36:28.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.075+0000 7f782c64eb80 1 -- 172.21.15.99:0/223909012 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d5c62c400 con 0x563d5c71a850 2024-03-30T00:36:28.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.075+0000 7f77f3400700 1 -- 172.21.15.99:0/223909012 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f7804003680 con 0x563d5c71a850 2024-03-30T00:36:28.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.076+0000 7f77f3400700 1 -- 172.21.15.99:0/223909012 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7804003840 con 0x563d5c71a850 2024-03-30T00:36:28.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.076+0000 7f782c64eb80 10 client.25625 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:28.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.076+0000 7f782c64eb80 1 -- 172.21.15.99:0/223909012 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563d5c499be0 con 0x563d5c71a850 2024-03-30T00:36:28.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.076+0000 7f77f3400700 1 -- 172.21.15.99:0/223909012 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563d5c62c400 con 0x563d5c71a850 2024-03-30T00:36:28.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.077+0000 7f77f3400700 1 -- 172.21.15.99:0/223909012 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7804010c60 con 0x563d5c71a850 2024-03-30T00:36:28.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.077+0000 7f782c64eb80 10 client.25625 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:28.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.077+0000 7f782c64eb80 10 client.25625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:28.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.077+0000 7f782c64eb80 20 client.25625 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:28.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.077+0000 7f782c64eb80 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563d5c7302c0 0x563d5c7326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.077+0000 7f782c64eb80 4 client.25625 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:28.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.077+0000 7f782c64eb80 1 -- 172.21.15.99:0/223909012 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d5c664780 con 0x563d5c7302c0 2024-03-30T00:36:28.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.077+0000 7f7811400700 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563d5c7302c0 0x563d5c7326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.078+0000 7f77f3400700 10 client.25625 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.078+0000 7f7811400700 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563d5c7302c0 0x563d5c7326e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e01222d80d1ec8d server_cookie=30ef9fea4419eec8 in_seq=0 out_seq=0 2024-03-30T00:36:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.078+0000 7f77f3400700 1 -- 172.21.15.99:0/223909012 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563d5c664780 con 0x563d5c7302c0 2024-03-30T00:36:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.078+0000 7f77f3400700 10 client.25625 handle_command_reply: tid=0 2024-03-30T00:36:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.078+0000 7f782c64eb80 1 client.25625 shutdown 2024-03-30T00:36:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f782c64eb80 1 -- 172.21.15.99:0/223909012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 msgr2=0x563d5c7244a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f782c64eb80 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c7244a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78040128b0 tx=0x7f7804012bc0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f782c64eb80 1 -- 172.21.15.99:0/223909012 shutdown_connections 2024-03-30T00:36:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f782c64eb80 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563d5c7302c0 0x563d5c7326e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f782c64eb80 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 0x563d5c72ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f77f3400700 0 client.25625 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f782c64eb80 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c7244a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f782c64eb80 1 --2- 172.21.15.99:0/223909012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c7249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f782c64eb80 1 -- 172.21.15.99:0/223909012 >> 172.21.15.99:0/223909012 conn(0x563d5c679720 msgr2=0x563d5c7191f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f782c64eb80 1 -- 172.21.15.99:0/223909012 shutdown_connections 2024-03-30T00:36:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f782c64eb80 1 -- 172.21.15.99:0/223909012 wait complete. 2024-03-30T00:36:28.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.079+0000 7f782c64eb80 20 client.25625 trim_cache size 0 max 16384 2024-03-30T00:36:28.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.100+0000 7f782c64eb80 1 Processor -- start 2024-03-30T00:36:28.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.100+0000 7f782c64eb80 1 -- start start 2024-03-30T00:36:28.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.100+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c71c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.100+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c72b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.100+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 0x563d5c7239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.100+0000 7f782c64eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c71a3f0 con 0x563d5c71c050 2024-03-30T00:36:28.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.100+0000 7f782c64eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c7254e0 con 0x563d5c71a850 2024-03-30T00:36:28.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.100+0000 7f782c64eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c732a70 con 0x563d5c71b1b0 2024-03-30T00:36:28.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.101+0000 7f7812800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 0x563d5c7239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.101+0000 7f7812800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 0x563d5c7239b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59572/0 (socket says 172.21.15.99:59572) 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.101+0000 7f7812800700 1 -- 172.21.15.99:0/1237287846 learned_addr learned my addr 172.21.15.99:0/1237287846 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.101+0000 7f7811400700 1 --2- 172.21.15.99:0/1237287846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c72b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.101+0000 7f7811e00700 1 --2- 172.21.15.99:0/1237287846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c71c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.101+0000 7f7812800700 1 -- 172.21.15.99:0/1237287846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 msgr2=0x563d5c72b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.101+0000 7f7812800700 1 --2- 172.21.15.99:0/1237287846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c72b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.101+0000 7f7812800700 1 -- 172.21.15.99:0/1237287846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 msgr2=0x563d5c71c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.101+0000 7f7812800700 1 --2- 172.21.15.99:0/1237287846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c71c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.101+0000 7f7812800700 1 -- 172.21.15.99:0/1237287846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d5c631f20 con 0x563d5c71c050 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.101+0000 7f7811e00700 1 --2- 172.21.15.99:0/1237287846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c71c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.102+0000 7f7812800700 1 --2- 172.21.15.99:0/1237287846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 0x563d5c7239b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f780c00d010 tx=0x7f780c09c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6aceb588a7f6f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.102+0000 7f77ffa00700 1 -- 172.21.15.99:0/1237287846 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c0b8070 con 0x563d5c71c050 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.102+0000 7f77ffa00700 1 -- 172.21.15.99:0/1237287846 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f780c09b550 con 0x563d5c71c050 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.102+0000 7f77ffa00700 1 -- 172.21.15.99:0/1237287846 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c0123f0 con 0x563d5c71c050 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.102+0000 7f782c64eb80 1 -- 172.21.15.99:0/1237287846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 msgr2=0x563d5c7239b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.102+0000 7f782c64eb80 1 --2- 172.21.15.99:0/1237287846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 0x563d5c7239b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f780c00d010 tx=0x7f780c09c2f0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.103+0000 7f782c64eb80 1 -- 172.21.15.99:0/1237287846 shutdown_connections 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.103+0000 7f782c64eb80 1 --2- 172.21.15.99:0/1237287846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71c050 0x563d5c7239b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.103+0000 7f782c64eb80 1 --2- 172.21.15.99:0/1237287846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71a850 0x563d5c71c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.103+0000 7f782c64eb80 1 --2- 172.21.15.99:0/1237287846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c72b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.103+0000 7f782c64eb80 1 -- 172.21.15.99:0/1237287846 >> 172.21.15.99:0/1237287846 conn(0x563d5c679720 msgr2=0x563d5c70e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.103+0000 7f782c64eb80 1 -- 172.21.15.99:0/1237287846 shutdown_connections 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.103+0000 7f782c64eb80 1 -- 172.21.15.99:0/1237287846 wait complete. 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.103+0000 7f782c64eb80 1 -- start start 2024-03-30T00:36:28.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.103+0000 7f782c64eb80 1 Processor -- start 2024-03-30T00:36:28.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.104+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71a850 0x563d5c72b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.104+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c72b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.104+0000 7f782c64eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71c050 0x563d5c729a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.104+0000 7f782c64eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c72a070 con 0x563d5c71a850 2024-03-30T00:36:28.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.104+0000 7f782c64eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c72a1e0 con 0x563d5c71c050 2024-03-30T00:36:28.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.104+0000 7f782c64eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5c72a3a0 con 0x563d5c71b1b0 2024-03-30T00:36:28.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.104+0000 7f782c64eb80 10 client.0 _ll_register_callbacks cb 0x563d5c71bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:28.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.104+0000 7f7811e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71a850 0x563d5c72b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.104+0000 7f7811e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71a850 0x563d5c72b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59580/0 (socket says 172.21.15.99:59580) 2024-03-30T00:36:28.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.104+0000 7f7811e00700 1 -- 172.21.15.99:0/29244458 learned_addr learned my addr 172.21.15.99:0/29244458 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f7811400700 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c72b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f7812800700 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71c050 0x563d5c729a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f7811e00700 1 -- 172.21.15.99:0/29244458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 msgr2=0x563d5c72b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f7811e00700 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c72b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f7811e00700 1 -- 172.21.15.99:0/29244458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71c050 msgr2=0x563d5c729a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f7811e00700 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71c050 0x563d5c729a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f7811e00700 1 -- 172.21.15.99:0/29244458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d5c4ed430 con 0x563d5c71a850 2024-03-30T00:36:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f7812800700 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71c050 0x563d5c729a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f7811e00700 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71a850 0x563d5c72b2b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f78040104a0 tx=0x7f780400ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dba88996c0832d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f77f3400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f77f3400700 1 -- 172.21.15.99:0/29244458 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780400ec10 con 0x563d5c71a850 2024-03-30T00:36:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f77f3400700 1 -- 172.21.15.99:0/29244458 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f780400e5c0 con 0x563d5c71a850 2024-03-30T00:36:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.105+0000 7f782c64eb80 1 -- 172.21.15.99:0/29244458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f780c01de90 con 0x563d5c71a850 2024-03-30T00:36:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.106+0000 7f77f3400700 1 -- 172.21.15.99:0/29244458 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f780400ed70 con 0x563d5c71a850 2024-03-30T00:36:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.106+0000 7f782c64eb80 10 client.15930 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.106+0000 7f782c64eb80 1 -- 172.21.15.99:0/29244458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563d5c4f1d50 con 0x563d5c71a850 2024-03-30T00:36:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.106+0000 7f77f3400700 1 -- 172.21.15.99:0/29244458 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f780403b020 con 0x563d5c71a850 2024-03-30T00:36:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.106+0000 7f77f3400700 1 -- 172.21.15.99:0/29244458 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7804003880 con 0x563d5c71a850 2024-03-30T00:36:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.106+0000 7f782c64eb80 10 client.15930 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.106+0000 7f782c64eb80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.106+0000 7f782c64eb80 20 client.15930 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.106+0000 7f782c64eb80 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563d5c7853b0 0x563d5c7877d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.106+0000 7f782c64eb80 4 client.15930 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T00:36:28.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.106+0000 7f782c64eb80 1 -- 172.21.15.99:0/29244458 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563d5c4bad70 con 0x563d5c7853b0 2024-03-30T00:36:28.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.107+0000 7f7811400700 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563d5c7853b0 0x563d5c7877d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.107+0000 7f7811400700 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563d5c7853b0 0x563d5c7877d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a8ded40ad846d2 server_cookie=e3393b87f84043 in_seq=0 out_seq=0 2024-03-30T00:36:28.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.107+0000 7f77f3400700 10 client.15930 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:28.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.108+0000 7f77f3400700 1 -- 172.21.15.99:0/29244458 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563d5c4bad70 con 0x563d5c7853b0 2024-03-30T00:36:28.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.108+0000 7f77f3400700 10 client.15930 handle_command_reply: tid=0 2024-03-30T00:36:28.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.108+0000 7f782c64eb80 1 client.15930 shutdown 2024-03-30T00:36:28.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f782c64eb80 1 -- 172.21.15.99:0/29244458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71a850 msgr2=0x563d5c72b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f782c64eb80 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71a850 0x563d5c72b2b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f78040104a0 tx=0x7f780400ad80 comp rx=0 tx=0).stop 2024-03-30T00:36:28.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f782c64eb80 1 -- 172.21.15.99:0/29244458 shutdown_connections 2024-03-30T00:36:28.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f782c64eb80 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x563d5c7853b0 0x563d5c7877d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f782c64eb80 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563d5c71c050 0x563d5c729a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f782c64eb80 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d5c71a850 0x563d5c72b2b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f782c64eb80 1 --2- 172.21.15.99:0/29244458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5c71b1b0 0x563d5c72b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f782c64eb80 1 -- 172.21.15.99:0/29244458 >> 172.21.15.99:0/29244458 conn(0x563d5c679720 msgr2=0x563d5c7239b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f77f3400700 0 client.15930 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f782c64eb80 1 -- 172.21.15.99:0/29244458 shutdown_connections 2024-03-30T00:36:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f782c64eb80 1 -- 172.21.15.99:0/29244458 wait complete. 2024-03-30T00:36:28.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.109+0000 7f782c64eb80 20 client.15930 trim_cache size 0 max 16384 2024-03-30T00:36:28.122 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:36:28.123 INFO:teuthology.orchestra.run.smithi099.stdout: "return_code": 0, 2024-03-30T00:36:28.123 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_tag": "d1ec7919-6ea4-456e-8da7-b149b92d9937", 2024-03-30T00:36:28.123 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": "asynchronous" 2024-03-30T00:36:28.123 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:36:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 -- 172.21.15.99:0/1758703927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77f80a11a0 msgr2=0x7f77f80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77f80a11a0 0x7f77f80a3660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f780000b6c0 tx=0x7f7800012890 comp rx=0 tx=0).stop 2024-03-30T00:36:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 -- 172.21.15.99:0/1758703927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7814102910 msgr2=0x7f78140736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7814102910 0x7f78140736e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f780c007b60 tx=0x7f780c009440 comp rx=0 tx=0).stop 2024-03-30T00:36:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 -- 172.21.15.99:0/1758703927 shutdown_connections 2024-03-30T00:36:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7814102910 0x7f78140736e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7814100d80 0x7f7814074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f77f80a11a0 0x7f77f80a3660 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 --2- 172.21.15.99:0/1758703927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7814101f70 0x7f78140731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 -- 172.21.15.99:0/1758703927 >> 172.21.15.99:0/1758703927 conn(0x7f78140fc2f0 msgr2=0x7f78141052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 -- 172.21.15.99:0/1758703927 shutdown_connections 2024-03-30T00:36:28.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.110+0000 7f7818c00700 1 -- 172.21.15.99:0/1758703927 wait complete. 2024-03-30T00:36:28.138 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T00:36:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.359+0000 7f43e8a00700 1 -- 172.21.15.99:0/2072806647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e4103fc0 msgr2=0x7f43e41063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.359+0000 7f43e8a00700 1 --2- 172.21.15.99:0/2072806647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e4103fc0 0x7f43e41063b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009990 tx=0x7f43d0009ca0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.359+0000 7f43e8a00700 1 -- 172.21.15.99:0/2072806647 shutdown_connections 2024-03-30T00:36:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.359+0000 7f43e8a00700 1 --2- 172.21.15.99:0/2072806647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e41068f0 0x7f43e4108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.359+0000 7f43e8a00700 1 --2- 172.21.15.99:0/2072806647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e4101690 0x7f43e4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.359+0000 7f43e8a00700 1 --2- 172.21.15.99:0/2072806647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e4103fc0 0x7f43e41063b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.359+0000 7f43e8a00700 1 -- 172.21.15.99:0/2072806647 >> 172.21.15.99:0/2072806647 conn(0x7f43e40fb000 msgr2=0x7f43e40fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.359+0000 7f43e8a00700 1 -- 172.21.15.99:0/2072806647 shutdown_connections 2024-03-30T00:36:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.359+0000 7f43e8a00700 1 -- 172.21.15.99:0/2072806647 wait complete. 2024-03-30T00:36:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.360+0000 7f43e8a00700 1 Processor -- start 2024-03-30T00:36:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.360+0000 7f43e8a00700 1 -- start start 2024-03-30T00:36:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e8a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e4101690 0x7f43e4198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e8a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4103fc0 0x7f43e4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e8a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e41068f0 0x7f43e41a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e8a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41a0800 con 0x7f43e41068f0 2024-03-30T00:36:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e8a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41a0970 con 0x7f43e4101690 2024-03-30T00:36:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e8a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41a0b30 con 0x7f43e4103fc0 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e3200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e41068f0 0x7f43e41a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e3200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e41068f0 0x7f43e41a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59598/0 (socket says 172.21.15.99:59598) 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e3200700 1 -- 172.21.15.99:0/2364082271 learned_addr learned my addr 172.21.15.99:0/2364082271 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e1e00700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4103fc0 0x7f43e4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e2800700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e4101690 0x7f43e4198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e3200700 1 -- 172.21.15.99:0/2364082271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4103fc0 msgr2=0x7f43e4199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e3200700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4103fc0 0x7f43e4199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e3200700 1 -- 172.21.15.99:0/2364082271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e4101690 msgr2=0x7f43e4198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e3200700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e4101690 0x7f43e4198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.361+0000 7f43e3200700 1 -- 172.21.15.99:0/2364082271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d00095f0 con 0x7f43e41068f0 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.362+0000 7f43e3200700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e41068f0 0x7f43e41a0230 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00ef30 tx=0x7f43dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.362+0000 7f43db400700 1 -- 172.21.15.99:0/2364082271 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc018070 con 0x7f43e41068f0 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.362+0000 7f43db400700 1 -- 172.21.15.99:0/2364082271 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43dc00f040 con 0x7f43e41068f0 2024-03-30T00:36:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.362+0000 7f43db400700 1 -- 172.21.15.99:0/2364082271 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00cbc0 con 0x7f43e41068f0 2024-03-30T00:36:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.362+0000 7f43e8a00700 1 -- 172.21.15.99:0/2364082271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e41a0db0 con 0x7f43e41068f0 2024-03-30T00:36:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.362+0000 7f43e8a00700 1 -- 172.21.15.99:0/2364082271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e41a11e0 con 0x7f43e41068f0 2024-03-30T00:36:28.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.367+0000 7f43db400700 1 -- 172.21.15.99:0/2364082271 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f43dc007500 con 0x7f43e41068f0 2024-03-30T00:36:28.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.368+0000 7f43fcf8fb80 1 Processor -- start 2024-03-30T00:36:28.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.368+0000 7f43fcf8fb80 1 -- start start 2024-03-30T00:36:28.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.368+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00332e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.368+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 0x55de00333bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.368+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00334610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.368+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0028a990 con 0x55de003329e0 2024-03-30T00:36:28.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.368+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de00288350 con 0x55de00333340 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.368+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de00283c40 con 0x55de00334190 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.368+0000 7f43db400700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f43c80a11a0 0x7f43c80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43db400700 1 -- 172.21.15.99:0/2364082271 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f43dc0cc3c0 con 0x7f43e41068f0 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00332e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00332e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59604/0 (socket says 172.21.15.99:59604) 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43e1e00700 1 -- 172.21.15.99:0/3824301542 learned_addr learned my addr 172.21.15.99:0/3824301542 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43e3200700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f43c80a11a0 0x7f43c80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43e1e00700 1 --2- 172.21.15.99:0/3824301542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00334610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43e2800700 1 --2- 172.21.15.99:0/3824301542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 0x55de00333bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43e1e00700 1 -- 172.21.15.99:0/3824301542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 msgr2=0x55de00334610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43e1e00700 1 --2- 172.21.15.99:0/3824301542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00334610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43e1e00700 1 -- 172.21.15.99:0/3824301542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 msgr2=0x55de00333bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43e1e00700 1 --2- 172.21.15.99:0/3824301542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 0x55de00333bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.369+0000 7f43e1e00700 1 -- 172.21.15.99:0/3824301542 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de001053b0 con 0x55de003329e0 2024-03-30T00:36:28.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.370+0000 7f43e3200700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f43c80a11a0 0x7f43c80a3660 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f43dc095d20 tx=0x7f43dc009ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.370+0000 7f43e1e00700 1 --2- 172.21.15.99:0/3824301542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00332e00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55de00333970 tx=0x7f43d001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a829826fd9c17c69 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.370+0000 7f43cf400700 1 -- 172.21.15.99:0/3824301542 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00040f0 con 0x55de003329e0 2024-03-30T00:36:28.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.370+0000 7f43cf400700 1 -- 172.21.15.99:0/3824301542 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43d000b710 con 0x55de003329e0 2024-03-30T00:36:28.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.370+0000 7f43cf400700 1 -- 172.21.15.99:0/3824301542 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d000f6c0 con 0x55de003329e0 2024-03-30T00:36:28.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.371+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3824301542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 msgr2=0x55de00332e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.371+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3824301542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00332e00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55de00333970 tx=0x7f43d001a040 comp rx=0 tx=0).stop 2024-03-30T00:36:28.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.371+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3824301542 shutdown_connections 2024-03-30T00:36:28.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.371+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3824301542 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00334610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.371+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3824301542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00332e00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.371+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3824301542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 0x55de00333bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.371+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3824301542 >> 172.21.15.99:0/3824301542 conn(0x55de002916a0 msgr2=0x55de0032dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.371+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3824301542 shutdown_connections 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.371+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3824301542 wait complete. 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.372+0000 7f43fcf8fb80 1 -- start start 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.372+0000 7f43fcf8fb80 1 Processor -- start 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.372+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de003329e0 0x55de0033c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.372+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de00333340 0x55de0033cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.372+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00343dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.372+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0021fd80 con 0x55de00333340 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.372+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0021fa20 con 0x55de003329e0 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.372+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de001076a0 con 0x55de00334190 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.372+0000 7f43fcf8fb80 10 client.0 _ll_register_callbacks cb 0x55de00229470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:28.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e2800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de00333340 0x55de0033cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e2800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de00333340 0x55de0033cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59606/0 (socket says 172.21.15.99:59606) 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e2800700 1 -- 172.21.15.99:0/2701146110 learned_addr learned my addr 172.21.15.99:0/2701146110 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e1e00700 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de003329e0 0x55de0033c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e1e00700 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00343dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e2800700 1 -- 172.21.15.99:0/2701146110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 msgr2=0x55de00343dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e2800700 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00343dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e2800700 1 -- 172.21.15.99:0/2701146110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de003329e0 msgr2=0x55de0033c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e2800700 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de003329e0 0x55de0033c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e2800700 1 -- 172.21.15.99:0/2701146110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d0020040 con 0x55de00333340 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e1e00700 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de003329e0 0x55de0033c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43e2800700 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de00333340 0x55de0033cd80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f43d4012840 tx=0x7f43d4012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdb1da1d580d928 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.373+0000 7f43c3e00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.374+0000 7f43c3e00700 1 -- 172.21.15.99:0/2701146110 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4017070 con 0x55de00333340 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.374+0000 7f43c3e00700 1 -- 172.21.15.99:0/2701146110 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f43d4004030 con 0x55de00333340 2024-03-30T00:36:28.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.374+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/2701146110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de00244380 con 0x55de00333340 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.374+0000 7f43c3e00700 1 -- 172.21.15.99:0/2701146110 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43d4004230 con 0x55de00333340 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.374+0000 7f43fcf8fb80 10 client.15942 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.374+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/2701146110 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55de000b1b60 con 0x55de00333340 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.374+0000 7f43c3e00700 1 -- 172.21.15.99:0/2701146110 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55de00244380 con 0x55de00333340 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.374+0000 7f43c3e00700 1 -- 172.21.15.99:0/2701146110 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f43d400f5e0 con 0x55de00333340 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.374+0000 7f43fcf8fb80 10 client.15942 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.374+0000 7f43fcf8fb80 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.375+0000 7f43fcf8fb80 20 client.15942 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.375+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55de00348660 0x55de0034aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.375+0000 7f43fcf8fb80 4 client.15942 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.375+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/2701146110 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de0027c700 con 0x55de00348660 2024-03-30T00:36:28.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.375+0000 7f43e1e00700 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55de00348660 0x55de0034aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.375+0000 7f43e1e00700 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55de00348660 0x55de0034aa80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e761e28ff957cd0 server_cookie=d59f84d0e116c307 in_seq=0 out_seq=0 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.375+0000 7f43c3e00700 10 client.15942 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.376+0000 7f43c3e00700 1 -- 172.21.15.99:0/2701146110 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55de0027c700 con 0x55de00348660 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.376+0000 7f43c3e00700 10 client.15942 handle_command_reply: tid=0 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.376+0000 7f43fcf8fb80 1 client.15942 shutdown 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.376+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/2701146110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de00333340 msgr2=0x55de0033cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.376+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de00333340 0x55de0033cd80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f43d4012840 tx=0x7f43d4012c00 comp rx=0 tx=0).stop 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.376+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/2701146110 shutdown_connections 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.376+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55de00348660 0x55de0034aa80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.376+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00343dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.376+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de003329e0 0x55de0033c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.376+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/2701146110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de00333340 0x55de0033cd80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.376+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/2701146110 >> 172.21.15.99:0/2701146110 conn(0x55de002916a0 msgr2=0x55de003313c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.377+0000 7f43c3e00700 0 client.15942 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:28.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.377+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/2701146110 shutdown_connections 2024-03-30T00:36:28.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.377+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/2701146110 wait complete. 2024-03-30T00:36:28.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.377+0000 7f43fcf8fb80 20 client.15942 trim_cache size 0 max 16384 2024-03-30T00:36:28.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.398+0000 7f43fcf8fb80 1 Processor -- start 2024-03-30T00:36:28.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.398+0000 7f43fcf8fb80 1 -- start start 2024-03-30T00:36:28.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.398+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00334570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.398+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00333340 0x55de00343d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.398+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00334190 0x55de003373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.398+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de00333dc0 con 0x55de003329e0 2024-03-30T00:36:28.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.398+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0032eed0 con 0x55de00334190 2024-03-30T00:36:28.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de0032e640 con 0x55de00333340 2024-03-30T00:36:28.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00334570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43e2800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00333340 0x55de00343d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00334570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59616/0 (socket says 172.21.15.99:59616) 2024-03-30T00:36:28.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43e1e00700 1 -- 172.21.15.99:0/3693649069 learned_addr learned my addr 172.21.15.99:0/3693649069 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:28.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43e1e00700 1 --2- 172.21.15.99:0/3693649069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00334190 0x55de003373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43e2800700 1 -- 172.21.15.99:0/3693649069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00334190 msgr2=0x55de003373a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43e2800700 1 --2- 172.21.15.99:0/3693649069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00334190 0x55de003373a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43e2800700 1 -- 172.21.15.99:0/3693649069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 msgr2=0x55de00334570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43e2800700 1 --2- 172.21.15.99:0/3693649069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00334570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43e2800700 1 -- 172.21.15.99:0/3693649069 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de00249ea0 con 0x55de00333340 2024-03-30T00:36:28.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.399+0000 7f43e1e00700 1 --2- 172.21.15.99:0/3693649069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00334570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:28.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.400+0000 7f43e2800700 1 --2- 172.21.15.99:0/3693649069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00333340 0x55de00343d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f43d4000c00 tx=0x7f43d400ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6ed620439e50d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.400+0000 7f43cfe00700 1 -- 172.21.15.99:0/3693649069 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4017070 con 0x55de00333340 2024-03-30T00:36:28.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.400+0000 7f43cfe00700 1 -- 172.21.15.99:0/3693649069 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43d4018040 con 0x55de00333340 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.400+0000 7f43cfe00700 1 -- 172.21.15.99:0/3693649069 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d401c680 con 0x55de00333340 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.401+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3693649069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00333340 msgr2=0x55de00343d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.401+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3693649069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00333340 0x55de00343d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f43d4000c00 tx=0x7f43d400ff70 comp rx=0 tx=0).stop 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.401+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3693649069 shutdown_connections 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.401+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3693649069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00334190 0x55de003373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.401+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3693649069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00334570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.401+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3693649069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00333340 0x55de00343d80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.401+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3693649069 >> 172.21.15.99:0/3693649069 conn(0x55de002916a0 msgr2=0x55de003267f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.401+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3693649069 shutdown_connections 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.401+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3693649069 wait complete. 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.402+0000 7f43fcf8fb80 1 -- start start 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.402+0000 7f43fcf8fb80 1 Processor -- start 2024-03-30T00:36:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.402+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00343690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.402+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 0x55de00343bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.402+0000 7f43fcf8fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00341ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.402+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de00342380 con 0x55de003329e0 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.402+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de003424f0 con 0x55de00333340 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.402+0000 7f43fcf8fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de003426b0 con 0x55de00334190 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.402+0000 7f43fcf8fb80 10 client.0 _ll_register_callbacks cb 0x55de00229470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43e2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 0x55de00343bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00343690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43e2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 0x55de00343bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:56222/0 (socket says 172.21.15.99:56222) 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43e2800700 1 -- 172.21.15.99:0/3270724299 learned_addr learned my addr 172.21.15.99:0/3270724299 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43e1e00700 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00341ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43e2800700 1 -- 172.21.15.99:0/3270724299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 msgr2=0x55de00341ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43e2800700 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00341ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43e2800700 1 -- 172.21.15.99:0/3270724299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 msgr2=0x55de00343690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:28.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43e2800700 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00343690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43e2800700 1 -- 172.21.15.99:0/3270724299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de001053b0 con 0x55de00333340 2024-03-30T00:36:28.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43e2800700 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 0x55de00343bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f43d400fa70 tx=0x7f43d4009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fca365bc244eb345 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:28.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.403+0000 7f43c3e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:36:28.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.404+0000 7f43c3e00700 1 -- 172.21.15.99:0/3270724299 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4034070 con 0x55de00333340 2024-03-30T00:36:28.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.404+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3270724299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43d000b710 con 0x55de00333340 2024-03-30T00:36:28.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.404+0000 7f43c3e00700 1 -- 172.21.15.99:0/3270724299 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f43d4000e40 con 0x55de00333340 2024-03-30T00:36:28.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.404+0000 7f43c3e00700 1 -- 172.21.15.99:0/3270724299 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43d4017040 con 0x55de00333340 2024-03-30T00:36:28.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.404+0000 7f43fcf8fb80 10 client.25643 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:28.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.404+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3270724299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55de000b1b60 con 0x55de00333340 2024-03-30T00:36:28.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.404+0000 7f43c3e00700 1 -- 172.21.15.99:0/3270724299 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43d403e020 con 0x55de00333340 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.404+0000 7f43c3e00700 1 -- 172.21.15.99:0/3270724299 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f43d4037030 con 0x55de00333340 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.405+0000 7f43fcf8fb80 10 client.25643 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.405+0000 7f43fcf8fb80 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.405+0000 7f43fcf8fb80 20 client.25643 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.405+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55de0039d490 0x55de0039f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.405+0000 7f43fcf8fb80 4 client.25643 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.405+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3270724299 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55de000d2cf0 con 0x55de0039d490 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.405+0000 7f43e1e00700 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55de0039d490 0x55de0039f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43c3e00700 10 client.25643 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.405+0000 7f43e1e00700 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55de0039d490 0x55de0039f880 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ccf16bc4c806dc9 server_cookie=17db8f8f3fee481b in_seq=0 out_seq=0 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43c3e00700 1 -- 172.21.15.99:0/3270724299 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55de000d2cf0 con 0x55de0039d490 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43c3e00700 10 client.25643 handle_command_reply: tid=0 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43fcf8fb80 1 client.25643 shutdown 2024-03-30T00:36:28.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3270724299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 msgr2=0x55de00343bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 0x55de00343bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f43d400fa70 tx=0x7f43d4009370 comp rx=0 tx=0).stop 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3270724299 shutdown_connections 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55de0039d490 0x55de0039f880 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55de00334190 0x55de00341ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55de003329e0 0x55de00343690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43fcf8fb80 1 --2- 172.21.15.99:0/3270724299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de00333340 0x55de00343bd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43c3e00700 0 client.25643 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3270724299 >> 172.21.15.99:0/3270724299 conn(0x55de002916a0 msgr2=0x55de003373a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3270724299 shutdown_connections 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.406+0000 7f43fcf8fb80 1 -- 172.21.15.99:0/3270724299 wait complete. 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.407+0000 7f43fcf8fb80 20 client.25643 trim_cache size 0 max 16384 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:36:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:36:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": { 2024-03-30T00:36:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "d1ec7919-6ea4-456e-8da7-b149b92d9937": { 2024-03-30T00:36:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/", 2024-03-30T00:36:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tag": "d1ec7919-6ea4-456e-8da7-b149b92d9937", 2024-03-30T00:36:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "options": "recursive,force" 2024-03-30T00:36:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:36:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:36:28.418 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.407+0000 7f43e8a00700 1 -- 172.21.15.99:0/2364082271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f43c80a11a0 msgr2=0x7f43c80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.407+0000 7f43e8a00700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f43c80a11a0 0x7f43c80a3660 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f43dc095d20 tx=0x7f43dc009ed0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.407+0000 7f43e8a00700 1 -- 172.21.15.99:0/2364082271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e41068f0 msgr2=0x7f43e41a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.407+0000 7f43e8a00700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e41068f0 0x7f43e41a0230 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00ef30 tx=0x7f43dc00c3a0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.407+0000 7f43e8a00700 1 -- 172.21.15.99:0/2364082271 shutdown_connections 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.407+0000 7f43e8a00700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e41068f0 0x7f43e41a0230 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.407+0000 7f43e8a00700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f43c80a11a0 0x7f43c80a3660 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.407+0000 7f43e8a00700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43e4101690 0x7f43e4198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.407+0000 7f43e8a00700 1 --2- 172.21.15.99:0/2364082271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43e4103fc0 0x7f43e4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.407+0000 7f43e8a00700 1 -- 172.21.15.99:0/2364082271 >> 172.21.15.99:0/2364082271 conn(0x7f43e40fb000 msgr2=0x7f43e40ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.408+0000 7f43e8a00700 1 -- 172.21.15.99:0/2364082271 shutdown_connections 2024-03-30T00:36:28.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:28.408+0000 7f43e8a00700 1 -- 172.21.15.99:0/2364082271 wait complete. 2024-03-30T00:36:28.433 INFO:tasks.cephfs.filesystem:scrub status for tag:d1ec7919-6ea4-456e-8da7-b149b92d9937 - {'path': '/', 'tag': 'd1ec7919-6ea4-456e-8da7-b149b92d9937', 'options': 'recursive,force'} 2024-03-30T00:36:28.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:28 smithi099 ceph-mon[38079]: pgmap v356: 97 pgs: 97 active+clean; 201 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-03-30T00:36:28.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:28 smithi142 ceph-mon[41779]: pgmap v356: 97 pgs: 97 active+clean; 201 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-03-30T00:36:28.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:28 smithi161 ceph-mon[42221]: pgmap v356: 97 pgs: 97 active+clean; 201 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-03-30T00:36:29.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:29 smithi142 ceph-mon[41779]: scrub queued for path: / 2024-03-30T00:36:29.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:29 smithi142 ceph-mon[41779]: scrub summary: idle+waiting paths [/] 2024-03-30T00:36:29.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:29 smithi142 ceph-mon[41779]: scrub summary: active paths [/] 2024-03-30T00:36:29.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:29 smithi142 ceph-mon[41779]: scrub summary: idle+waiting paths [/] 2024-03-30T00:36:29.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:29 smithi142 ceph-mon[41779]: scrub summary: idle 2024-03-30T00:36:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:29 smithi161 ceph-mon[42221]: scrub queued for path: / 2024-03-30T00:36:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:29 smithi161 ceph-mon[42221]: scrub summary: idle+waiting paths [/] 2024-03-30T00:36:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:29 smithi161 ceph-mon[42221]: scrub summary: active paths [/] 2024-03-30T00:36:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:29 smithi161 ceph-mon[42221]: scrub summary: idle+waiting paths [/] 2024-03-30T00:36:29.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:29 smithi161 ceph-mon[42221]: scrub summary: idle 2024-03-30T00:36:29.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:29 smithi099 ceph-mon[38079]: scrub queued for path: / 2024-03-30T00:36:29.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:29 smithi099 ceph-mon[38079]: scrub summary: idle+waiting paths [/] 2024-03-30T00:36:29.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:29 smithi099 ceph-mon[38079]: scrub summary: active paths [/] 2024-03-30T00:36:29.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:29 smithi099 ceph-mon[38079]: scrub summary: idle+waiting paths [/] 2024-03-30T00:36:29.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:29 smithi099 ceph-mon[38079]: scrub summary: idle 2024-03-30T00:36:30.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:30 smithi142 ceph-mon[41779]: pgmap v357: 97 pgs: 97 active+clean; 238 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 18 MiB/s wr, 22 op/s 2024-03-30T00:36:30.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:30 smithi161 ceph-mon[42221]: pgmap v357: 97 pgs: 97 active+clean; 238 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 18 MiB/s wr, 22 op/s 2024-03-30T00:36:30.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:30 smithi099 ceph-mon[38079]: pgmap v357: 97 pgs: 97 active+clean; 238 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 18 MiB/s wr, 22 op/s 2024-03-30T00:36:32.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:32 smithi142 ceph-mon[41779]: pgmap v358: 97 pgs: 97 active+clean; 254 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 16 MiB/s wr, 108 op/s 2024-03-30T00:36:32.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:32 smithi161 ceph-mon[42221]: pgmap v358: 97 pgs: 97 active+clean; 254 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 16 MiB/s wr, 108 op/s 2024-03-30T00:36:32.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:32 smithi099 ceph-mon[38079]: pgmap v358: 97 pgs: 97 active+clean; 254 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 16 MiB/s wr, 108 op/s 2024-03-30T00:36:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.495+0000 7f14bc400700 1 -- 172.21.15.99:0/814388398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b4074d90 msgr2=0x7f14b40731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.495+0000 7f14bc400700 1 --2- 172.21.15.99:0/814388398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b4074d90 0x7f14b40731e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009990 tx=0x7f14a8009ca0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.495+0000 7f14bc400700 1 -- 172.21.15.99:0/814388398 shutdown_connections 2024-03-30T00:36:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.495+0000 7f14bc400700 1 --2- 172.21.15.99:0/814388398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4108be0 0x7f14b410b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.495+0000 7f14bc400700 1 --2- 172.21.15.99:0/814388398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b4074d90 0x7f14b40731e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.495+0000 7f14bc400700 1 --2- 172.21.15.99:0/814388398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b40737b0 0x7f14b4073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.495+0000 7f14bc400700 1 -- 172.21.15.99:0/814388398 >> 172.21.15.99:0/814388398 conn(0x7f14b40fc2a0 msgr2=0x7f14b40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.495+0000 7f14bc400700 1 -- 172.21.15.99:0/814388398 shutdown_connections 2024-03-30T00:36:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.496+0000 7f14bc400700 1 -- 172.21.15.99:0/814388398 wait complete. 2024-03-30T00:36:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14bc400700 1 Processor -- start 2024-03-30T00:36:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14bc400700 1 -- start start 2024-03-30T00:36:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14bc400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b40737b0 0x7f14b4198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14bc400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b4074d90 0x7f14b41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14bc400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4108be0 0x7f14b41a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14bc400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41a0880 con 0x7f14b4108be0 2024-03-30T00:36:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14bc400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41a09f0 con 0x7f14b4074d90 2024-03-30T00:36:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14bc400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41a0bb0 con 0x7f14b40737b0 2024-03-30T00:36:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14b9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b4074d90 0x7f14b41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14ba400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4108be0 0x7f14b41a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14b9a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b40737b0 0x7f14b4198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14b9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b4074d90 0x7f14b41992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:56238/0 (socket says 172.21.15.99:56238) 2024-03-30T00:36:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.497+0000 7f14b9000700 1 -- 172.21.15.99:0/658490931 learned_addr learned my addr 172.21.15.99:0/658490931 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.498+0000 7f14b9000700 1 -- 172.21.15.99:0/658490931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b40737b0 msgr2=0x7f14b4198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.498+0000 7f14b9000700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b40737b0 0x7f14b4198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.498+0000 7f14b9000700 1 -- 172.21.15.99:0/658490931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4108be0 msgr2=0x7f14b41a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.498+0000 7f14b9000700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4108be0 0x7f14b41a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.498+0000 7f14b9000700 1 -- 172.21.15.99:0/658490931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a80095f0 con 0x7f14b4074d90 2024-03-30T00:36:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.498+0000 7f14ba400700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4108be0 0x7f14b41a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.499+0000 7f14b9000700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b4074d90 0x7f14b41992d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e9a0 tx=0x7f14a000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.499+0000 7f14b2000700 1 -- 172.21.15.99:0/658490931 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0009790 con 0x7f14b4074d90 2024-03-30T00:36:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.499+0000 7f14b2000700 1 -- 172.21.15.99:0/658490931 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14a0010430 con 0x7f14b4074d90 2024-03-30T00:36:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.500+0000 7f14bc400700 1 -- 172.21.15.99:0/658490931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b41a0e90 con 0x7f14b4074d90 2024-03-30T00:36:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.500+0000 7f14bc400700 1 -- 172.21.15.99:0/658490931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b41a1310 con 0x7f14b4074d90 2024-03-30T00:36:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.500+0000 7f14b2000700 1 -- 172.21.15.99:0/658490931 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000cea0 con 0x7f14b4074d90 2024-03-30T00:36:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.506+0000 7f14cfebeb80 1 Processor -- start 2024-03-30T00:36:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.506+0000 7f14cfebeb80 1 -- start start 2024-03-30T00:36:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.507+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b813a10 0x55703b813e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.507+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b814ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.507+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b815270 0x55703b815700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.507+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b763930 con 0x55703b813a10 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.507+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b7612f0 con 0x55703b815270 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.507+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b75cbe0 con 0x55703b814370 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.507+0000 7f14b9a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b814ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.507+0000 7f14b9a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b814ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:59610/0 (socket says 172.21.15.99:59610) 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.507+0000 7f14ba400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b813a10 0x55703b813e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.507+0000 7f14b9a00700 1 -- 172.21.15.99:0/162920267 learned_addr learned my addr 172.21.15.99:0/162920267 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.508+0000 7f14b2000700 1 -- 172.21.15.99:0/658490931 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f14a000ca10 con 0x7f14b4074d90 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.508+0000 7f14b9a00700 1 -- 172.21.15.99:0/162920267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b815270 msgr2=0x55703b815700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.508+0000 7f14b9a00700 1 --2- 172.21.15.99:0/162920267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b815270 0x55703b815700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.508+0000 7f14b9a00700 1 -- 172.21.15.99:0/162920267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b813a10 msgr2=0x55703b813e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.508+0000 7f14b9a00700 1 --2- 172.21.15.99:0/162920267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b813a10 0x55703b813e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.508+0000 7f14b9a00700 1 -- 172.21.15.99:0/162920267 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55703b5de350 con 0x55703b814370 2024-03-30T00:36:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.508+0000 7f14b9a00700 1 --2- 172.21.15.99:0/162920267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b814ce0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009990 tx=0x7f14a8005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b99a55b5207b728a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.508+0000 7f14a6a00700 1 -- 172.21.15.99:0/162920267 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a801d070 con 0x55703b814370 2024-03-30T00:36:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.509+0000 7f14a6a00700 1 -- 172.21.15.99:0/162920267 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14a801e040 con 0x55703b814370 2024-03-30T00:36:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.509+0000 7f14a6a00700 1 -- 172.21.15.99:0/162920267 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800bb90 con 0x55703b814370 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.509+0000 7f14b2000700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f149c0a11a0 0x7f149c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.509+0000 7f14cfebeb80 1 -- 172.21.15.99:0/162920267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 msgr2=0x55703b814ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.509+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/162920267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b814ce0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009990 tx=0x7f14a8005ea0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.510+0000 7f14cfebeb80 1 -- 172.21.15.99:0/162920267 shutdown_connections 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.510+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/162920267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b815270 0x55703b815700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.510+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/162920267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b813a10 0x55703b813e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.510+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/162920267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b814ce0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.510+0000 7f14cfebeb80 1 -- 172.21.15.99:0/162920267 >> 172.21.15.99:0/162920267 conn(0x55703b76a640 msgr2=0x55703b806050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.510+0000 7f14b2000700 1 -- 172.21.15.99:0/658490931 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f14a0014070 con 0x7f14b4074d90 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.510+0000 7f14ba400700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f149c0a11a0 0x7f149c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.510+0000 7f14cfebeb80 1 -- 172.21.15.99:0/162920267 shutdown_connections 2024-03-30T00:36:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.510+0000 7f14cfebeb80 1 -- 172.21.15.99:0/162920267 wait complete. 2024-03-30T00:36:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.510+0000 7f14ba400700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f149c0a11a0 0x7f149c0a3660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f14ac003fd0 tx=0x7f14ac00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.511+0000 7f14cfebeb80 1 -- start start 2024-03-30T00:36:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.511+0000 7f14cfebeb80 1 Processor -- start 2024-03-30T00:36:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b813a10 0x55703b81d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b814370 0x55703b81dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b824f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b6f8d20 con 0x55703b815270 2024-03-30T00:36:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b6f89c0 con 0x55703b814370 2024-03-30T00:36:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b5e0640 con 0x55703b813a10 2024-03-30T00:36:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14cfebeb80 10 client.0 _ll_register_callbacks cb 0x55703b702410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14ba400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b824f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14ba400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b824f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59656/0 (socket says 172.21.15.99:59656) 2024-03-30T00:36:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14ba400700 1 -- 172.21.15.99:0/1660300921 learned_addr learned my addr 172.21.15.99:0/1660300921 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14b9000700 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b814370 0x55703b81dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14b9a00700 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b813a10 0x55703b81d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14ba400700 1 -- 172.21.15.99:0/1660300921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b813a10 msgr2=0x55703b81d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14ba400700 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b813a10 0x55703b81d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14ba400700 1 -- 172.21.15.99:0/1660300921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b814370 msgr2=0x55703b81dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14ba400700 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b814370 0x55703b81dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.513+0000 7f14ba400700 1 -- 172.21.15.99:0/1660300921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b410bc70 con 0x55703b815270 2024-03-30T00:36:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.514+0000 7f14ba400700 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b824f00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009ec0 tx=0x7f14ac004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad803268c0e5c49 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.514+0000 7f149b400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:36:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.514+0000 7f149b400700 1 -- 172.21.15.99:0/1660300921 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00e900 con 0x55703b815270 2024-03-30T00:36:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.514+0000 7f149b400700 1 -- 172.21.15.99:0/1660300921 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f14ac00ea60 con 0x55703b815270 2024-03-30T00:36:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.514+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1660300921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55703b71d320 con 0x55703b815270 2024-03-30T00:36:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.515+0000 7f149b400700 1 -- 172.21.15.99:0/1660300921 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14ac00ec20 con 0x55703b815270 2024-03-30T00:36:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.515+0000 7f14cfebeb80 10 client.15960 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.515+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1660300921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55703b58ab00 con 0x55703b815270 2024-03-30T00:36:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.515+0000 7f149b400700 1 -- 172.21.15.99:0/1660300921 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55703b71d320 con 0x55703b815270 2024-03-30T00:36:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.515+0000 7f149b400700 1 -- 172.21.15.99:0/1660300921 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f14ac016400 con 0x55703b815270 2024-03-30T00:36:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.515+0000 7f14cfebeb80 10 client.15960 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.515+0000 7f14cfebeb80 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.515+0000 7f14cfebeb80 20 client.15960 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.515+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55703b829870 0x55703b82bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.515+0000 7f14cfebeb80 4 client.15960 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.515+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1660300921 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55703b7556a0 con 0x55703b829870 2024-03-30T00:36:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.516+0000 7f14b9a00700 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55703b829870 0x55703b82bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.516+0000 7f14b9a00700 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55703b829870 0x55703b82bc90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2943e1d12a2de70a server_cookie=9864bfa11b312f66 in_seq=0 out_seq=0 2024-03-30T00:36:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.516+0000 7f149b400700 10 client.15960 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.517+0000 7f149b400700 1 -- 172.21.15.99:0/1660300921 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55703b7556a0 con 0x55703b829870 2024-03-30T00:36:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.517+0000 7f149b400700 10 client.15960 handle_command_reply: tid=0 2024-03-30T00:36:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.517+0000 7f14cfebeb80 1 client.15960 shutdown 2024-03-30T00:36:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1660300921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 msgr2=0x55703b824f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b824f00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009ec0 tx=0x7f14ac004b40 comp rx=0 tx=0).stop 2024-03-30T00:36:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1660300921 shutdown_connections 2024-03-30T00:36:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55703b829870 0x55703b82bc90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b824f00 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b813a10 0x55703b81d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/1660300921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b814370 0x55703b81dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1660300921 >> 172.21.15.99:0/1660300921 conn(0x55703b76a640 msgr2=0x55703b8127c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f149b400700 0 client.15960 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1660300921 shutdown_connections 2024-03-30T00:36:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1660300921 wait complete. 2024-03-30T00:36:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.518+0000 7f14cfebeb80 20 client.15960 trim_cache size 0 max 16384 2024-03-30T00:36:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.543+0000 7f14cfebeb80 1 Processor -- start 2024-03-30T00:36:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.543+0000 7f14cfebeb80 1 -- start start 2024-03-30T00:36:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b813a10 0x55703b800500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b824ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b81d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b806d70 con 0x55703b815270 2024-03-30T00:36:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b81d4f0 con 0x55703b813a10 2024-03-30T00:36:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b8065e0 con 0x55703b814370 2024-03-30T00:36:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14ba400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b81d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14b9000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b824ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14b9000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b824ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:59624/0 (socket says 172.21.15.99:59624) 2024-03-30T00:36:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14b9000700 1 -- 172.21.15.99:0/1759397880 learned_addr learned my addr 172.21.15.99:0/1759397880 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14ba400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b81d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59662/0 (socket says 172.21.15.99:59662) 2024-03-30T00:36:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.544+0000 7f14b9a00700 1 --2- 172.21.15.99:0/1759397880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b813a10 0x55703b800500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.545+0000 7f14b9a00700 1 -- 172.21.15.99:0/1759397880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 msgr2=0x55703b824ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.545+0000 7f14b9a00700 1 --2- 172.21.15.99:0/1759397880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b824ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.545+0000 7f14b9a00700 1 -- 172.21.15.99:0/1759397880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 msgr2=0x55703b81d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.545+0000 7f14b9a00700 1 --2- 172.21.15.99:0/1759397880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b81d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.545+0000 7f14b9a00700 1 -- 172.21.15.99:0/1759397880 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55703b722e40 con 0x55703b813a10 2024-03-30T00:36:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.545+0000 7f14ba400700 1 --2- 172.21.15.99:0/1759397880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b81d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.545+0000 7f14b9a00700 1 --2- 172.21.15.99:0/1759397880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b813a10 0x55703b800500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f14a80098e0 tx=0x7f14a801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=574a4f29c1fec047 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.545+0000 7f14a7400700 1 -- 172.21.15.99:0/1759397880 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a801d040 con 0x55703b813a10 2024-03-30T00:36:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.546+0000 7f14a7400700 1 -- 172.21.15.99:0/1759397880 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14a8003b10 con 0x55703b813a10 2024-03-30T00:36:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.546+0000 7f14a7400700 1 -- 172.21.15.99:0/1759397880 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8004bd0 con 0x55703b813a10 2024-03-30T00:36:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.546+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1759397880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b813a10 msgr2=0x55703b800500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.546+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/1759397880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b813a10 0x55703b800500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f14a80098e0 tx=0x7f14a801a040 comp rx=0 tx=0).stop 2024-03-30T00:36:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.546+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1759397880 shutdown_connections 2024-03-30T00:36:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.546+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/1759397880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b815270 0x55703b81d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.546+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/1759397880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b813a10 0x55703b800500 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.546+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/1759397880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b824ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.546+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1759397880 >> 172.21.15.99:0/1759397880 conn(0x55703b76a640 msgr2=0x55703b81b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.546+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1759397880 shutdown_connections 2024-03-30T00:36:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.547+0000 7f14cfebeb80 1 -- 172.21.15.99:0/1759397880 wait complete. 2024-03-30T00:36:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.547+0000 7f14cfebeb80 1 -- start start 2024-03-30T00:36:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.547+0000 7f14cfebeb80 1 Processor -- start 2024-03-30T00:36:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.548+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b813a10 0x55703b822c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.548+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b8231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.548+0000 7f14cfebeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b815270 0x55703b824080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.548+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b8245c0 con 0x55703b813a10 2024-03-30T00:36:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.548+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b824730 con 0x55703b815270 2024-03-30T00:36:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.548+0000 7f14cfebeb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703b8248f0 con 0x55703b814370 2024-03-30T00:36:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.548+0000 7f14cfebeb80 10 client.0 _ll_register_callbacks cb 0x55703b702410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.549+0000 7f14ba400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b815270 0x55703b824080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.549+0000 7f14ba400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b815270 0x55703b824080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:56272/0 (socket says 172.21.15.99:56272) 2024-03-30T00:36:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.549+0000 7f14ba400700 1 -- 172.21.15.99:0/2951836302 learned_addr learned my addr 172.21.15.99:0/2951836302 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.549+0000 7f14b9a00700 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b813a10 0x55703b822c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.549+0000 7f14b9000700 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b8231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.549+0000 7f14ba400700 1 -- 172.21.15.99:0/2951836302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 msgr2=0x55703b8231b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.549+0000 7f14ba400700 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b8231b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.549+0000 7f14ba400700 1 -- 172.21.15.99:0/2951836302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b813a10 msgr2=0x55703b822c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.549+0000 7f14ba400700 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b813a10 0x55703b822c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.549+0000 7f14ba400700 1 -- 172.21.15.99:0/2951836302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55703b58ab00 con 0x55703b815270 2024-03-30T00:36:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.549+0000 7f14b9a00700 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b813a10 0x55703b822c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.550+0000 7f14ba400700 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b815270 0x55703b824080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14ac006090 tx=0x7f14ac009a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddf8c35af5e37d7b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.550+0000 7f149b400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:36:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.550+0000 7f149b400700 1 -- 172.21.15.99:0/2951836302 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00e900 con 0x55703b815270 2024-03-30T00:36:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.550+0000 7f14cfebeb80 1 -- 172.21.15.99:0/2951836302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14a801e040 con 0x55703b815270 2024-03-30T00:36:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.550+0000 7f149b400700 1 -- 172.21.15.99:0/2951836302 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f14ac00ea60 con 0x55703b815270 2024-03-30T00:36:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.550+0000 7f149b400700 1 -- 172.21.15.99:0/2951836302 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14ac00ec20 con 0x55703b815270 2024-03-30T00:36:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.551+0000 7f14cfebeb80 10 client.25658 fetch_fsmap learned FSMap version 24 2024-03-30T00:36:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.551+0000 7f14cfebeb80 1 -- 172.21.15.99:0/2951836302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55703b5de350 con 0x55703b815270 2024-03-30T00:36:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.551+0000 7f149b400700 1 -- 172.21.15.99:0/2951836302 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14a801e040 con 0x55703b815270 2024-03-30T00:36:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.552+0000 7f149b400700 1 -- 172.21.15.99:0/2951836302 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f14ac03f030 con 0x55703b815270 2024-03-30T00:36:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.552+0000 7f14cfebeb80 10 client.25658 fetch_fsmap finished waiting for FSMap version 24 2024-03-30T00:36:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.552+0000 7f14cfebeb80 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.552+0000 7f14cfebeb80 20 client.25658 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.552+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55703b87e490 0x55703b8808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.552+0000 7f14cfebeb80 4 client.25658 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:36:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.552+0000 7f14cfebeb80 1 -- 172.21.15.99:0/2951836302 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55703b5abc90 con 0x55703b87e490 2024-03-30T00:36:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.552+0000 7f14b9a00700 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55703b87e490 0x55703b8808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.553+0000 7f149b400700 10 client.25658 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.553+0000 7f14b9a00700 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55703b87e490 0x55703b8808b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84209f78e3458d4 server_cookie=bfe6b5fa87d2dd97 in_seq=0 out_seq=0 2024-03-30T00:36:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f149b400700 1 -- 172.21.15.99:0/2951836302 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55703b5abc90 con 0x55703b87e490 2024-03-30T00:36:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f149b400700 10 client.25658 handle_command_reply: tid=0 2024-03-30T00:36:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f14cfebeb80 1 client.25658 shutdown 2024-03-30T00:36:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f14cfebeb80 1 -- 172.21.15.99:0/2951836302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b815270 msgr2=0x55703b824080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b815270 0x55703b824080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14ac006090 tx=0x7f14ac009a50 comp rx=0 tx=0).stop 2024-03-30T00:36:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f14cfebeb80 1 -- 172.21.15.99:0/2951836302 shutdown_connections 2024-03-30T00:36:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55703b87e490 0x55703b8808b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.570 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55703b815270 0x55703b824080 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55703b813a10 0x55703b822c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f149b400700 0 client.25658 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f14cfebeb80 1 --2- 172.21.15.99:0/2951836302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55703b814370 0x55703b8231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.563+0000 7f14cfebeb80 1 -- 172.21.15.99:0/2951836302 >> 172.21.15.99:0/2951836302 conn(0x55703b76a640 msgr2=0x55703b81d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.564+0000 7f14cfebeb80 1 -- 172.21.15.99:0/2951836302 shutdown_connections 2024-03-30T00:36:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.564+0000 7f14cfebeb80 1 -- 172.21.15.99:0/2951836302 wait complete. 2024-03-30T00:36:33.571 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.564+0000 7f14cfebeb80 20 client.25658 trim_cache size 0 max 16384 2024-03-30T00:36:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:36:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:36:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:36:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:36:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.565+0000 7f14bc400700 1 -- 172.21.15.99:0/658490931 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f149c0a11a0 msgr2=0x7f149c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.565+0000 7f14bc400700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f149c0a11a0 0x7f149c0a3660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f14ac003fd0 tx=0x7f14ac00b040 comp rx=0 tx=0).stop 2024-03-30T00:36:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.565+0000 7f14bc400700 1 -- 172.21.15.99:0/658490931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b4074d90 msgr2=0x7f14b41992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.565+0000 7f14bc400700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b4074d90 0x7f14b41992d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e9a0 tx=0x7f14a000ecb0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.565+0000 7f14bc400700 1 -- 172.21.15.99:0/658490931 shutdown_connections 2024-03-30T00:36:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.565+0000 7f14bc400700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4108be0 0x7f14b41a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.565+0000 7f14bc400700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14b40737b0 0x7f14b4198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.565+0000 7f14bc400700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f149c0a11a0 0x7f149c0a3660 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.565+0000 7f14bc400700 1 --2- 172.21.15.99:0/658490931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14b4074d90 0x7f14b41992d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.565+0000 7f14bc400700 1 -- 172.21.15.99:0/658490931 >> 172.21.15.99:0/658490931 conn(0x7f14b40fc2a0 msgr2=0x7f14b4104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.566+0000 7f14bc400700 1 -- 172.21.15.99:0/658490931 shutdown_connections 2024-03-30T00:36:33.573 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:33.566+0000 7f14bc400700 1 -- 172.21.15.99:0/658490931 wait complete. 2024-03-30T00:36:34.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:34 smithi142 ceph-mon[41779]: pgmap v359: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 MiB/s wr, 395 op/s 2024-03-30T00:36:34.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:34 smithi161 ceph-mon[42221]: pgmap v359: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 MiB/s wr, 395 op/s 2024-03-30T00:36:34.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:34 smithi099 ceph-mon[38079]: pgmap v359: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 15 MiB/s wr, 395 op/s 2024-03-30T00:36:36.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:36 smithi142 ceph-mon[41779]: pgmap v360: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 15 MiB/s wr, 976 op/s 2024-03-30T00:36:36.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:36 smithi161 ceph-mon[42221]: pgmap v360: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 15 MiB/s wr, 976 op/s 2024-03-30T00:36:36.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:36 smithi099 ceph-mon[38079]: pgmap v360: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 15 MiB/s wr, 976 op/s 2024-03-30T00:36:38.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:38 smithi142 ceph-mon[41779]: pgmap v361: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-03-30T00:36:38.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:38 smithi161 ceph-mon[42221]: pgmap v361: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-03-30T00:36:38.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:38 smithi099 ceph-mon[38079]: pgmap v361: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-03-30T00:36:40.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:40 smithi142 ceph-mon[41779]: pgmap v362: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 14 MiB/s wr, 1.96k op/s 2024-03-30T00:36:40.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:40 smithi142 ceph-mon[41779]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:36:40.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:40 smithi161 ceph-mon[42221]: pgmap v362: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 14 MiB/s wr, 1.96k op/s 2024-03-30T00:36:40.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:40 smithi161 ceph-mon[42221]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:36:40.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:40 smithi099 ceph-mon[38079]: pgmap v362: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 14 MiB/s wr, 1.96k op/s 2024-03-30T00:36:40.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:40 smithi099 ceph-mon[38079]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:36:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:42 smithi142 ceph-mon[41779]: pgmap v363: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 2.34k op/s 2024-03-30T00:36:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:42 smithi161 ceph-mon[42221]: pgmap v363: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 2.34k op/s 2024-03-30T00:36:42.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:42 smithi099 ceph-mon[38079]: pgmap v363: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 2.34k op/s 2024-03-30T00:36:44.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:44 smithi142 ceph-mon[41779]: pgmap v364: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 2.67k op/s 2024-03-30T00:36:44.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:44 smithi161 ceph-mon[42221]: pgmap v364: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 2.67k op/s 2024-03-30T00:36:44.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:44 smithi099 ceph-mon[38079]: pgmap v364: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 2.67k op/s 2024-03-30T00:36:46.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:46 smithi142 ceph-mon[41779]: pgmap v365: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 13 MiB/s wr, 2.97k op/s 2024-03-30T00:36:46.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:46 smithi161 ceph-mon[42221]: pgmap v365: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 13 MiB/s wr, 2.97k op/s 2024-03-30T00:36:46.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:46 smithi099 ceph-mon[38079]: pgmap v365: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 13 MiB/s wr, 2.97k op/s 2024-03-30T00:36:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.772+0000 7f3807800700 1 -- 172.21.15.99:0/4101389344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3800101780 msgr2=0x7f3800103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.772+0000 7f3807800700 1 --2- 172.21.15.99:0/4101389344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3800101780 0x7f3800103b70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009990 tx=0x7f37f4009ca0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.772+0000 7f3807800700 1 -- 172.21.15.99:0/4101389344 shutdown_connections 2024-03-30T00:36:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.772+0000 7f3807800700 1 --2- 172.21.15.99:0/4101389344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38001069e0 0x7f3800108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.772+0000 7f3807800700 1 --2- 172.21.15.99:0/4101389344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3800101780 0x7f3800103b70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.772+0000 7f3807800700 1 --2- 172.21.15.99:0/4101389344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38001040b0 0x7f38001064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.772+0000 7f3807800700 1 -- 172.21.15.99:0/4101389344 >> 172.21.15.99:0/4101389344 conn(0x7f38000fb170 msgr2=0x7f38000fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.772+0000 7f3807800700 1 -- 172.21.15.99:0/4101389344 shutdown_connections 2024-03-30T00:36:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.772+0000 7f3807800700 1 -- 172.21.15.99:0/4101389344 wait complete. 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.773+0000 7f3807800700 1 Processor -- start 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.774+0000 7f3807800700 1 -- start start 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.774+0000 7f3807800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3800101780 0x7f3800196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.774+0000 7f3807800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38001040b0 0x7f38001970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.774+0000 7f3807800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38001069e0 0x7f380019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.774+0000 7f3807800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380019e6b0 con 0x7f3800101780 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.774+0000 7f3807800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380019e7f0 con 0x7f38001069e0 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.774+0000 7f3807800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380019e980 con 0x7f38001040b0 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.774+0000 7f3804e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3800101780 0x7f3800196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.774+0000 7f3804e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3800101780 0x7f3800196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:32944/0 (socket says 172.21.15.99:32944) 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.774+0000 7f3804e00700 1 -- 172.21.15.99:0/3121539292 learned_addr learned my addr 172.21.15.99:0/3121539292 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.774+0000 7f37ffe00700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38001040b0 0x7f38001970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.775+0000 7f3805800700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38001069e0 0x7f380019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.775+0000 7f3804e00700 1 -- 172.21.15.99:0/3121539292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38001040b0 msgr2=0x7f38001970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.775+0000 7f3804e00700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38001040b0 0x7f38001970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.775+0000 7f3804e00700 1 -- 172.21.15.99:0/3121539292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38001069e0 msgr2=0x7f380019e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.775+0000 7f3804e00700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38001069e0 0x7f380019e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.775+0000 7f3804e00700 1 -- 172.21.15.99:0/3121539292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec009520 con 0x7f3800101780 2024-03-30T00:36:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.775+0000 7f3805800700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38001069e0 0x7f380019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.776+0000 7f3804e00700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3800101780 0x7f3800196b80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f37f4005740 tx=0x7f37f4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.776+0000 7f37fd600700 1 -- 172.21.15.99:0/3121539292 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f401d070 con 0x7f3800101780 2024-03-30T00:36:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.776+0000 7f37fd600700 1 -- 172.21.15.99:0/3121539292 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37f400beb0 con 0x7f3800101780 2024-03-30T00:36:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.776+0000 7f37fd600700 1 -- 172.21.15.99:0/3121539292 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4021410 con 0x7f3800101780 2024-03-30T00:36:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.776+0000 7f3807800700 1 -- 172.21.15.99:0/3121539292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f40095f0 con 0x7f3800101780 2024-03-30T00:36:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.776+0000 7f3807800700 1 -- 172.21.15.99:0/3121539292 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380019ee40 con 0x7f3800101780 2024-03-30T00:36:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.780+0000 7f37fd600700 1 -- 172.21.15.99:0/3121539292 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f37f400f720 con 0x7f3800101780 2024-03-30T00:36:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.782+0000 7f37fd600700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37e80a11a0 0x7f37e80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.782+0000 7f37fd600700 1 -- 172.21.15.99:0/3121539292 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f37f40cdd00 con 0x7f3800101780 2024-03-30T00:36:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.782+0000 7f381b334b80 1 Processor -- start 2024-03-30T00:36:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.783+0000 7f37ffe00700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37e80a11a0 0x7f37e80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.783+0000 7f37ffe00700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37e80a11a0 0x7f37e80a3660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009df0 tx=0x7f37ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.783+0000 7f381b334b80 1 -- start start 2024-03-30T00:36:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.784+0000 7f381b334b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7610 0x5558291f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.784+0000 7f381b334b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7f70 0x5558291f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.784+0000 7f381b334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f8e70 0x5558291f9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.784+0000 7f381b334b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55582914b930 con 0x5558291f8e70 2024-03-30T00:36:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.784+0000 7f381b334b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558291492f0 con 0x5558291f7610 2024-03-30T00:36:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.784+0000 7f381b334b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555829144be0 con 0x5558291f7f70 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.784+0000 7f37ffe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7f70 0x5558291f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.784+0000 7f3804e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7610 0x5558291f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.784+0000 7f37ffe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7f70 0x5558291f88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:58006/0 (socket says 172.21.15.99:58006) 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.784+0000 7f37ffe00700 1 -- 172.21.15.99:0/4254121585 learned_addr learned my addr 172.21.15.99:0/4254121585 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.785+0000 7f3805800700 1 --2- 172.21.15.99:0/4254121585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f8e70 0x5558291f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.785+0000 7f37ffe00700 1 -- 172.21.15.99:0/4254121585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7610 msgr2=0x5558291f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.785+0000 7f37ffe00700 1 --2- 172.21.15.99:0/4254121585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7610 0x5558291f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.785+0000 7f37ffe00700 1 -- 172.21.15.99:0/4254121585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f8e70 msgr2=0x5558291f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.785+0000 7f37ffe00700 1 --2- 172.21.15.99:0/4254121585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f8e70 0x5558291f9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.785+0000 7f37ffe00700 1 -- 172.21.15.99:0/4254121585 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555828fc6350 con 0x5558291f7f70 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.785+0000 7f3805800700 1 --2- 172.21.15.99:0/4254121585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f8e70 0x5558291f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.785+0000 7f37ffe00700 1 --2- 172.21.15.99:0/4254121585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7f70 0x5558291f88e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0094f0 tx=0x7f37ec014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f097f25d4cf9479 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.786+0000 7f37f2000700 1 -- 172.21.15.99:0/4254121585 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec026630 con 0x5558291f7f70 2024-03-30T00:36:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.786+0000 7f37f2000700 1 -- 172.21.15.99:0/4254121585 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37ec021070 con 0x5558291f7f70 2024-03-30T00:36:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.786+0000 7f37f2000700 1 -- 172.21.15.99:0/4254121585 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec01d670 con 0x5558291f7f70 2024-03-30T00:36:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.787+0000 7f381b334b80 1 -- 172.21.15.99:0/4254121585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7f70 msgr2=0x5558291f88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.787+0000 7f381b334b80 1 --2- 172.21.15.99:0/4254121585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7f70 0x5558291f88e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0094f0 tx=0x7f37ec014000 comp rx=0 tx=0).stop 2024-03-30T00:36:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.787+0000 7f381b334b80 1 -- 172.21.15.99:0/4254121585 shutdown_connections 2024-03-30T00:36:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.787+0000 7f381b334b80 1 --2- 172.21.15.99:0/4254121585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f8e70 0x5558291f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.787+0000 7f381b334b80 1 --2- 172.21.15.99:0/4254121585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7610 0x5558291f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.787+0000 7f381b334b80 1 --2- 172.21.15.99:0/4254121585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7f70 0x5558291f88e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.787+0000 7f381b334b80 1 -- 172.21.15.99:0/4254121585 >> 172.21.15.99:0/4254121585 conn(0x555829152640 msgr2=0x5558291ee050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.787+0000 7f381b334b80 1 -- 172.21.15.99:0/4254121585 shutdown_connections 2024-03-30T00:36:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.787+0000 7f381b334b80 1 -- 172.21.15.99:0/4254121585 wait complete. 2024-03-30T00:36:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.788+0000 7f381b334b80 1 -- start start 2024-03-30T00:36:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.788+0000 7f381b334b80 1 Processor -- start 2024-03-30T00:36:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.789+0000 7f381b334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7610 0x5558292016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.789+0000 7f381b334b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829201c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.789+0000 7f381b334b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 0x555829208b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.789+0000 7f381b334b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558290e0d20 con 0x5558291f7610 2024-03-30T00:36:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.789+0000 7f381b334b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558290e09c0 con 0x5558291f7f70 2024-03-30T00:36:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.789+0000 7f381b334b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555828fc8640 con 0x5558291f8e70 2024-03-30T00:36:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.789+0000 7f381b334b80 10 client.0 _ll_register_callbacks cb 0x5558290ea410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.789+0000 7f37ffe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829201c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.789+0000 7f37ffe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829201c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:59344/0 (socket says 172.21.15.99:59344) 2024-03-30T00:36:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.789+0000 7f37ffe00700 1 -- 172.21.15.99:0/3867801185 learned_addr learned my addr 172.21.15.99:0/3867801185 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f3804e00700 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7610 0x5558292016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f3805800700 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 0x555829208b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f37ffe00700 1 -- 172.21.15.99:0/3867801185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 msgr2=0x555829208b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f37ffe00700 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 0x555829208b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f37ffe00700 1 -- 172.21.15.99:0/3867801185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7610 msgr2=0x5558292016d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f37ffe00700 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7610 0x5558292016d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f37ffe00700 1 -- 172.21.15.99:0/3867801185 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec017070 con 0x5558291f7f70 2024-03-30T00:36:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f3805800700 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 0x555829208b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f3804e00700 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7610 0x5558292016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f37ffe00700 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829201c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f37ec011ab0 tx=0x7f37ec01cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd64dc5c7bee6b69 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f37e6a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:36:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.790+0000 7f37e6a00700 1 -- 172.21.15.99:0/3867801185 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec01c7a0 con 0x5558291f7f70 2024-03-30T00:36:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.791+0000 7f37e6a00700 1 -- 172.21.15.99:0/3867801185 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f37ec01c900 con 0x5558291f7f70 2024-03-30T00:36:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.791+0000 7f381b334b80 1 -- 172.21.15.99:0/3867801185 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555829105320 con 0x5558291f7f70 2024-03-30T00:36:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.791+0000 7f37e6a00700 1 -- 172.21.15.99:0/3867801185 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37ec005040 con 0x5558291f7f70 2024-03-30T00:36:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.791+0000 7f381b334b80 10 client.25670 fetch_fsmap learned FSMap version 25 2024-03-30T00:36:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.791+0000 7f381b334b80 1 -- 172.21.15.99:0/3867801185 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555828f72b00 con 0x5558291f7f70 2024-03-30T00:36:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.791+0000 7f37e6a00700 1 -- 172.21.15.99:0/3867801185 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555829105320 con 0x5558291f7f70 2024-03-30T00:36:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.792+0000 7f37e6a00700 1 -- 172.21.15.99:0/3867801185 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f37ec003c70 con 0x5558291f7f70 2024-03-30T00:36:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.792+0000 7f381b334b80 10 client.25670 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:36:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.792+0000 7f381b334b80 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.792+0000 7f381b334b80 20 client.25670 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.792+0000 7f381b334b80 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55582920d260 0x55582920f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.792+0000 7f381b334b80 4 client.25670 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.792+0000 7f381b334b80 1 -- 172.21.15.99:0/3867801185 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55582913d6a0 con 0x55582920d260 2024-03-30T00:36:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.793+0000 7f3804e00700 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55582920d260 0x55582920f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.794+0000 7f37e6a00700 10 client.25670 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.793+0000 7f3804e00700 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55582920d260 0x55582920f680 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4660e48ded348ee server_cookie=b721009d52dd8f10 in_seq=0 out_seq=0 2024-03-30T00:36:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.794+0000 7f37e6a00700 1 -- 172.21.15.99:0/3867801185 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55582913d6a0 con 0x55582920d260 2024-03-30T00:36:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.794+0000 7f37e6a00700 10 client.25670 handle_command_reply: tid=0 2024-03-30T00:36:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.794+0000 7f381b334b80 1 client.25670 shutdown 2024-03-30T00:36:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f381b334b80 1 -- 172.21.15.99:0/3867801185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 msgr2=0x555829201c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f381b334b80 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829201c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f37ec011ab0 tx=0x7f37ec01cca0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f381b334b80 1 -- 172.21.15.99:0/3867801185 shutdown_connections 2024-03-30T00:36:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f381b334b80 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55582920d260 0x55582920f680 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f381b334b80 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 0x555829208b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f381b334b80 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7610 0x5558292016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f37e6a00700 0 client.25670 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f381b334b80 1 --2- 172.21.15.99:0/3867801185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829201c10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f381b334b80 1 -- 172.21.15.99:0/3867801185 >> 172.21.15.99:0/3867801185 conn(0x555829152640 msgr2=0x5558291f1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f381b334b80 1 -- 172.21.15.99:0/3867801185 shutdown_connections 2024-03-30T00:36:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f381b334b80 1 -- 172.21.15.99:0/3867801185 wait complete. 2024-03-30T00:36:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.795+0000 7f381b334b80 20 client.25670 trim_cache size 0 max 16384 2024-03-30T00:36:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.821+0000 7f381b334b80 1 Processor -- start 2024-03-30T00:36:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.821+0000 7f381b334b80 1 -- start start 2024-03-30T00:36:46.828 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.821+0000 7f381b334b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7610 0x555829200c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f381b334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7f70 0x5558291ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f381b334b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f8e70 0x555829208b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f381b334b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558291b9780 con 0x5558291f7f70 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f381b334b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558291fc960 con 0x5558291f8e70 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f381b334b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555829209ce0 con 0x5558291f7610 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f3804e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7610 0x555829200c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f3804e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7610 0x555829200c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:58022/0 (socket says 172.21.15.99:58022) 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f3804e00700 1 -- 172.21.15.99:0/2768147358 learned_addr learned my addr 172.21.15.99:0/2768147358 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f37ffe00700 1 --2- 172.21.15.99:0/2768147358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7f70 0x5558291ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f3804e00700 1 -- 172.21.15.99:0/2768147358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f8e70 msgr2=0x555829208b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f3804e00700 1 --2- 172.21.15.99:0/2768147358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f8e70 0x555829208b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.829 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f3804e00700 1 -- 172.21.15.99:0/2768147358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7f70 msgr2=0x5558291ff580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f3804e00700 1 --2- 172.21.15.99:0/2768147358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7f70 0x5558291ff580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f3804e00700 1 -- 172.21.15.99:0/2768147358 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55582910ae40 con 0x5558291f7610 2024-03-30T00:36:46.830 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.822+0000 7f37ffe00700 1 --2- 172.21.15.99:0/2768147358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7f70 0x5558291ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.823+0000 7f3804e00700 1 --2- 172.21.15.99:0/2768147358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7610 0x555829200c50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f37f40095c0 tx=0x7f37f40a1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9446ff29315456b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.823+0000 7f37f2a00700 1 -- 172.21.15.99:0/2768147358 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40a34f0 con 0x5558291f7610 2024-03-30T00:36:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.823+0000 7f37f2a00700 1 -- 172.21.15.99:0/2768147358 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37f40cb830 con 0x5558291f7610 2024-03-30T00:36:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.823+0000 7f37f2a00700 1 -- 172.21.15.99:0/2768147358 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f409b360 con 0x5558291f7610 2024-03-30T00:36:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.824+0000 7f381b334b80 1 -- 172.21.15.99:0/2768147358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7610 msgr2=0x555829200c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.824+0000 7f381b334b80 1 --2- 172.21.15.99:0/2768147358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7610 0x555829200c50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f37f40095c0 tx=0x7f37f40a1040 comp rx=0 tx=0).stop 2024-03-30T00:36:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.824+0000 7f381b334b80 1 -- 172.21.15.99:0/2768147358 shutdown_connections 2024-03-30T00:36:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.824+0000 7f381b334b80 1 --2- 172.21.15.99:0/2768147358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f8e70 0x555829208b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.824+0000 7f381b334b80 1 --2- 172.21.15.99:0/2768147358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f7610 0x555829200c50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.824+0000 7f381b334b80 1 --2- 172.21.15.99:0/2768147358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7f70 0x5558291ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.824+0000 7f381b334b80 1 -- 172.21.15.99:0/2768147358 >> 172.21.15.99:0/2768147358 conn(0x555829152640 msgr2=0x555829213fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.824+0000 7f381b334b80 1 -- 172.21.15.99:0/2768147358 shutdown_connections 2024-03-30T00:36:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.824+0000 7f381b334b80 1 -- 172.21.15.99:0/2768147358 wait complete. 2024-03-30T00:36:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.825+0000 7f381b334b80 1 -- start start 2024-03-30T00:36:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.825+0000 7f381b334b80 1 Processor -- start 2024-03-30T00:36:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.826+0000 7f381b334b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7610 0x5558292065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.826+0000 7f381b334b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829206b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.826+0000 7f381b334b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 0x555829207aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.826+0000 7f37ffe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829206b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.826+0000 7f3804e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7610 0x5558292065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.826+0000 7f37ffe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829206b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:59364/0 (socket says 172.21.15.99:59364) 2024-03-30T00:36:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.826+0000 7f37ffe00700 1 -- 172.21.15.99:0/1747427521 learned_addr learned my addr 172.21.15.99:0/1747427521 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.827+0000 7f381b334b80 1 -- 172.21.15.99:0/1747427521 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555829207fe0 con 0x5558291f7610 2024-03-30T00:36:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.827+0000 7f381b334b80 1 -- 172.21.15.99:0/1747427521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555829208150 con 0x5558291f7f70 2024-03-30T00:36:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.827+0000 7f381b334b80 1 -- 172.21.15.99:0/1747427521 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555829208310 con 0x5558291f8e70 2024-03-30T00:36:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.827+0000 7f381b334b80 10 client.0 _ll_register_callbacks cb 0x5558290ea410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.827+0000 7f3805800700 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 0x555829207aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.827+0000 7f37ffe00700 1 -- 172.21.15.99:0/1747427521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 msgr2=0x555829207aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.827+0000 7f37ffe00700 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 0x555829207aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.827+0000 7f37ffe00700 1 -- 172.21.15.99:0/1747427521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7610 msgr2=0x5558292065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.827+0000 7f37ffe00700 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7610 0x5558292065f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.827+0000 7f37ffe00700 1 -- 172.21.15.99:0/1747427521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380019ee40 con 0x5558291f7f70 2024-03-30T00:36:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.827+0000 7f3805800700 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 0x555829207aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.828+0000 7f37ffe00700 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829206b30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00e610 tx=0x7f37ec0265d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbb94325d0ed4cdb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.828+0000 7f37e6a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:36:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.828+0000 7f37e6a00700 1 -- 172.21.15.99:0/1747427521 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec01c410 con 0x5558291f7f70 2024-03-30T00:36:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.828+0000 7f37e6a00700 1 -- 172.21.15.99:0/1747427521 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f37ec026720 con 0x5558291f7f70 2024-03-30T00:36:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.828+0000 7f381b334b80 1 -- 172.21.15.99:0/1747427521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37ec021070 con 0x5558291f7f70 2024-03-30T00:36:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.828+0000 7f37e6a00700 1 -- 172.21.15.99:0/1747427521 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37ec026960 con 0x5558291f7f70 2024-03-30T00:36:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.829+0000 7f381b334b80 10 client.25673 fetch_fsmap learned FSMap version 25 2024-03-30T00:36:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.829+0000 7f381b334b80 1 -- 172.21.15.99:0/1747427521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555828fcac70 con 0x5558291f7f70 2024-03-30T00:36:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.829+0000 7f37e6a00700 1 -- 172.21.15.99:0/1747427521 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37ec021070 con 0x5558291f7f70 2024-03-30T00:36:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.829+0000 7f37e6a00700 1 -- 172.21.15.99:0/1747427521 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f37ec003680 con 0x5558291f7f70 2024-03-30T00:36:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.829+0000 7f381b334b80 10 client.25673 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:36:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.829+0000 7f381b334b80 10 client.25673 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.829+0000 7f381b334b80 20 client.25673 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.829+0000 7f381b334b80 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x555829261930 0x555829263d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.829+0000 7f381b334b80 4 client.25673 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:36:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.829+0000 7f381b334b80 1 -- 172.21.15.99:0/1747427521 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555828f93c90 con 0x555829261930 2024-03-30T00:36:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.830+0000 7f3804e00700 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x555829261930 0x555829263d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.831+0000 7f3804e00700 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x555829261930 0x555829263d50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b34fb7e3a0434b1 server_cookie=753813d6145dfe57 in_seq=0 out_seq=0 2024-03-30T00:36:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.831+0000 7f37e6a00700 10 client.25673 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.842+0000 7f37e6a00700 1 -- 172.21.15.99:0/1747427521 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555828f93c90 con 0x555829261930 2024-03-30T00:36:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.842+0000 7f37e6a00700 10 client.25673 handle_command_reply: tid=0 2024-03-30T00:36:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.842+0000 7f381b334b80 1 client.25673 shutdown 2024-03-30T00:36:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.842+0000 7f381b334b80 1 -- 172.21.15.99:0/1747427521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 msgr2=0x555829206b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.842+0000 7f381b334b80 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829206b30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00e610 tx=0x7f37ec0265d0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.843+0000 7f381b334b80 1 -- 172.21.15.99:0/1747427521 shutdown_connections 2024-03-30T00:36:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.843+0000 7f381b334b80 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x555829261930 0x555829263d50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.843+0000 7f381b334b80 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5558291f8e70 0x555829207aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.843+0000 7f381b334b80 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5558291f7610 0x5558292065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.843+0000 7f381b334b80 1 --2- 172.21.15.99:0/1747427521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558291f7f70 0x555829206b30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.843+0000 7f37e6a00700 0 client.25673 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.843+0000 7f381b334b80 1 -- 172.21.15.99:0/1747427521 >> 172.21.15.99:0/1747427521 conn(0x555829152640 msgr2=0x5558291ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.843+0000 7f381b334b80 1 -- 172.21.15.99:0/1747427521 shutdown_connections 2024-03-30T00:36:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.843+0000 7f381b334b80 1 -- 172.21.15.99:0/1747427521 wait complete. 2024-03-30T00:36:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.843+0000 7f381b334b80 20 client.25673 trim_cache size 0 max 16384 2024-03-30T00:36:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:36:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:36:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:36:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 -- 172.21.15.99:0/3121539292 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37e80a11a0 msgr2=0x7f37e80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37e80a11a0 0x7f37e80a3660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009df0 tx=0x7f37ec009260 comp rx=0 tx=0).stop 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 -- 172.21.15.99:0/3121539292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3800101780 msgr2=0x7f3800196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3800101780 0x7f3800196b80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f37f4005740 tx=0x7f37f4005770 comp rx=0 tx=0).stop 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 -- 172.21.15.99:0/3121539292 shutdown_connections 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f37e80a11a0 0x7f37e80a3660 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38001069e0 0x7f380019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3800101780 0x7f3800196b80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 --2- 172.21.15.99:0/3121539292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38001040b0 0x7f38001970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 -- 172.21.15.99:0/3121539292 >> 172.21.15.99:0/3121539292 conn(0x7f38000fb170 msgr2=0x7f3800103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 -- 172.21.15.99:0/3121539292 shutdown_connections 2024-03-30T00:36:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:36:46.844+0000 7f3807800700 1 -- 172.21.15.99:0/3121539292 wait complete. 2024-03-30T00:36:48.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:48 smithi161 ceph-mon[42221]: pgmap v366: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 2.75k op/s 2024-03-30T00:36:48.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:48 smithi099 ceph-mon[38079]: pgmap v366: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 2.75k op/s 2024-03-30T00:36:48.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:48 smithi142 ceph-mon[41779]: pgmap v366: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 2.75k op/s 2024-03-30T00:36:50.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:50 smithi161 ceph-mon[42221]: pgmap v367: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 3.02k op/s 2024-03-30T00:36:50.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:50 smithi099 ceph-mon[38079]: pgmap v367: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 3.02k op/s 2024-03-30T00:36:50.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:50 smithi142 ceph-mon[41779]: pgmap v367: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 14 MiB/s wr, 3.02k op/s 2024-03-30T00:36:52.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:52 smithi161 ceph-mon[42221]: pgmap v368: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 2.77k op/s 2024-03-30T00:36:52.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:52 smithi099 ceph-mon[38079]: pgmap v368: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 2.77k op/s 2024-03-30T00:36:52.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:52 smithi142 ceph-mon[41779]: pgmap v368: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 MiB/s wr, 2.77k op/s 2024-03-30T00:36:54.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:54 smithi161 ceph-mon[42221]: pgmap v369: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 2.78k op/s 2024-03-30T00:36:54.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:54 smithi099 ceph-mon[38079]: pgmap v369: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 2.78k op/s 2024-03-30T00:36:54.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:54 smithi142 ceph-mon[41779]: pgmap v369: 97 pgs: 97 active+clean; 258 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 2.78k op/s 2024-03-30T00:36:56.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:56 smithi161 ceph-mon[42221]: pgmap v370: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 13 MiB/s wr, 2.96k op/s 2024-03-30T00:36:56.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:56 smithi099 ceph-mon[38079]: pgmap v370: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 13 MiB/s wr, 2.96k op/s 2024-03-30T00:36:56.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:56 smithi142 ceph-mon[41779]: pgmap v370: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 13 MiB/s wr, 2.96k op/s 2024-03-30T00:36:58.434 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T00:36:58.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:36:58 smithi161 ceph-mon[42221]: pgmap v371: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 2.73k op/s 2024-03-30T00:36:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.714+0000 7f08a4a00700 1 -- 172.21.15.99:0/3359835950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a0108c30 msgr2=0x7f08a010b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.714+0000 7f08a4a00700 1 --2- 172.21.15.99:0/3359835950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a0108c30 0x7f08a010b0c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f089800b0a0 tx=0x7f089800b3b0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.714+0000 7f08a4a00700 1 -- 172.21.15.99:0/3359835950 shutdown_connections 2024-03-30T00:36:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.714+0000 7f08a4a00700 1 --2- 172.21.15.99:0/3359835950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a0108c30 0x7f08a010b0c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.714+0000 7f08a4a00700 1 --2- 172.21.15.99:0/3359835950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a0074db0 0x7f08a0073200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.714+0000 7f08a4a00700 1 --2- 172.21.15.99:0/3359835950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a00737d0 0x7f08a0073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.714+0000 7f08a4a00700 1 -- 172.21.15.99:0/3359835950 >> 172.21.15.99:0/3359835950 conn(0x7f08a00fc2f0 msgr2=0x7f08a00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.714+0000 7f08a4a00700 1 -- 172.21.15.99:0/3359835950 shutdown_connections 2024-03-30T00:36:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.714+0000 7f08a4a00700 1 -- 172.21.15.99:0/3359835950 wait complete. 2024-03-30T00:36:58.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.715+0000 7f08a4a00700 1 Processor -- start 2024-03-30T00:36:58.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.715+0000 7f08a4a00700 1 -- start start 2024-03-30T00:36:58.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.716+0000 7f08a4a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a00737d0 0x7f08a0198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.716+0000 7f08a4a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a0074db0 0x7f08a0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.716+0000 7f08a4a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a0108c30 0x7f08a01a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.716+0000 7f08a4a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a01a0a20 con 0x7f08a00737d0 2024-03-30T00:36:58.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.716+0000 7f08a4a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a01a0b60 con 0x7f08a0074db0 2024-03-30T00:36:58.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.716+0000 7f08a4a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a01a0cf0 con 0x7f08a0108c30 2024-03-30T00:36:58.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.716+0000 7f089de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a00737d0 0x7f08a0198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.716+0000 7f089de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a00737d0 0x7f08a0198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43932/0 (socket says 172.21.15.99:43932) 2024-03-30T00:36:58.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.716+0000 7f089de00700 1 -- 172.21.15.99:0/2273475521 learned_addr learned my addr 172.21.15.99:0/2273475521 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:58.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.716+0000 7f089d400700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a0074db0 0x7f08a0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.716+0000 7f089e800700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a0108c30 0x7f08a01a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.717+0000 7f089de00700 1 -- 172.21.15.99:0/2273475521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a0108c30 msgr2=0x7f08a01a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:58.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.717+0000 7f089de00700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a0108c30 0x7f08a01a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.717+0000 7f089de00700 1 -- 172.21.15.99:0/2273475521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a0074db0 msgr2=0x7f08a0199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:58.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.717+0000 7f089de00700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a0074db0 0x7f08a0199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.717+0000 7f089de00700 1 -- 172.21.15.99:0/2273475521 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0898009d00 con 0x7f08a00737d0 2024-03-30T00:36:58.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.717+0000 7f089e800700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a0108c30 0x7f08a01a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:58.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.717+0000 7f089de00700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a00737d0 0x7f08a0198ef0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f089000e9a0 tx=0x7f089000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:58.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.718+0000 7f0896a00700 1 -- 172.21.15.99:0/2273475521 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890009790 con 0x7f08a00737d0 2024-03-30T00:36:58.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.718+0000 7f0896a00700 1 -- 172.21.15.99:0/2273475521 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f089000cbd0 con 0x7f08a00737d0 2024-03-30T00:36:58.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.718+0000 7f0896a00700 1 -- 172.21.15.99:0/2273475521 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890018680 con 0x7f08a00737d0 2024-03-30T00:36:58.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.718+0000 7f08a4a00700 1 -- 172.21.15.99:0/2273475521 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a01a0fd0 con 0x7f08a00737d0 2024-03-30T00:36:58.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.718+0000 7f08a4a00700 1 -- 172.21.15.99:0/2273475521 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a01a13f0 con 0x7f08a00737d0 2024-03-30T00:36:58.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.723+0000 7f0896a00700 1 -- 172.21.15.99:0/2273475521 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f089000cd40 con 0x7f08a00737d0 2024-03-30T00:36:58.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.724+0000 7f08b84f0b80 1 Processor -- start 2024-03-30T00:36:58.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.724+0000 7f08b84f0b80 1 -- start start 2024-03-30T00:36:58.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.724+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3974f7d0 0x562b3974fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.724+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750130 0x562b397509f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.724+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750fd0 0x562b39751450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.724+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b396a7990 con 0x562b3974f7d0 2024-03-30T00:36:58.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.724+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b396a5350 con 0x562b39750130 2024-03-30T00:36:58.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.724+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b396a0c40 con 0x562b39750fd0 2024-03-30T00:36:58.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f089de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750130 0x562b397509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f0896a00700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08840a11a0 0x7f08840a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f089de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750130 0x562b397509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48586/0 (socket says 172.21.15.99:48586) 2024-03-30T00:36:58.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f089de00700 1 -- 172.21.15.99:0/3076118683 learned_addr learned my addr 172.21.15.99:0/3076118683 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f089e800700 1 --2- 172.21.15.99:0/3076118683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3974f7d0 0x562b3974fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f089d400700 1 --2- 172.21.15.99:0/3076118683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750fd0 0x562b39751450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f0896a00700 1 -- 172.21.15.99:0/2273475521 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f0890014070 con 0x7f08a00737d0 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f089de00700 1 -- 172.21.15.99:0/3076118683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750fd0 msgr2=0x562b39751450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f089de00700 1 --2- 172.21.15.99:0/3076118683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750fd0 0x562b39751450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f089de00700 1 -- 172.21.15.99:0/3076118683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3974f7d0 msgr2=0x562b3974fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f089de00700 1 --2- 172.21.15.99:0/3076118683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3974f7d0 0x562b3974fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f089de00700 1 -- 172.21.15.99:0/3076118683 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b395223b0 con 0x562b39750130 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.725+0000 7f089e800700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08840a11a0 0x7f08840a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.726+0000 7f089de00700 1 --2- 172.21.15.99:0/3076118683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750130 0x562b397509f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08900ca590 tx=0x7f0890010960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b524a8d21f0637e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.726+0000 7f089e800700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08840a11a0 0x7f08840a3660 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0898009fd0 tx=0x7f089800b3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.727+0000 7f088b000700 1 -- 172.21.15.99:0/3076118683 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890099840 con 0x562b39750130 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.727+0000 7f088b000700 1 -- 172.21.15.99:0/3076118683 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f089009a5e0 con 0x562b39750130 2024-03-30T00:36:58.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.727+0000 7f088b000700 1 -- 172.21.15.99:0/3076118683 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0890099630 con 0x562b39750130 2024-03-30T00:36:58.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.728+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3076118683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750130 msgr2=0x562b397509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:58.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.728+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3076118683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750130 0x562b397509f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08900ca590 tx=0x7f0890010960 comp rx=0 tx=0).stop 2024-03-30T00:36:58.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.728+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3076118683 shutdown_connections 2024-03-30T00:36:58.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.728+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3076118683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750fd0 0x562b39751450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.728+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3076118683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3974f7d0 0x562b3974fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.728+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3076118683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750130 0x562b397509f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.728+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3076118683 >> 172.21.15.99:0/3076118683 conn(0x562b396ae6a0 msgr2=0x562b3974a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:58.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.728+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3076118683 shutdown_connections 2024-03-30T00:36:58.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.728+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3076118683 wait complete. 2024-03-30T00:36:58.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.729+0000 7f08b84f0b80 1 -- start start 2024-03-30T00:36:58.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.729+0000 7f08b84f0b80 1 Processor -- start 2024-03-30T00:36:58.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3974f7d0 0x562b39759620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 0x562b39759b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:36:58 smithi099 ceph-mon[38079]: pgmap v371: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 2.73k op/s 2024-03-30T00:36:58.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750fd0 0x562b39760ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3963cd80 con 0x562b3974f7d0 2024-03-30T00:36:58.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3963ca20 con 0x562b39750fd0 2024-03-30T00:36:58.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b395246a0 con 0x562b39750130 2024-03-30T00:36:58.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f08b84f0b80 10 client.0 _ll_register_callbacks cb 0x562b39646470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:58.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f089e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750fd0 0x562b39760ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f089e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750fd0 0x562b39760ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48598/0 (socket says 172.21.15.99:48598) 2024-03-30T00:36:58.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f089e800700 1 -- 172.21.15.99:0/3341387914 learned_addr learned my addr 172.21.15.99:0/3341387914 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f089d400700 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 0x562b39759b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.730+0000 7f089de00700 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3974f7d0 0x562b39759620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.731+0000 7f089e800700 1 -- 172.21.15.99:0/3341387914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 msgr2=0x562b39759b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.731+0000 7f089e800700 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 0x562b39759b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.731+0000 7f089e800700 1 -- 172.21.15.99:0/3341387914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3974f7d0 msgr2=0x562b39759620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.731+0000 7f089e800700 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3974f7d0 0x562b39759620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.731+0000 7f089e800700 1 -- 172.21.15.99:0/3341387914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a01a13f0 con 0x562b39750fd0 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.731+0000 7f089de00700 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3974f7d0 0x562b39759620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.731+0000 7f089e800700 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750fd0 0x562b39760ba0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0898015000 tx=0x7f08980099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90b2ddb1db288b6c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.731+0000 7f087f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.731+0000 7f087f400700 1 -- 172.21.15.99:0/3341387914 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089800e040 con 0x562b39750fd0 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.731+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3341387914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b39661380 con 0x562b39750fd0 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.731+0000 7f087f400700 1 -- 172.21.15.99:0/3341387914 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f08980050c0 con 0x562b39750fd0 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.732+0000 7f087f400700 1 -- 172.21.15.99:0/3341387914 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0898005280 con 0x562b39750fd0 2024-03-30T00:36:58.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.732+0000 7f08b84f0b80 10 client.25685 fetch_fsmap learned FSMap version 25 2024-03-30T00:36:58.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.732+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3341387914 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b394ceb60 con 0x562b39750fd0 2024-03-30T00:36:58.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.732+0000 7f087f400700 1 -- 172.21.15.99:0/3341387914 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562b39661380 con 0x562b39750fd0 2024-03-30T00:36:58.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.732+0000 7f087f400700 1 -- 172.21.15.99:0/3341387914 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f089801e990 con 0x562b39750fd0 2024-03-30T00:36:58.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.732+0000 7f08b84f0b80 10 client.25685 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:36:58.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.732+0000 7f08b84f0b80 10 client.25685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:58.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.732+0000 7f08b84f0b80 20 client.25685 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:58.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.732+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562b39765420 0x562b39767840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.732+0000 7f08b84f0b80 4 client.25685 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:58.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.732+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3341387914 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b39699700 con 0x562b39765420 2024-03-30T00:36:58.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.733+0000 7f089de00700 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562b39765420 0x562b39767840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.734+0000 7f087f400700 10 client.25685 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:58.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.734+0000 7f089de00700 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562b39765420 0x562b39767840 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af167c6a1e2cf56d server_cookie=e98da97a00b02cf8 in_seq=0 out_seq=0 2024-03-30T00:36:58.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f087f400700 1 -- 172.21.15.99:0/3341387914 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562b39699700 con 0x562b39765420 2024-03-30T00:36:58.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f087f400700 10 client.25685 handle_command_reply: tid=0 2024-03-30T00:36:58.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f08b84f0b80 1 client.25685 shutdown 2024-03-30T00:36:58.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3341387914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750fd0 msgr2=0x562b39760ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:58.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750fd0 0x562b39760ba0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0898015000 tx=0x7f08980099e0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3341387914 shutdown_connections 2024-03-30T00:36:58.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562b39765420 0x562b39767840 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b39750fd0 0x562b39760ba0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3974f7d0 0x562b39759620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f087f400700 0 client.25685 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:58.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3341387914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 0x562b39759b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3341387914 >> 172.21.15.99:0/3341387914 conn(0x562b396ae6a0 msgr2=0x562b3974e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:58.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.735+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3341387914 shutdown_connections 2024-03-30T00:36:58.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.736+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3341387914 wait complete. 2024-03-30T00:36:58.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.736+0000 7f08b84f0b80 20 client.25685 trim_cache size 0 max 16384 2024-03-30T00:36:58.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.762+0000 7f08b84f0b80 1 Processor -- start 2024-03-30T00:36:58.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.762+0000 7f08b84f0b80 1 -- start start 2024-03-30T00:36:58.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.762+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 0x562b39749f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.762+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b39750130 0x562b39760b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.762+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750fd0 0x562b397435a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.762+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b39762900 con 0x562b39750130 2024-03-30T00:36:58.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.763+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3974b660 con 0x562b3974f7d0 2024-03-30T00:36:58.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.763+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3974b8a0 con 0x562b39750fd0 2024-03-30T00:36:58.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.763+0000 7f089de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 0x562b39749f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.763+0000 7f089d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b39750130 0x562b39760b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.763+0000 7f089de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 0x562b39749f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48604/0 (socket says 172.21.15.99:48604) 2024-03-30T00:36:58.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.763+0000 7f089de00700 1 -- 172.21.15.99:0/2152809729 learned_addr learned my addr 172.21.15.99:0/2152809729 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:58.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.763+0000 7f089e800700 1 --2- 172.21.15.99:0/2152809729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750fd0 0x562b397435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.764+0000 7f089de00700 1 -- 172.21.15.99:0/2152809729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750fd0 msgr2=0x562b397435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:58.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.764+0000 7f089de00700 1 --2- 172.21.15.99:0/2152809729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750fd0 0x562b397435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.764+0000 7f089de00700 1 -- 172.21.15.99:0/2152809729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b39750130 msgr2=0x562b39760b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.764+0000 7f089de00700 1 --2- 172.21.15.99:0/2152809729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b39750130 0x562b39760b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.764+0000 7f089de00700 1 -- 172.21.15.99:0/2152809729 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b39666ea0 con 0x562b3974f7d0 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.764+0000 7f089de00700 1 --2- 172.21.15.99:0/2152809729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 0x562b39749f10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0890010b20 tx=0x7f089009a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88712d75dafc9bf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.764+0000 7f088ba00700 1 -- 172.21.15.99:0/2152809729 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08900a23e0 con 0x562b3974f7d0 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.764+0000 7f088ba00700 1 -- 172.21.15.99:0/2152809729 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0890098280 con 0x562b3974f7d0 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.764+0000 7f088ba00700 1 -- 172.21.15.99:0/2152809729 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08900a2110 con 0x562b3974f7d0 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.765+0000 7f08b84f0b80 1 -- 172.21.15.99:0/2152809729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 msgr2=0x562b39749f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.765+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/2152809729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 0x562b39749f10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0890010b20 tx=0x7f089009a750 comp rx=0 tx=0).stop 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.765+0000 7f08b84f0b80 1 -- 172.21.15.99:0/2152809729 shutdown_connections 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.765+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/2152809729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750fd0 0x562b397435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.765+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/2152809729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 0x562b39749f10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.765+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/2152809729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b39750130 0x562b39760b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.765+0000 7f08b84f0b80 1 -- 172.21.15.99:0/2152809729 >> 172.21.15.99:0/2152809729 conn(0x562b396ae6a0 msgr2=0x562b39757550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:58.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.765+0000 7f08b84f0b80 1 -- 172.21.15.99:0/2152809729 shutdown_connections 2024-03-30T00:36:58.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.765+0000 7f08b84f0b80 1 -- 172.21.15.99:0/2152809729 wait complete. 2024-03-30T00:36:58.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.766+0000 7f08b84f0b80 1 -- start start 2024-03-30T00:36:58.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.766+0000 7f08b84f0b80 1 Processor -- start 2024-03-30T00:36:58.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.767+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 0x562b39760700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.767+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 0x562b39760c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.767+0000 7f08b84f0b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b39750fd0 0x562b3975ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.767+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3975f560 con 0x562b39750fd0 2024-03-30T00:36:58.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.767+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3975f6d0 con 0x562b3974f7d0 2024-03-30T00:36:58.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.767+0000 7f08b84f0b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b3975f890 con 0x562b39750130 2024-03-30T00:36:58.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.767+0000 7f08b84f0b80 10 client.0 _ll_register_callbacks cb 0x562b39646470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:58.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.767+0000 7f089d400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 0x562b39760c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.767+0000 7f089d400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 0x562b39760c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:36542/0 (socket says 172.21.15.99:36542) 2024-03-30T00:36:58.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.767+0000 7f089d400700 1 -- 172.21.15.99:0/3374915152 learned_addr learned my addr 172.21.15.99:0/3374915152 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:58.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.767+0000 7f089e800700 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b39750fd0 0x562b3975ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.768+0000 7f089de00700 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 0x562b39760700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.768+0000 7f089d400700 1 -- 172.21.15.99:0/3374915152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 msgr2=0x562b39760700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:58.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.768+0000 7f089d400700 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 0x562b39760700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.768+0000 7f089d400700 1 -- 172.21.15.99:0/3374915152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b39750fd0 msgr2=0x562b3975ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:58.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.768+0000 7f089d400700 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b39750fd0 0x562b3975ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.768+0000 7f089d400700 1 -- 172.21.15.99:0/3374915152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b395223b0 con 0x562b39750130 2024-03-30T00:36:58.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.769+0000 7f089e800700 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b39750fd0 0x562b3975ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:58.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.769+0000 7f089de00700 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 0x562b39760700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:36:58.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.769+0000 7f089d400700 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 0x562b39760c40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f088c012980 tx=0x7f088c012c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b86433209345d1d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:58.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.769+0000 7f087f400700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:36:58.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.770+0000 7f087f400700 1 -- 172.21.15.99:0/3374915152 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c00f9d0 con 0x562b39750130 2024-03-30T00:36:58.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.770+0000 7f087f400700 1 -- 172.21.15.99:0/3374915152 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f088c00fb30 con 0x562b39750130 2024-03-30T00:36:58.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.770+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3374915152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f089009a5e0 con 0x562b39750130 2024-03-30T00:36:58.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.770+0000 7f087f400700 1 -- 172.21.15.99:0/3374915152 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f088c012ed0 con 0x562b39750130 2024-03-30T00:36:58.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.770+0000 7f08b84f0b80 10 client.25702 fetch_fsmap learned FSMap version 25 2024-03-30T00:36:58.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.770+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3374915152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b39526cd0 con 0x562b39750130 2024-03-30T00:36:58.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.771+0000 7f087f400700 1 -- 172.21.15.99:0/3374915152 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f088c03c030 con 0x562b39750130 2024-03-30T00:36:58.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.771+0000 7f087f400700 1 -- 172.21.15.99:0/3374915152 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f088c036060 con 0x562b39750130 2024-03-30T00:36:58.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.771+0000 7f08b84f0b80 10 client.25702 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:36:58.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.771+0000 7f08b84f0b80 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:36:58.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.771+0000 7f08b84f0b80 20 client.25702 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:58.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.771+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562b397ba900 0x562b397bcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:58.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.771+0000 7f08b84f0b80 4 client.25702 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:36:58.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.771+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3374915152 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562b394efcf0 con 0x562b397ba900 2024-03-30T00:36:58.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.772+0000 7f089de00700 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562b397ba900 0x562b397bcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:58.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.773+0000 7f089de00700 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562b397ba900 0x562b397bcd20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67fb6e3fb8441e2 server_cookie=a9efb057468839a6 in_seq=0 out_seq=0 2024-03-30T00:36:58.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.773+0000 7f087f400700 10 client.25702 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:58.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.773+0000 7f087f400700 1 -- 172.21.15.99:0/3374915152 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562b394efcf0 con 0x562b397ba900 2024-03-30T00:36:58.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.773+0000 7f087f400700 10 client.25702 handle_command_reply: tid=0 2024-03-30T00:36:58.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.773+0000 7f08b84f0b80 1 client.25702 shutdown 2024-03-30T00:36:58.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.773+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3374915152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 msgr2=0x562b39760c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:58.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.773+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 0x562b39760c40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f088c012980 tx=0x7f088c012c90 comp rx=0 tx=0).stop 2024-03-30T00:36:58.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.774+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3374915152 shutdown_connections 2024-03-30T00:36:58.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.774+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562b397ba900 0x562b397bcd20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.774+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b39750fd0 0x562b3975ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.774+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b3974f7d0 0x562b39760700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.774+0000 7f08b84f0b80 1 --2- 172.21.15.99:0/3374915152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562b39750130 0x562b39760c40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.774+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3374915152 >> 172.21.15.99:0/3374915152 conn(0x562b396ae6a0 msgr2=0x562b397435a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:58.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.774+0000 7f087f400700 0 client.25702 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:36:58.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.774+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3374915152 shutdown_connections 2024-03-30T00:36:58.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.774+0000 7f08b84f0b80 1 -- 172.21.15.99:0/3374915152 wait complete. 2024-03-30T00:36:58.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.775+0000 7f08b84f0b80 20 client.25702 trim_cache size 0 max 16384 2024-03-30T00:36:58.783 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:36:58.783 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "no active scrubs running", 2024-03-30T00:36:58.783 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": {} 2024-03-30T00:36:58.783 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.775+0000 7f08a4a00700 1 -- 172.21.15.99:0/2273475521 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08840a11a0 msgr2=0x7f08840a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.775+0000 7f08a4a00700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08840a11a0 0x7f08840a3660 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0898009fd0 tx=0x7f089800b3b0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.775+0000 7f08a4a00700 1 -- 172.21.15.99:0/2273475521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a00737d0 msgr2=0x7f08a0198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.775+0000 7f08a4a00700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a00737d0 0x7f08a0198ef0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f089000e9a0 tx=0x7f089000ecb0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.776+0000 7f08a4a00700 1 -- 172.21.15.99:0/2273475521 shutdown_connections 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.776+0000 7f08a4a00700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08a0108c30 0x7f08a01a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.776+0000 7f08a4a00700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08a00737d0 0x7f08a0198ef0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.776+0000 7f08a4a00700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f08840a11a0 0x7f08840a3660 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.776+0000 7f08a4a00700 1 --2- 172.21.15.99:0/2273475521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a0074db0 0x7f08a0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.776+0000 7f08a4a00700 1 -- 172.21.15.99:0/2273475521 >> 172.21.15.99:0/2273475521 conn(0x7f08a00fc2f0 msgr2=0x7f08a0104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.776+0000 7f08a4a00700 1 -- 172.21.15.99:0/2273475521 shutdown_connections 2024-03-30T00:36:58.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:58.776+0000 7f08a4a00700 1 -- 172.21.15.99:0/2273475521 wait complete. 2024-03-30T00:36:58.812 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:36:58.812 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:36:58.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:36:58 smithi142 ceph-mon[41779]: pgmap v371: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 2.73k op/s 2024-03-30T00:36:59.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.070+0000 7f7e6d600700 1 -- 172.21.15.99:0/2389376453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e681017a0 msgr2=0x7f7e68103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.070+0000 7f7e6d600700 1 --2- 172.21.15.99:0/2389376453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e681017a0 0x7f7e68103b90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009990 tx=0x7f7e58009ca0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.070+0000 7f7e6d600700 1 -- 172.21.15.99:0/2389376453 shutdown_connections 2024-03-30T00:36:59.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.070+0000 7f7e6d600700 1 --2- 172.21.15.99:0/2389376453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e68106a00 0x7f7e68108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.070+0000 7f7e6d600700 1 --2- 172.21.15.99:0/2389376453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e681017a0 0x7f7e68103b90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.070+0000 7f7e6d600700 1 --2- 172.21.15.99:0/2389376453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e681040d0 0x7f7e681064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.070+0000 7f7e6d600700 1 -- 172.21.15.99:0/2389376453 >> 172.21.15.99:0/2389376453 conn(0x7f7e680fb0d0 msgr2=0x7f7e680fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:59.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.070+0000 7f7e6d600700 1 -- 172.21.15.99:0/2389376453 shutdown_connections 2024-03-30T00:36:59.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.070+0000 7f7e6d600700 1 -- 172.21.15.99:0/2389376453 wait complete. 2024-03-30T00:36:59.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.071+0000 7f7e6d600700 1 Processor -- start 2024-03-30T00:36:59.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.071+0000 7f7e6d600700 1 -- start start 2024-03-30T00:36:59.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.072+0000 7f7e6d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e681017a0 0x7f7e681010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.072+0000 7f7e6d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e681040d0 0x7f7e680ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.072+0000 7f7e6d600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e68106a00 0x7f7e680ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.072+0000 7f7e6d600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6819a9b0 con 0x7f7e681017a0 2024-03-30T00:36:59.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.072+0000 7f7e6d600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681001d0 con 0x7f7e681040d0 2024-03-30T00:36:59.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.072+0000 7f7e6d600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68100390 con 0x7f7e68106a00 2024-03-30T00:36:59.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.072+0000 7f7e66a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e681017a0 0x7f7e681010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.072+0000 7f7e67400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e68106a00 0x7f7e680ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.072+0000 7f7e66a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e681017a0 0x7f7e681010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43998/0 (socket says 172.21.15.99:43998) 2024-03-30T00:36:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.072+0000 7f7e66a00700 1 -- 172.21.15.99:0/698046900 learned_addr learned my addr 172.21.15.99:0/698046900 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:59.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.072+0000 7f7e66000700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e681040d0 0x7f7e680ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.073+0000 7f7e67400700 1 -- 172.21.15.99:0/698046900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e681040d0 msgr2=0x7f7e680ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.073+0000 7f7e67400700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e681040d0 0x7f7e680ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.073+0000 7f7e67400700 1 -- 172.21.15.99:0/698046900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e681017a0 msgr2=0x7f7e681010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.073+0000 7f7e67400700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e681017a0 0x7f7e681010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.073+0000 7f7e67400700 1 -- 172.21.15.99:0/698046900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e580095f0 con 0x7f7e68106a00 2024-03-30T00:36:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.073+0000 7f7e66a00700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e681017a0 0x7f7e681010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.073+0000 7f7e67400700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e68106a00 0x7f7e680ffc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7e60007f10 tx=0x7f7e6000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.073+0000 7f7e5f400700 1 -- 172.21.15.99:0/698046900 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6000ddd0 con 0x7f7e68106a00 2024-03-30T00:36:59.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.073+0000 7f7e6d600700 1 -- 172.21.15.99:0/698046900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e681a5360 con 0x7f7e68106a00 2024-03-30T00:36:59.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.074+0000 7f7e6d600700 1 -- 172.21.15.99:0/698046900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e681a5780 con 0x7f7e68106a00 2024-03-30T00:36:59.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.074+0000 7f7e5f400700 1 -- 172.21.15.99:0/698046900 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e6000f040 con 0x7f7e68106a00 2024-03-30T00:36:59.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.074+0000 7f7e5f400700 1 -- 172.21.15.99:0/698046900 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e600059c0 con 0x7f7e68106a00 2024-03-30T00:36:59.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.075+0000 7f7e6d600700 1 -- 172.21.15.99:0/698046900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e48005320 con 0x7f7e68106a00 2024-03-30T00:36:59.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.078+0000 7f7e5f400700 1 -- 172.21.15.99:0/698046900 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7e60005020 con 0x7f7e68106a00 2024-03-30T00:36:59.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.079+0000 7f7e5f400700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e4c0a11a0 0x7f7e4c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.079+0000 7f7e5f400700 1 -- 172.21.15.99:0/698046900 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f7e600cc070 con 0x7f7e68106a00 2024-03-30T00:36:59.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.079+0000 7f7e66a00700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e4c0a11a0 0x7f7e4c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.080+0000 7f7e66a00700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e4c0a11a0 0x7f7e4c0a3660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800b5c0 tx=0x7f7e58003720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:59.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.085+0000 7f7e5f400700 1 -- 172.21.15.99:0/698046900 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7e60095780 con 0x7f7e68106a00 2024-03-30T00:36:59.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.499+0000 7f7e6d600700 1 -- 172.21.15.99:0/698046900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e48005b50 con 0x7f7e68106a00 2024-03-30T00:36:59.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.500+0000 7f7e5f400700 1 -- 172.21.15.99:0/698046900 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10861 (secure 0 0 0) 0x7f7e60098da0 con 0x7f7e68106a00 2024-03-30T00:36:59.508 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:36:59.508 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:36:39.285588+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:36:59.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 -- 172.21.15.99:0/698046900 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e4c0a11a0 msgr2=0x7f7e4c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e4c0a11a0 0x7f7e4c0a3660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800b5c0 tx=0x7f7e58003720 comp rx=0 tx=0).stop 2024-03-30T00:36:59.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 -- 172.21.15.99:0/698046900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e68106a00 msgr2=0x7f7e680ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e68106a00 0x7f7e680ffc90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7e60007f10 tx=0x7f7e6000d3b0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 -- 172.21.15.99:0/698046900 shutdown_connections 2024-03-30T00:36:59.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e68106a00 0x7f7e680ffc90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e681017a0 0x7f7e681010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e4c0a11a0 0x7f7e4c0a3660 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 --2- 172.21.15.99:0/698046900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e681040d0 0x7f7e680ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 -- 172.21.15.99:0/698046900 >> 172.21.15.99:0/698046900 conn(0x7f7e680fb0d0 msgr2=0x7f7e68102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:59.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 -- 172.21.15.99:0/698046900 shutdown_connections 2024-03-30T00:36:59.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.503+0000 7f7e6d600700 1 -- 172.21.15.99:0/698046900 wait complete. 2024-03-30T00:36:59.512 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-03-30T00:36:59.536 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T00:36:59.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.807+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3994727320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa0101f40 msgr2=0x7f8aa01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.807+0000 7f8aa5a00700 1 --2- 172.21.15.99:0/3994727320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa0101f40 0x7f8aa01043d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0098f0 tx=0x7f8a8c009c00 comp rx=0 tx=0).stop 2024-03-30T00:36:59.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.808+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3994727320 shutdown_connections 2024-03-30T00:36:59.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.808+0000 7f8aa5a00700 1 --2- 172.21.15.99:0/3994727320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aa0104910 0x7f8aa0104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.808+0000 7f8aa5a00700 1 --2- 172.21.15.99:0/3994727320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa00ff5d0 0x7f8aa0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.808+0000 7f8aa5a00700 1 --2- 172.21.15.99:0/3994727320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa0101f40 0x7f8aa01043d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.808+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3994727320 >> 172.21.15.99:0/3994727320 conn(0x7f8aa00fb1d0 msgr2=0x7f8aa00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:59.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.808+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3994727320 shutdown_connections 2024-03-30T00:36:59.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.808+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3994727320 wait complete. 2024-03-30T00:36:59.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.809+0000 7f8aa5a00700 1 Processor -- start 2024-03-30T00:36:59.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.810+0000 7f8aa5a00700 1 -- start start 2024-03-30T00:36:59.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.810+0000 7f8aa5a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa00ff5d0 0x7f8aa0071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.810+0000 7f8aa5a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aa0101f40 0x7f8aa0071a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.810+0000 7f8aa5a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa0104910 0x7f8aa0071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.810+0000 7f8aa5a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0193d20 con 0x7f8aa0104910 2024-03-30T00:36:59.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.810+0000 7f8aa5a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0193e90 con 0x7f8aa00ff5d0 2024-03-30T00:36:59.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.810+0000 7f8aa5a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0194050 con 0x7f8aa0101f40 2024-03-30T00:36:59.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.811+0000 7f8a9ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa00ff5d0 0x7f8aa0071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.811+0000 7f8a9ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa00ff5d0 0x7f8aa0071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48644/0 (socket says 172.21.15.99:48644) 2024-03-30T00:36:59.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.811+0000 7f8a9ea00700 1 -- 172.21.15.99:0/3527537681 learned_addr learned my addr 172.21.15.99:0/3527537681 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:59.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.811+0000 7f8a9e000700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aa0101f40 0x7f8aa0071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.811+0000 7f8a9ea00700 1 -- 172.21.15.99:0/3527537681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aa0101f40 msgr2=0x7f8aa0071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.811+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aa0101f40 0x7f8aa0071a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.811+0000 7f8a9ea00700 1 -- 172.21.15.99:0/3527537681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa0104910 msgr2=0x7f8aa0071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.811+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa0104910 0x7f8aa0071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.811+0000 7f8a9ea00700 1 -- 172.21.15.99:0/3527537681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9001a600 con 0x7f8aa00ff5d0 2024-03-30T00:36:59.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.812+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa00ff5d0 0x7f8aa0071530 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a9001fe80 tx=0x7f8a9001d5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:59.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.814+0000 7f8a97400700 1 -- 172.21.15.99:0/3527537681 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9001dde0 con 0x7f8aa00ff5d0 2024-03-30T00:36:59.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.814+0000 7f8a9e000700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aa0101f40 0x7f8aa0071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:59.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.814+0000 7f8a97400700 1 -- 172.21.15.99:0/3527537681 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a900044b0 con 0x7f8aa00ff5d0 2024-03-30T00:36:59.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.814+0000 7f8a97400700 1 -- 172.21.15.99:0/3527537681 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9001dde0 con 0x7f8aa00ff5d0 2024-03-30T00:36:59.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.814+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3527537681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8c009520 con 0x7f8aa00ff5d0 2024-03-30T00:36:59.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.814+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3527537681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa0194630 con 0x7f8aa00ff5d0 2024-03-30T00:36:59.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.828+0000 7f8a97400700 1 -- 172.21.15.99:0/3527537681 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f8a90004020 con 0x7f8aa00ff5d0 2024-03-30T00:36:59.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.829+0000 7f8ab95c3b80 1 Processor -- start 2024-03-30T00:36:59.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.829+0000 7f8ab95c3b80 1 -- start start 2024-03-30T00:36:59.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.830+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec847d0 0x55f4aec84bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.830+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85130 0x55f4aec859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.830+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec85fd0 0x55f4aec86450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.830+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aebdc990 con 0x55f4aec847d0 2024-03-30T00:36:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.830+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aebda350 con 0x55f4aec85130 2024-03-30T00:36:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.830+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aebd5c40 con 0x55f4aec85fd0 2024-03-30T00:36:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.830+0000 7f8a9e000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec85fd0 0x55f4aec86450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.830+0000 7f8a9e000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec85fd0 0x55f4aec86450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:36570/0 (socket says 172.21.15.99:36570) 2024-03-30T00:36:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.830+0000 7f8a9e000700 1 -- 172.21.15.99:0/440849538 learned_addr learned my addr 172.21.15.99:0/440849538 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:59.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.830+0000 7f8a9f400700 1 --2- 172.21.15.99:0/440849538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec847d0 0x55f4aec84bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.831+0000 7f8a9e000700 1 -- 172.21.15.99:0/440849538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85130 msgr2=0x55f4aec859f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:36:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.831+0000 7f8a9e000700 1 --2- 172.21.15.99:0/440849538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85130 0x55f4aec859f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.831+0000 7f8a9e000700 1 -- 172.21.15.99:0/440849538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec847d0 msgr2=0x55f4aec84bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.831+0000 7f8a9e000700 1 --2- 172.21.15.99:0/440849538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec847d0 0x55f4aec84bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.831+0000 7f8a9e000700 1 -- 172.21.15.99:0/440849538 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4aea573b0 con 0x55f4aec85fd0 2024-03-30T00:36:59.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.831+0000 7f8a9f400700 1 --2- 172.21.15.99:0/440849538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec847d0 0x55f4aec84bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:36:59.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.831+0000 7f8a9e000700 1 --2- 172.21.15.99:0/440849538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec85fd0 0x55f4aec86450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0098f0 tx=0x7f8a8c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc754c7d400fc767 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:59.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.832+0000 7f8a8be00700 1 -- 172.21.15.99:0/440849538 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c01c050 con 0x55f4aec85fd0 2024-03-30T00:36:59.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.832+0000 7f8a8be00700 1 -- 172.21.15.99:0/440849538 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a8c01d040 con 0x55f4aec85fd0 2024-03-30T00:36:59.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.832+0000 7f8a8be00700 1 -- 172.21.15.99:0/440849538 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c01c050 con 0x55f4aec85fd0 2024-03-30T00:36:59.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.832+0000 7f8a97400700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a840a1280 0x7f8a840a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.832+0000 7f8a97400700 1 -- 172.21.15.99:0/3527537681 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f8a90025070 con 0x7f8aa00ff5d0 2024-03-30T00:36:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.833+0000 7f8a9f400700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a840a1280 0x7f8a840a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.833+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/440849538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec85fd0 msgr2=0x55f4aec86450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.833+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/440849538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec85fd0 0x55f4aec86450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0098f0 tx=0x7f8a8c005dc0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.833+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/440849538 shutdown_connections 2024-03-30T00:36:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.833+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/440849538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec85fd0 0x55f4aec86450 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.833+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/440849538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec847d0 0x55f4aec84bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.833+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/440849538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85130 0x55f4aec859f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.833+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/440849538 >> 172.21.15.99:0/440849538 conn(0x55f4aebe36a0 msgr2=0x55f4aec7f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.833+0000 7f8a9f400700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a840a1280 0x7f8a840a3740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0072fb0 tx=0x7f8a9800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.833+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/440849538 shutdown_connections 2024-03-30T00:36:59.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.834+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/440849538 wait complete. 2024-03-30T00:36:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.834+0000 7f8ab95c3b80 1 -- start start 2024-03-30T00:36:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.835+0000 7f8ab95c3b80 1 Processor -- start 2024-03-30T00:36:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.835+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec8e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.835+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85130 0x55f4aec8e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.835+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec95990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.835+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aeb71d80 con 0x55f4aec85130 2024-03-30T00:36:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.835+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aeb71a20 con 0x55f4aec85fd0 2024-03-30T00:36:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.835+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aea596a0 con 0x55f4aec847d0 2024-03-30T00:36:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.835+0000 7f8ab95c3b80 10 client.0 _ll_register_callbacks cb 0x55f4aeb7b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.835+0000 7f8a9f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec95990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.835+0000 7f8a9f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec95990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48662/0 (socket says 172.21.15.99:48662) 2024-03-30T00:36:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.835+0000 7f8a9f400700 1 -- 172.21.15.99:0/3626693794 learned_addr learned my addr 172.21.15.99:0/3626693794 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.836+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec8e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.836+0000 7f8a9e000700 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85130 0x55f4aec8e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.836+0000 7f8a9f400700 1 -- 172.21.15.99:0/3626693794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 msgr2=0x55f4aec8e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:59.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.836+0000 7f8a9f400700 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec8e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.836+0000 7f8a9f400700 1 -- 172.21.15.99:0/3626693794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85130 msgr2=0x55f4aec8e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.836+0000 7f8a9f400700 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85130 0x55f4aec8e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.836+0000 7f8a9f400700 1 -- 172.21.15.99:0/3626693794 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa01058c0 con 0x55f4aec85fd0 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.836+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec8e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.836+0000 7f8a9f400700 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec95990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8a98015b20 tx=0x7f8a98009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57a4eda23b4838f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.836+0000 7f8a88c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.837+0000 7f8a88c00700 1 -- 172.21.15.99:0/3626693794 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800e050 con 0x55f4aec85fd0 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.837+0000 7f8a88c00700 1 -- 172.21.15.99:0/3626693794 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f8a98016c90 con 0x55f4aec85fd0 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.837+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/3626693794 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4aeb96380 con 0x55f4aec85fd0 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.837+0000 7f8a88c00700 1 -- 172.21.15.99:0/3626693794 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a98016e90 con 0x55f4aec85fd0 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.837+0000 7f8ab95c3b80 10 client.25700 fetch_fsmap learned FSMap version 25 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.837+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/3626693794 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f4aea03b60 con 0x55f4aec85fd0 2024-03-30T00:36:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.838+0000 7f8a88c00700 1 -- 172.21.15.99:0/3626693794 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f4aeb96380 con 0x55f4aec85fd0 2024-03-30T00:36:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.838+0000 7f8a88c00700 1 -- 172.21.15.99:0/3626693794 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8a9802b410 con 0x55f4aec85fd0 2024-03-30T00:36:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.838+0000 7f8ab95c3b80 10 client.25700 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:36:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.838+0000 7f8ab95c3b80 10 client.25700 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-03-30T00:36:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.838+0000 7f8ab95c3b80 20 client.25700 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.838+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55f4aec9a280 0x55f4aec9c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.838+0000 7f8ab95c3b80 4 client.25700 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:36:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.838+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/3626693794 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4aebce700 con 0x55f4aec9a280 2024-03-30T00:36:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.838+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55f4aec9a280 0x55f4aec9c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.839+0000 7f8a88c00700 10 client.25700 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:36:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.839+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55f4aec9a280 0x55f4aec9c6a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a78724c2950478c server_cookie=a0fa33fe90f3c79 in_seq=0 out_seq=0 2024-03-30T00:36:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.839+0000 7f8a88c00700 1 -- 172.21.15.99:0/3626693794 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f4aebce700 con 0x55f4aec9a280 2024-03-30T00:36:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.839+0000 7f8a88c00700 10 client.25700 handle_command_reply: tid=0 2024-03-30T00:36:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.839+0000 7f8ab95c3b80 1 client.25700 shutdown 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/3626693794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 msgr2=0x55f4aec95990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec95990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8a98015b20 tx=0x7f8a98009920 comp rx=0 tx=0).stop 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/3626693794 shutdown_connections 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55f4aec9a280 0x55f4aec9c6a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec95990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec8e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8a88c00700 0 client.25700 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/3626693794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85130 0x55f4aec8e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/3626693794 >> 172.21.15.99:0/3626693794 conn(0x55f4aebe36a0 msgr2=0x55f4aec83170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/3626693794 shutdown_connections 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/3626693794 wait complete. 2024-03-30T00:36:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.840+0000 7f8ab95c3b80 20 client.25700 trim_cache size 0 max 16384 2024-03-30T00:36:59.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.854+0000 7f8ab95c3b80 1 Processor -- start 2024-03-30T00:36:59.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8ab95c3b80 1 -- start start 2024-03-30T00:36:59.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec96040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85130 0x55f4aec95950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85fd0 0x55f4aec863d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aec97720 con 0x55f4aec85fd0 2024-03-30T00:36:59.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aec8e020 con 0x55f4aec85130 2024-03-30T00:36:59.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aec84f70 con 0x55f4aec847d0 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8a9ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec96040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8a9e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85130 0x55f4aec95950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8a9ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec96040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:36588/0 (socket says 172.21.15.99:36588) 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8a9f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85fd0 0x55f4aec863d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.855+0000 7f8a9ea00700 1 -- 172.21.15.99:0/940962457 learned_addr learned my addr 172.21.15.99:0/940962457 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.856+0000 7f8a9ea00700 1 -- 172.21.15.99:0/940962457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85130 msgr2=0x55f4aec95950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.856+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/940962457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85130 0x55f4aec95950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.856+0000 7f8a9ea00700 1 -- 172.21.15.99:0/940962457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85fd0 msgr2=0x55f4aec863d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.856+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/940962457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85fd0 0x55f4aec863d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.856+0000 7f8a9ea00700 1 -- 172.21.15.99:0/940962457 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4aeb9bea0 con 0x55f4aec847d0 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.856+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/940962457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec96040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a900aac50 tx=0x7f8a90003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33a469798f8061e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:59.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.856+0000 7f8a94c00700 1 -- 172.21.15.99:0/940962457 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a900ab650 con 0x55f4aec847d0 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.856+0000 7f8a94c00700 1 -- 172.21.15.99:0/940962457 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a900aaf50 con 0x55f4aec847d0 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.856+0000 7f8a94c00700 1 -- 172.21.15.99:0/940962457 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a900a92f0 con 0x55f4aec847d0 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.857+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/940962457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 msgr2=0x55f4aec96040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.857+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/940962457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec96040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a900aac50 tx=0x7f8a90003680 comp rx=0 tx=0).stop 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.857+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/940962457 shutdown_connections 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.857+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/940962457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85fd0 0x55f4aec863d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.857+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/940962457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec96040 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.857+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/940962457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85130 0x55f4aec95950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.857+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/940962457 >> 172.21.15.99:0/940962457 conn(0x55f4aebe36a0 msgr2=0x55f4aec857e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.857+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/940962457 shutdown_connections 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.857+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/940962457 wait complete. 2024-03-30T00:36:59.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.857+0000 7f8ab95c3b80 1 -- start start 2024-03-30T00:36:59.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.858+0000 7f8ab95c3b80 1 Processor -- start 2024-03-30T00:36:59.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.858+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec93e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.858+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85130 0x55f4aec943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.858+0000 7f8ab95c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec94d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.858+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aec952c0 con 0x55f4aec85130 2024-03-30T00:36:59.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.858+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aec95430 con 0x55f4aec85fd0 2024-03-30T00:36:59.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.858+0000 7f8ab95c3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4aec955f0 con 0x55f4aec847d0 2024-03-30T00:36:59.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.858+0000 7f8ab95c3b80 10 client.0 _ll_register_callbacks cb 0x55f4aeb7b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:36:59.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec93e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec94d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9ea00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec93e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:36594/0 (socket says 172.21.15.99:36594) 2024-03-30T00:36:59.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec94d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48668/0 (socket says 172.21.15.99:48668) 2024-03-30T00:36:59.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9ea00700 1 -- 172.21.15.99:0/1968041183 learned_addr learned my addr 172.21.15.99:0/1968041183 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:36:59.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9e000700 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85130 0x55f4aec943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9f400700 1 -- 172.21.15.99:0/1968041183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 msgr2=0x55f4aec93e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:59.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9f400700 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec93e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9f400700 1 -- 172.21.15.99:0/1968041183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85130 msgr2=0x55f4aec943d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:36:59.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9f400700 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85130 0x55f4aec943d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9f400700 1 -- 172.21.15.99:0/1968041183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa0194630 con 0x55f4aec85fd0 2024-03-30T00:36:59.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec93e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:36:59.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a9f400700 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec94d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8a9802b840 tx=0x7f8a98015ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=497a38adc8567c9e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:36:59.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.859+0000 7f8a88c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.860+0000 7f8a88c00700 1 -- 172.21.15.99:0/1968041183 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800e040 con 0x55f4aec85fd0 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.860+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/1968041183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a8c01d040 con 0x55f4aec85fd0 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.860+0000 7f8a88c00700 1 -- 172.21.15.99:0/1968041183 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f8a9802bdf0 con 0x55f4aec85fd0 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.860+0000 7f8a88c00700 1 -- 172.21.15.99:0/1968041183 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a980095e0 con 0x55f4aec85fd0 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.860+0000 7f8ab95c3b80 10 client.25706 fetch_fsmap learned FSMap version 25 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.860+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/1968041183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f4aea5bcd0 con 0x55f4aec85fd0 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.861+0000 7f8a88c00700 1 -- 172.21.15.99:0/1968041183 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a8c01d040 con 0x55f4aec85fd0 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.861+0000 7f8a88c00700 1 -- 172.21.15.99:0/1968041183 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8a98035430 con 0x55f4aec85fd0 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.861+0000 7f8ab95c3b80 10 client.25706 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.861+0000 7f8ab95c3b80 10 client.25706 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.861+0000 7f8ab95c3b80 20 client.25706 populate_metadata read hostname 'smithi099' 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.861+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55f4aecef170 0x55f4aecf1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.861+0000 7f8ab95c3b80 4 client.25706 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:36:59.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.861+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/1968041183 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f4aea24cf0 con 0x55f4aecef170 2024-03-30T00:36:59.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.861+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55f4aecef170 0x55f4aecf1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:36:59.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8a9ea00700 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55f4aecef170 0x55f4aecf1590 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e58aa4784c62cb9 server_cookie=33a6620d1245b709 in_seq=0 out_seq=0 2024-03-30T00:36:59.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8a88c00700 10 client.25706 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:36:59.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8a88c00700 1 -- 172.21.15.99:0/1968041183 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f4aea24cf0 con 0x55f4aecef170 2024-03-30T00:36:59.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8a88c00700 10 client.25706 handle_command_reply: tid=0 2024-03-30T00:36:59.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8ab95c3b80 1 client.25706 shutdown 2024-03-30T00:36:59.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/1968041183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 msgr2=0x55f4aec94d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec94d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8a9802b840 tx=0x7f8a98015ee0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/1968041183 shutdown_connections 2024-03-30T00:36:59.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x55f4aecef170 0x55f4aecf1590 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f4aec85fd0 0x55f4aec94d80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4aec847d0 0x55f4aec93e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8ab95c3b80 1 --2- 172.21.15.99:0/1968041183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4aec85130 0x55f4aec943d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/1968041183 >> 172.21.15.99:0/1968041183 conn(0x55f4aebe36a0 msgr2=0x55f4aec857e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:59.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.862+0000 7f8a88c00700 0 client.25706 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:36:59.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/1968041183 shutdown_connections 2024-03-30T00:36:59.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8ab95c3b80 1 -- 172.21.15.99:0/1968041183 wait complete. 2024-03-30T00:36:59.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8ab95c3b80 20 client.25706 trim_cache size 0 max 16384 2024-03-30T00:36:59.875 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:36:59.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3527537681 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a840a1280 msgr2=0x7f8a840a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8aa5a00700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a840a1280 0x7f8a840a3740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0072fb0 tx=0x7f8a9800b040 comp rx=0 tx=0).stop 2024-03-30T00:36:59.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3527537681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa00ff5d0 msgr2=0x7f8aa0071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:36:59.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8aa5a00700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa00ff5d0 0x7f8aa0071530 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a9001fe80 tx=0x7f8a9001d5c0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3527537681 shutdown_connections 2024-03-30T00:36:59.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8aa5a00700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa0104910 0x7f8aa0071fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8aa5a00700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a840a1280 0x7f8a840a3740 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8aa5a00700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8aa00ff5d0 0x7f8aa0071530 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8aa5a00700 1 --2- 172.21.15.99:0/3527537681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aa0101f40 0x7f8aa0071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:36:59.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.863+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3527537681 >> 172.21.15.99:0/3527537681 conn(0x7f8aa00fb1d0 msgr2=0x7f8aa00fb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:36:59.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.864+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3527537681 shutdown_connections 2024-03-30T00:36:59.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:36:59.864+0000 7f8aa5a00700 1 -- 172.21.15.99:0/3527537681 wait complete. 2024-03-30T00:36:59.892 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T00:37:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.049+0000 7f3ab8200700 1 -- 172.21.15.99:0/4004936331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab0100d40 msgr2=0x7f3ab0101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.049+0000 7f3ab8200700 1 --2- 172.21.15.99:0/4004936331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab0100d40 0x7f3ab0101160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00099c0 tx=0x7f3aa0009cd0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.049+0000 7f3ab8200700 1 -- 172.21.15.99:0/4004936331 shutdown_connections 2024-03-30T00:37:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.049+0000 7f3ab8200700 1 --2- 172.21.15.99:0/4004936331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab01028d0 0x7f3ab0109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.049+0000 7f3ab8200700 1 --2- 172.21.15.99:0/4004936331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab0100d40 0x7f3ab0101160 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.049+0000 7f3ab8200700 1 --2- 172.21.15.99:0/4004936331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab0101f30 0x7f3ab0102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.049+0000 7f3ab8200700 1 -- 172.21.15.99:0/4004936331 >> 172.21.15.99:0/4004936331 conn(0x7f3ab00fc2b0 msgr2=0x7f3ab00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.050+0000 7f3ab8200700 1 -- 172.21.15.99:0/4004936331 shutdown_connections 2024-03-30T00:37:00.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.050+0000 7f3ab8200700 1 -- 172.21.15.99:0/4004936331 wait complete. 2024-03-30T00:37:00.058 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.051+0000 7f3ab8200700 1 Processor -- start 2024-03-30T00:37:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.052+0000 7f3ab8200700 1 -- start start 2024-03-30T00:37:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.052+0000 7f3ab8200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0100d40 0x7f3ab0198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.052+0000 7f3ab8200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab0101f30 0x7f3ab0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.052+0000 7f3ab8200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab01028d0 0x7f3ab01a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.052+0000 7f3ab8200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab01a0900 con 0x7f3ab0100d40 2024-03-30T00:37:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.052+0000 7f3ab8200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab01a0a70 con 0x7f3ab0101f30 2024-03-30T00:37:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.052+0000 7f3ab8200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab01a0c30 con 0x7f3ab01028d0 2024-03-30T00:37:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.052+0000 7f3ab4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab0101f30 0x7f3ab0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.053+0000 7f3ab4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab0101f30 0x7f3ab0199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48686/0 (socket says 172.21.15.99:48686) 2024-03-30T00:37:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.053+0000 7f3ab4e00700 1 -- 172.21.15.99:0/3674358840 learned_addr learned my addr 172.21.15.99:0/3674358840 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.053+0000 7f3ab6200700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab01028d0 0x7f3ab01a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.053+0000 7f3ab5800700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0100d40 0x7f3ab0198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.053+0000 7f3ab4e00700 1 -- 172.21.15.99:0/3674358840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab01028d0 msgr2=0x7f3ab01a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.053+0000 7f3ab4e00700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab01028d0 0x7f3ab01a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.053+0000 7f3ab4e00700 1 -- 172.21.15.99:0/3674358840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0100d40 msgr2=0x7f3ab0198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.053+0000 7f3ab4e00700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0100d40 0x7f3ab0198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.053+0000 7f3ab4e00700 1 -- 172.21.15.99:0/3674358840 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa00095f0 con 0x7f3ab0101f30 2024-03-30T00:37:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.054+0000 7f3ab4e00700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab0101f30 0x7f3ab0199350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400d980 tx=0x7f3aa400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.054+0000 7f3aae000700 1 -- 172.21.15.99:0/3674358840 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4004220 con 0x7f3ab0101f30 2024-03-30T00:37:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.054+0000 7f3ab8200700 1 -- 172.21.15.99:0/3674358840 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab010b040 con 0x7f3ab0101f30 2024-03-30T00:37:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.054+0000 7f3ab6200700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab01028d0 0x7f3ab01a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.055+0000 7f3aae000700 1 -- 172.21.15.99:0/3674358840 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3aa4007640 con 0x7f3ab0101f30 2024-03-30T00:37:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.055+0000 7f3aae000700 1 -- 172.21.15.99:0/3674358840 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4004220 con 0x7f3ab0101f30 2024-03-30T00:37:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.055+0000 7f3ab8200700 1 -- 172.21.15.99:0/3674358840 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab010b5a0 con 0x7f3ab0101f30 2024-03-30T00:37:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.060+0000 7f3acbc94b80 1 Processor -- start 2024-03-30T00:37:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.060+0000 7f3acbc94b80 1 -- start start 2024-03-30T00:37:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.061+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bc9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.061+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.061+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bcaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.061+0000 7f3acbc94b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879b21930 con 0x562879bc9b60 2024-03-30T00:37:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.061+0000 7f3acbc94b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879b1f2f0 con 0x562879bcaa60 2024-03-30T00:37:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.061+0000 7f3acbc94b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879b1abe0 con 0x562879bc9200 2024-03-30T00:37:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.061+0000 7f3ab6200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bc9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.061+0000 7f3ab6200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bc9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:36618/0 (socket says 172.21.15.99:36618) 2024-03-30T00:37:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.061+0000 7f3ab6200700 1 -- 172.21.15.99:0/2825809862 learned_addr learned my addr 172.21.15.99:0/2825809862 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.061+0000 7f3aae000700 1 -- 172.21.15.99:0/3674358840 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f3aa40052b0 con 0x7f3ab0101f30 2024-03-30T00:37:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.061+0000 7f3ab5800700 1 --2- 172.21.15.99:0/2825809862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.062+0000 7f3ab6200700 1 -- 172.21.15.99:0/2825809862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 msgr2=0x562879bcaef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:37:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.062+0000 7f3ab6200700 1 --2- 172.21.15.99:0/2825809862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bcaef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.062+0000 7f3ab6200700 1 -- 172.21.15.99:0/2825809862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 msgr2=0x562879bca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.062+0000 7f3ab6200700 1 --2- 172.21.15.99:0/2825809862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.062+0000 7f3ab6200700 1 -- 172.21.15.99:0/2825809862 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56287999c350 con 0x562879bc9200 2024-03-30T00:37:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.062+0000 7f3ab6200700 1 --2- 172.21.15.99:0/2825809862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bc9620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009440 tx=0x7f3aa8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fed62766c014f04f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.062+0000 7f3a9ea00700 1 -- 172.21.15.99:0/2825809862 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800e070 con 0x562879bc9200 2024-03-30T00:37:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.063+0000 7f3a9ea00700 1 -- 172.21.15.99:0/2825809862 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3aa800f040 con 0x562879bc9200 2024-03-30T00:37:00.071 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.063+0000 7f3a9ea00700 1 -- 172.21.15.99:0/2825809862 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8015bb0 con 0x562879bc9200 2024-03-30T00:37:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.063+0000 7f3acbc94b80 1 -- 172.21.15.99:0/2825809862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 msgr2=0x562879bc9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.063+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/2825809862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bc9620 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009440 tx=0x7f3aa8009920 comp rx=0 tx=0).stop 2024-03-30T00:37:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.063+0000 7f3acbc94b80 1 -- 172.21.15.99:0/2825809862 shutdown_connections 2024-03-30T00:37:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.063+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/2825809862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bcaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.063+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/2825809862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bc9620 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.063+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/2825809862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.063+0000 7f3acbc94b80 1 -- 172.21.15.99:0/2825809862 >> 172.21.15.99:0/2825809862 conn(0x562879b28640 msgr2=0x562879bc4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.064+0000 7f3acbc94b80 1 -- 172.21.15.99:0/2825809862 shutdown_connections 2024-03-30T00:37:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.064+0000 7f3acbc94b80 1 -- 172.21.15.99:0/2825809862 wait complete. 2024-03-30T00:37:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.064+0000 7f3aae000700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3a980a1110 0x7f3a980a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.064+0000 7f3aae000700 1 -- 172.21.15.99:0/3674358840 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f3aa40cc3e0 con 0x7f3ab0101f30 2024-03-30T00:37:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.064+0000 7f3acbc94b80 1 -- start start 2024-03-30T00:37:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.064+0000 7f3ab5800700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3a980a1110 0x7f3a980a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.064+0000 7f3acbc94b80 1 Processor -- start 2024-03-30T00:37:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.065+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bd31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.065+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bd36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.065+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bda720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.065+0000 7f3acbc94b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879ab6d20 con 0x562879bc9b60 2024-03-30T00:37:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.065+0000 7f3acbc94b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879ab69c0 con 0x562879bcaa60 2024-03-30T00:37:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.065+0000 7f3acbc94b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56287999e640 con 0x562879bc9200 2024-03-30T00:37:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.065+0000 7f3acbc94b80 10 client.0 _ll_register_callbacks cb 0x562879ac0410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.065+0000 7f3ab6200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bda720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.065+0000 7f3ab6200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bda720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48702/0 (socket says 172.21.15.99:48702) 2024-03-30T00:37:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.065+0000 7f3ab6200700 1 -- 172.21.15.99:0/1942039360 learned_addr learned my addr 172.21.15.99:0/1942039360 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.065+0000 7f3ab4e00700 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bd36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.066+0000 7f3ab6200700 1 -- 172.21.15.99:0/1942039360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 msgr2=0x562879bd31a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:37:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.066+0000 7f3ab6200700 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bd31a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.066+0000 7f3ab6200700 1 -- 172.21.15.99:0/1942039360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 msgr2=0x562879bd36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.066+0000 7f3ab6200700 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bd36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.066+0000 7f3ab6200700 1 -- 172.21.15.99:0/1942039360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa80090f0 con 0x562879bcaa60 2024-03-30T00:37:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.066+0000 7f3ab6200700 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bda720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8000f80 tx=0x7f3aa8005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b738affb78735a54 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.066+0000 7f3a93400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:37:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.066+0000 7f3ab5800700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3a980a1110 0x7f3a980a35d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3ab019a290 tx=0x7f3aa0022040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.066+0000 7f3a93400700 1 -- 172.21.15.99:0/1942039360 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800e040 con 0x562879bcaa60 2024-03-30T00:37:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.066+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1942039360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562879adb320 con 0x562879bcaa60 2024-03-30T00:37:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.066+0000 7f3a93400700 1 -- 172.21.15.99:0/1942039360 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f3aa8013b20 con 0x562879bcaa60 2024-03-30T00:37:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.067+0000 7f3a93400700 1 -- 172.21.15.99:0/1942039360 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aa8013ce0 con 0x562879bcaa60 2024-03-30T00:37:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.067+0000 7f3acbc94b80 10 client.25715 fetch_fsmap learned FSMap version 25 2024-03-30T00:37:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.067+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1942039360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562879948b00 con 0x562879bcaa60 2024-03-30T00:37:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.067+0000 7f3a93400700 1 -- 172.21.15.99:0/1942039360 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562879adb320 con 0x562879bcaa60 2024-03-30T00:37:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.068+0000 7f3a93400700 1 -- 172.21.15.99:0/1942039360 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3aa801c050 con 0x562879bcaa60 2024-03-30T00:37:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.068+0000 7f3acbc94b80 10 client.25715 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:37:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.068+0000 7f3acbc94b80 10 client.25715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.068+0000 7f3acbc94b80 20 client.25715 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.068+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562879bdee30 0x562879be1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.068+0000 7f3acbc94b80 4 client.25715 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:37:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.068+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1942039360 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562879b136a0 con 0x562879bdee30 2024-03-30T00:37:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.068+0000 7f3ab5800700 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562879bdee30 0x562879be1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.069+0000 7f3a93400700 10 client.25715 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.069+0000 7f3ab5800700 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562879bdee30 0x562879be1250 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e0f51bc27428d7b server_cookie=b1d4e408583d078f in_seq=0 out_seq=0 2024-03-30T00:37:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.070+0000 7f3a93400700 1 -- 172.21.15.99:0/1942039360 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562879b136a0 con 0x562879bdee30 2024-03-30T00:37:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.070+0000 7f3a93400700 10 client.25715 handle_command_reply: tid=0 2024-03-30T00:37:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.070+0000 7f3acbc94b80 1 client.25715 shutdown 2024-03-30T00:37:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.070+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1942039360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 msgr2=0x562879bda720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.070+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bda720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8000f80 tx=0x7f3aa8005b90 comp rx=0 tx=0).stop 2024-03-30T00:37:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.071+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1942039360 shutdown_connections 2024-03-30T00:37:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.071+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562879bdee30 0x562879be1250 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.071+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bda720 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.071+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bd31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.071+0000 7f3a93400700 0 client.25715 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.071+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/1942039360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bd36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.071+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1942039360 >> 172.21.15.99:0/1942039360 conn(0x562879b28640 msgr2=0x562879bc7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.071+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1942039360 shutdown_connections 2024-03-30T00:37:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.071+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1942039360 wait complete. 2024-03-30T00:37:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.071+0000 7f3acbc94b80 20 client.25715 trim_cache size 0 max 16384 2024-03-30T00:37:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.101+0000 7f3acbc94b80 1 Processor -- start 2024-03-30T00:37:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.102+0000 7f3acbc94b80 1 -- start start 2024-03-30T00:37:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.102+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bc95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.102+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bc9b60 0x562879bcaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.102+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bcaa60 0x562879bdadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.103+0000 7f3ab4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bc9b60 0x562879bcaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.103+0000 7f3ab4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bc9b60 0x562879bcaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48712/0 (socket says 172.21.15.99:48712) 2024-03-30T00:37:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.103+0000 7f3ab4e00700 1 -- 172.21.15.99:0/1754548862 learned_addr learned my addr 172.21.15.99:0/1754548862 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.103+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1754548862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879bdb8a0 con 0x562879bcaa60 2024-03-30T00:37:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.103+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1754548862 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879bc45e0 con 0x562879bc9b60 2024-03-30T00:37:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.103+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1754548862 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879be15e0 con 0x562879bc9200 2024-03-30T00:37:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.103+0000 7f3ab5800700 1 --2- 172.21.15.99:0/1754548862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bc95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.103+0000 7f3ab6200700 1 --2- 172.21.15.99:0/1754548862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bcaa60 0x562879bdadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.104+0000 7f3ab6200700 1 -- 172.21.15.99:0/1754548862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 msgr2=0x562879bc95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.104+0000 7f3ab6200700 1 --2- 172.21.15.99:0/1754548862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bc95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.105+0000 7f3ab6200700 1 -- 172.21.15.99:0/1754548862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bc9b60 msgr2=0x562879bcaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.105+0000 7f3ab6200700 1 --2- 172.21.15.99:0/1754548862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bc9b60 0x562879bcaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.105+0000 7f3ab6200700 1 -- 172.21.15.99:0/1754548862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562879ae0e40 con 0x562879bcaa60 2024-03-30T00:37:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.105+0000 7f3ab4e00700 1 --2- 172.21.15.99:0/1754548862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bc9b60 0x562879bcaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.105+0000 7f3ab5800700 1 --2- 172.21.15.99:0/1754548862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bc95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.105+0000 7f3ab6200700 1 --2- 172.21.15.99:0/1754548862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bcaa60 0x562879bdadd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8018000 tx=0x7f3aa8009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244448ee6d0f874b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.106+0000 7f3a9f400700 1 -- 172.21.15.99:0/1754548862 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80144b0 con 0x562879bcaa60 2024-03-30T00:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.106+0000 7f3a9f400700 1 -- 172.21.15.99:0/1754548862 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3aa8028360 con 0x562879bcaa60 2024-03-30T00:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.106+0000 7f3a9f400700 1 -- 172.21.15.99:0/1754548862 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80162d0 con 0x562879bcaa60 2024-03-30T00:37:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.107+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1754548862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bcaa60 msgr2=0x562879bdadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.107+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/1754548862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bcaa60 0x562879bdadd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8018000 tx=0x7f3aa8009690 comp rx=0 tx=0).stop 2024-03-30T00:37:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.108+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1754548862 shutdown_connections 2024-03-30T00:37:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.108+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/1754548862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bcaa60 0x562879bdadd0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.108+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/1754548862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bc95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.108+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/1754548862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bc9b60 0x562879bcaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.108+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1754548862 >> 172.21.15.99:0/1754548862 conn(0x562879b28640 msgr2=0x562879bd26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.108+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1754548862 shutdown_connections 2024-03-30T00:37:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.108+0000 7f3acbc94b80 1 -- 172.21.15.99:0/1754548862 wait complete. 2024-03-30T00:37:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.109+0000 7f3acbc94b80 1 -- start start 2024-03-30T00:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.111+0000 7f3acbc94b80 1 Processor -- start 2024-03-30T00:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.111+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bd8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.112+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bd8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.112+0000 7f3acbc94b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bd94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.112+0000 7f3acbc94b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879bd9a10 con 0x562879bc9b60 2024-03-30T00:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.112+0000 7f3acbc94b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879bd9b80 con 0x562879bcaa60 2024-03-30T00:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.112+0000 7f3acbc94b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562879bd9d40 con 0x562879bc9200 2024-03-30T00:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.112+0000 7f3acbc94b80 10 client.0 _ll_register_callbacks cb 0x562879ac0410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.113+0000 7f3ab5800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bd8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.113+0000 7f3ab5800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bd8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:36650/0 (socket says 172.21.15.99:36650) 2024-03-30T00:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.113+0000 7f3ab5800700 1 -- 172.21.15.99:0/4114153276 learned_addr learned my addr 172.21.15.99:0/4114153276 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.114+0000 7f3ab6200700 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bd94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.114+0000 7f3ab4e00700 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bd8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.114+0000 7f3ab5800700 1 -- 172.21.15.99:0/4114153276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 msgr2=0x562879bd94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.114+0000 7f3ab5800700 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bd94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.114+0000 7f3ab5800700 1 -- 172.21.15.99:0/4114153276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 msgr2=0x562879bd8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.114+0000 7f3ab5800700 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bd8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.115+0000 7f3ab5800700 1 -- 172.21.15.99:0/4114153276 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab0109ad0 con 0x562879bc9200 2024-03-30T00:37:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.115+0000 7f3ab4e00700 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bd8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:37:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.115+0000 7f3ab5800700 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bd8050 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000bdb0 tx=0x7f3aa0005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf9b42fad62a9f49 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.115+0000 7f3a93400700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:37:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.116+0000 7f3a93400700 1 -- 172.21.15.99:0/4114153276 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0034040 con 0x562879bc9200 2024-03-30T00:37:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.116+0000 7f3acbc94b80 1 -- 172.21.15.99:0/4114153276 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3aa800f040 con 0x562879bc9200 2024-03-30T00:37:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.116+0000 7f3a93400700 1 -- 172.21.15.99:0/4114153276 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f3aa0039b00 con 0x562879bc9200 2024-03-30T00:37:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.117+0000 7f3a93400700 1 -- 172.21.15.99:0/4114153276 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aa0025050 con 0x562879bc9200 2024-03-30T00:37:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.117+0000 7f3acbc94b80 10 client.25741 fetch_fsmap learned FSMap version 25 2024-03-30T00:37:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.117+0000 7f3acbc94b80 1 -- 172.21.15.99:0/4114153276 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f3ab010b5a0 con 0x562879bc9200 2024-03-30T00:37:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.117+0000 7f3a93400700 1 -- 172.21.15.99:0/4114153276 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3aa800f040 con 0x562879bc9200 2024-03-30T00:37:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.117+0000 7f3a93400700 1 -- 172.21.15.99:0/4114153276 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3aa000f460 con 0x562879bc9200 2024-03-30T00:37:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.118+0000 7f3acbc94b80 10 client.25741 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:37:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.118+0000 7f3acbc94b80 10 client.25741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.118+0000 7f3acbc94b80 20 client.25741 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.118+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562879c335f0 0x562879c35ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.118+0000 7f3acbc94b80 4 client.25741 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:37:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.118+0000 7f3acbc94b80 1 -- 172.21.15.99:0/4114153276 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562879969c90 con 0x562879c335f0 2024-03-30T00:37:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.118+0000 7f3ab4e00700 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562879c335f0 0x562879c35ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.119+0000 7f3ab4e00700 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562879c335f0 0x562879c35ab0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c847b5dc96d848c server_cookie=c1fc0e1e0e18c624 in_seq=0 out_seq=0 2024-03-30T00:37:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.120+0000 7f3a93400700 10 client.25741 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.131+0000 7f3a93400700 1 -- 172.21.15.99:0/4114153276 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562879969c90 con 0x562879c335f0 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.131+0000 7f3a93400700 10 client.25741 handle_command_reply: tid=0 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.131+0000 7f3acbc94b80 1 client.25741 shutdown 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.131+0000 7f3acbc94b80 1 -- 172.21.15.99:0/4114153276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 msgr2=0x562879bd8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.131+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bd8050 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000bdb0 tx=0x7f3aa0005040 comp rx=0 tx=0).stop 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.133+0000 7f3acbc94b80 1 -- 172.21.15.99:0/4114153276 shutdown_connections 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.133+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x562879c335f0 0x562879c35ab0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.133+0000 7f3a93400700 0 client.25741 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.133+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562879bcaa60 0x562879bd94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.133+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562879bc9200 0x562879bd8050 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.133+0000 7f3acbc94b80 1 --2- 172.21.15.99:0/4114153276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562879bc9b60 0x562879bd8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.133+0000 7f3acbc94b80 1 -- 172.21.15.99:0/4114153276 >> 172.21.15.99:0/4114153276 conn(0x562879b28640 msgr2=0x562879bd25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.133+0000 7f3acbc94b80 1 -- 172.21.15.99:0/4114153276 shutdown_connections 2024-03-30T00:37:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:37:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:37:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:37:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:37:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.133+0000 7f3acbc94b80 1 -- 172.21.15.99:0/4114153276 wait complete. 2024-03-30T00:37:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.134+0000 7f3acbc94b80 20 client.25741 trim_cache size 0 max 16384 2024-03-30T00:37:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.135+0000 7f3ab8200700 1 -- 172.21.15.99:0/3674358840 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3a980a1110 msgr2=0x7f3a980a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.135+0000 7f3ab8200700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3a980a1110 0x7f3a980a35d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3ab019a290 tx=0x7f3aa0022040 comp rx=0 tx=0).stop 2024-03-30T00:37:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.135+0000 7f3ab8200700 1 -- 172.21.15.99:0/3674358840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab0101f30 msgr2=0x7f3ab0199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.135+0000 7f3ab8200700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab0101f30 0x7f3ab0199350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400d980 tx=0x7f3aa400dc90 comp rx=0 tx=0).stop 2024-03-30T00:37:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.137+0000 7f3ab8200700 1 -- 172.21.15.99:0/3674358840 shutdown_connections 2024-03-30T00:37:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.137+0000 7f3ab8200700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ab01028d0 0x7f3ab01a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.137+0000 7f3ab8200700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ab0100d40 0x7f3ab0198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.137+0000 7f3ab8200700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3a980a1110 0x7f3a980a35d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.137+0000 7f3ab8200700 1 --2- 172.21.15.99:0/3674358840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab0101f30 0x7f3ab0199350 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.137+0000 7f3ab8200700 1 -- 172.21.15.99:0/3674358840 >> 172.21.15.99:0/3674358840 conn(0x7f3ab00fc2b0 msgr2=0x7f3ab01051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.137+0000 7f3ab8200700 1 -- 172.21.15.99:0/3674358840 shutdown_connections 2024-03-30T00:37:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:00.137+0000 7f3ab8200700 1 -- 172.21.15.99:0/3674358840 wait complete. 2024-03-30T00:37:00.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.196+0000 7fd7e3e00700 1 -- 172.21.15.99:0/4237736600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7dc108570 msgr2=0x7fd7dc1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.196+0000 7fd7e3e00700 1 --2- 172.21.15.99:0/4237736600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7dc108570 0x7fd7dc1089d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00099c0 tx=0x7fd7d0009cd0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.197+0000 7fd7e3e00700 1 -- 172.21.15.99:0/4237736600 shutdown_connections 2024-03-30T00:37:00.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.197+0000 7fd7e3e00700 1 --2- 172.21.15.99:0/4237736600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7dc108f10 0x7fd7dc10f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.197+0000 7fd7e3e00700 1 --2- 172.21.15.99:0/4237736600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc107380 0x7fd7dc1077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.197+0000 7fd7e3e00700 1 --2- 172.21.15.99:0/4237736600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7dc108570 0x7fd7dc1089d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.197+0000 7fd7e3e00700 1 -- 172.21.15.99:0/4237736600 >> 172.21.15.99:0/4237736600 conn(0x7fd7dc076320 msgr2=0x7fd7dc078760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.197+0000 7fd7e3e00700 1 -- 172.21.15.99:0/4237736600 shutdown_connections 2024-03-30T00:37:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.197+0000 7fd7e3e00700 1 -- 172.21.15.99:0/4237736600 wait complete. 2024-03-30T00:37:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.198+0000 7fd7e3e00700 1 Processor -- start 2024-03-30T00:37:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.198+0000 7fd7e3e00700 1 -- start start 2024-03-30T00:37:00.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7dc107380 0x7fd7dc19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e3e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc108570 0x7fd7dc19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7dc108f10 0x7fd7dc1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e3e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc1a4d90 con 0x7fd7dc107380 2024-03-30T00:37:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e3e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc1a4ed0 con 0x7fd7dc108f10 2024-03-30T00:37:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e3e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc1a5060 con 0x7fd7dc108570 2024-03-30T00:37:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7dc107380 0x7fd7dc19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e0a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc108570 0x7fd7dc19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7dc107380 0x7fd7dc19d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44136/0 (socket says 172.21.15.99:44136) 2024-03-30T00:37:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e1400700 1 -- 172.21.15.99:0/1570771318 learned_addr learned my addr 172.21.15.99:0/1570771318 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e1e00700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7dc108f10 0x7fd7dc1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e1400700 1 -- 172.21.15.99:0/1570771318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc108570 msgr2=0x7fd7dc19d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e1400700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc108570 0x7fd7dc19d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e1400700 1 -- 172.21.15.99:0/1570771318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7dc108f10 msgr2=0x7fd7dc1a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e1400700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7dc108f10 0x7fd7dc1a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.199+0000 7fd7e1400700 1 -- 172.21.15.99:0/1570771318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d00095f0 con 0x7fd7dc107380 2024-03-30T00:37:00.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.200+0000 7fd7e1400700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7dc107380 0x7fd7dc19d260 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00e970 tx=0x7fd7cc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.200+0000 7fd7da000700 1 -- 172.21.15.99:0/1570771318 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc009790 con 0x7fd7dc107380 2024-03-30T00:37:00.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.200+0000 7fd7da000700 1 -- 172.21.15.99:0/1570771318 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7cc0044b0 con 0x7fd7dc107380 2024-03-30T00:37:00.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.200+0000 7fd7e3e00700 1 -- 172.21.15.99:0/1570771318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc1a5340 con 0x7fd7dc107380 2024-03-30T00:37:00.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.200+0000 7fd7da000700 1 -- 172.21.15.99:0/1570771318 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc003dd0 con 0x7fd7dc107380 2024-03-30T00:37:00.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.200+0000 7fd7e3e00700 1 -- 172.21.15.99:0/1570771318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc1a58a0 con 0x7fd7dc107380 2024-03-30T00:37:00.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.205+0000 7fd7da000700 1 -- 172.21.15.99:0/1570771318 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fd7cc004000 con 0x7fd7dc107380 2024-03-30T00:37:00.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.207+0000 7fd7da000700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c40a11a0 0x7fd7c40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.207+0000 7fd7f79ebb80 1 Processor -- start 2024-03-30T00:37:00.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.207+0000 7fd7da000700 1 -- 172.21.15.99:0/1570771318 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fd7cc014070 con 0x7fd7dc107380 2024-03-30T00:37:00.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.207+0000 7fd7f79ebb80 1 -- start start 2024-03-30T00:37:00.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7e0a00700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c40a11a0 0x7fd7c40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef44bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45fd0 0x55e9aef46450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aee9c990 con 0x55e9aef45fd0 2024-03-30T00:37:00.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aee9a350 con 0x55e9aef447d0 2024-03-30T00:37:00.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aee95c40 con 0x55e9aef45130 2024-03-30T00:37:00.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7e1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef44bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7e1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef44bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48750/0 (socket says 172.21.15.99:48750) 2024-03-30T00:37:00.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7e1400700 1 -- 172.21.15.99:0/2147411274 learned_addr learned my addr 172.21.15.99:0/2147411274 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7e1400700 1 -- 172.21.15.99:0/2147411274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 msgr2=0x55e9aef459f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:37:00.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7e1400700 1 --2- 172.21.15.99:0/2147411274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef459f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7e1400700 1 -- 172.21.15.99:0/2147411274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45fd0 msgr2=0x55e9aef46450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7e1400700 1 --2- 172.21.15.99:0/2147411274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45fd0 0x55e9aef46450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.208+0000 7fd7e1400700 1 -- 172.21.15.99:0/2147411274 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9aed173b0 con 0x55e9aef447d0 2024-03-30T00:37:00.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.209+0000 7fd7e1400700 1 --2- 172.21.15.99:0/2147411274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef44bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0a1f00 tx=0x7fd7cc099af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f72541e40180878d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.209+0000 7fd7ca600700 1 -- 172.21.15.99:0/2147411274 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0b1070 con 0x55e9aef447d0 2024-03-30T00:37:00.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.209+0000 7fd7ca600700 1 -- 172.21.15.99:0/2147411274 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7cc099fb0 con 0x55e9aef447d0 2024-03-30T00:37:00.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.209+0000 7fd7ca600700 1 -- 172.21.15.99:0/2147411274 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc098350 con 0x55e9aef447d0 2024-03-30T00:37:00.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.210+0000 7fd7e0a00700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c40a11a0 0x7fd7c40a3660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000b580 tx=0x7fd7d001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.210+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2147411274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 msgr2=0x55e9aef44bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.210+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2147411274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef44bf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0a1f00 tx=0x7fd7cc099af0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.211+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2147411274 shutdown_connections 2024-03-30T00:37:00.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.211+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2147411274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45fd0 0x55e9aef46450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.211+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2147411274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef44bf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.211+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2147411274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.211+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2147411274 >> 172.21.15.99:0/2147411274 conn(0x55e9aeea36a0 msgr2=0x55e9aef3f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.211+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2147411274 shutdown_connections 2024-03-30T00:37:00.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.211+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2147411274 wait complete. 2024-03-30T00:37:00.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.212+0000 7fd7f79ebb80 1 -- start start 2024-03-30T00:37:00.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.212+0000 7fd7f79ebb80 1 Processor -- start 2024-03-30T00:37:00.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef447d0 0x55e9aef4e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef4ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef45fd0 0x55e9aef55d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aee31d80 con 0x55e9aef447d0 2024-03-30T00:37:00.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aee31a20 con 0x55e9aef45fd0 2024-03-30T00:37:00.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aed196a0 con 0x55e9aef45130 2024-03-30T00:37:00.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7f79ebb80 10 client.0 _ll_register_callbacks cb 0x55e9aee3b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:00.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7e1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef447d0 0x55e9aef4e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7e1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef447d0 0x55e9aef4e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44150/0 (socket says 172.21.15.99:44150) 2024-03-30T00:37:00.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7e1400700 1 -- 172.21.15.99:0/2286653768 learned_addr learned my addr 172.21.15.99:0/2286653768 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7e1e00700 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef45fd0 0x55e9aef55d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7e0a00700 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef4ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.213+0000 7fd7e1400700 1 -- 172.21.15.99:0/2286653768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 msgr2=0x55e9aef4ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.214+0000 7fd7e1400700 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef4ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.214+0000 7fd7e1400700 1 -- 172.21.15.99:0/2286653768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef45fd0 msgr2=0x55e9aef55d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.214+0000 7fd7e1400700 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef45fd0 0x55e9aef55d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.214+0000 7fd7e1400700 1 -- 172.21.15.99:0/2286653768 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc1a58a0 con 0x55e9aef447d0 2024-03-30T00:37:00.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.214+0000 7fd7e1e00700 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef45fd0 0x55e9aef55d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:00.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.214+0000 7fd7e1400700 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef447d0 0x55e9aef4e840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0ca9d0 tx=0x7fd7cc09a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad2b803151358ae server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.214+0000 7fd7bea00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:37:00.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.214+0000 7fd7bea00700 1 -- 172.21.15.99:0/2286653768 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc07b060 con 0x55e9aef447d0 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.214+0000 7fd7bea00700 1 -- 172.21.15.99:0/2286653768 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fd7cc0b2070 con 0x55e9aef447d0 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.214+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2286653768 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9aee56380 con 0x55e9aef447d0 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.215+0000 7fd7bea00700 1 -- 172.21.15.99:0/2286653768 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7cc0b8070 con 0x55e9aef447d0 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.215+0000 7fd7f79ebb80 10 client.16038 fetch_fsmap learned FSMap version 25 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.215+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2286653768 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e9aecc3b60 con 0x55e9aef447d0 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.215+0000 7fd7bea00700 1 -- 172.21.15.99:0/2286653768 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e9aee56380 con 0x55e9aef447d0 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.215+0000 7fd7bea00700 1 -- 172.21.15.99:0/2286653768 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd7cc0994a0 con 0x55e9aef447d0 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.215+0000 7fd7f79ebb80 10 client.16038 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.215+0000 7fd7f79ebb80 10 client.16038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.215+0000 7fd7f79ebb80 20 client.16038 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.215+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55e9aef5a120 0x55e9aef5c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.216+0000 7fd7f79ebb80 4 client.16038 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:37:00.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.216+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2286653768 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9aee8e700 con 0x55e9aef5a120 2024-03-30T00:37:00.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.216+0000 7fd7e0a00700 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55e9aef5a120 0x55e9aef5c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.217+0000 7fd7e0a00700 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55e9aef5a120 0x55e9aef5c540 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8089937b8e3fa6f server_cookie=1001fc8ca4cf382f in_seq=0 out_seq=0 2024-03-30T00:37:00.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.217+0000 7fd7bea00700 10 client.16038 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:37:00.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.218+0000 7fd7bea00700 1 -- 172.21.15.99:0/2286653768 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9aee8e700 con 0x55e9aef5a120 2024-03-30T00:37:00.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.218+0000 7fd7bea00700 10 client.16038 handle_command_reply: tid=0 2024-03-30T00:37:00.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.218+0000 7fd7f79ebb80 1 client.16038 shutdown 2024-03-30T00:37:00.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.218+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2286653768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef447d0 msgr2=0x55e9aef4e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.218+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef447d0 0x55e9aef4e840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0ca9d0 tx=0x7fd7cc09a120 comp rx=0 tx=0).stop 2024-03-30T00:37:00.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.219+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2286653768 shutdown_connections 2024-03-30T00:37:00.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.219+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55e9aef5a120 0x55e9aef5c540 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.219+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef45fd0 0x55e9aef55d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.219+0000 7fd7bea00700 0 client.16038 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:37:00.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.219+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef447d0 0x55e9aef4e840 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.219+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2286653768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef4ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.219+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2286653768 >> 172.21.15.99:0/2286653768 conn(0x55e9aeea36a0 msgr2=0x55e9aef43170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.219+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2286653768 shutdown_connections 2024-03-30T00:37:00.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.219+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2286653768 wait complete. 2024-03-30T00:37:00.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.219+0000 7fd7f79ebb80 20 client.16038 trim_cache size 0 max 16384 2024-03-30T00:37:00.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.240+0000 7fd7f79ebb80 1 Processor -- start 2024-03-30T00:37:00.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.240+0000 7fd7f79ebb80 1 -- start start 2024-03-30T00:37:00.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.240+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.240+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.240+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45fd0 0x55e9aef4dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.240+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aef404e0 con 0x55e9aef45fd0 2024-03-30T00:37:00.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.240+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aef45e80 con 0x55e9aef447d0 2024-03-30T00:37:00.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.240+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aef56ea0 con 0x55e9aef45130 2024-03-30T00:37:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.241+0000 7fd7e0a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.241+0000 7fd7e1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.241+0000 7fd7e0a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef463d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:36682/0 (socket says 172.21.15.99:36682) 2024-03-30T00:37:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.241+0000 7fd7e0a00700 1 -- 172.21.15.99:0/356732815 learned_addr learned my addr 172.21.15.99:0/356732815 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.241+0000 7fd7e1e00700 1 --2- 172.21.15.99:0/356732815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45fd0 0x55e9aef4dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.241+0000 7fd7e0a00700 1 -- 172.21.15.99:0/356732815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 msgr2=0x55e9aef563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.241+0000 7fd7e0a00700 1 --2- 172.21.15.99:0/356732815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.241+0000 7fd7e0a00700 1 -- 172.21.15.99:0/356732815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45fd0 msgr2=0x55e9aef4dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.241+0000 7fd7e0a00700 1 --2- 172.21.15.99:0/356732815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45fd0 0x55e9aef4dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.241+0000 7fd7e0a00700 1 -- 172.21.15.99:0/356732815 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9aee5bea0 con 0x55e9aef45130 2024-03-30T00:37:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.241+0000 7fd7e1400700 1 --2- 172.21.15.99:0/356732815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:00.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.242+0000 7fd7e0a00700 1 --2- 172.21.15.99:0/356732815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef463d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000bf10 tx=0x7fd7d0022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d243b35fec5f61eb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.242+0000 7fd7cb000700 1 -- 172.21.15.99:0/356732815 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d001d040 con 0x55e9aef45130 2024-03-30T00:37:00.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.242+0000 7fd7cb000700 1 -- 172.21.15.99:0/356732815 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7d0029040 con 0x55e9aef45130 2024-03-30T00:37:00.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.242+0000 7fd7cb000700 1 -- 172.21.15.99:0/356732815 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0023670 con 0x55e9aef45130 2024-03-30T00:37:00.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.243+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/356732815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 msgr2=0x55e9aef463d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.243+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/356732815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef463d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000bf10 tx=0x7fd7d0022000 comp rx=0 tx=0).stop 2024-03-30T00:37:00.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.244+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/356732815 shutdown_connections 2024-03-30T00:37:00.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.244+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/356732815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45fd0 0x55e9aef4dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.244+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/356732815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.244+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/356732815 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45130 0x55e9aef463d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.244+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/356732815 >> 172.21.15.99:0/356732815 conn(0x55e9aeea36a0 msgr2=0x55e9aef60ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.244+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/356732815 shutdown_connections 2024-03-30T00:37:00.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.244+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/356732815 wait complete. 2024-03-30T00:37:00.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.245+0000 7fd7f79ebb80 1 -- start start 2024-03-30T00:37:00.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.245+0000 7fd7f79ebb80 1 Processor -- start 2024-03-30T00:37:00.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef53ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45130 0x55e9aef54000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7f79ebb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45fd0 0x55e9aef54f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aef554b0 con 0x55e9aef45130 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7e1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef53ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aef55620 con 0x55e9aef447d0 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7f79ebb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aef557e0 con 0x55e9aef45fd0 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7e0a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45130 0x55e9aef54000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7e1e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45fd0 0x55e9aef54f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7e1e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45fd0 0x55e9aef54f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:36684/0 (socket says 172.21.15.99:36684) 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7e1e00700 1 -- 172.21.15.99:0/2227244790 learned_addr learned my addr 172.21.15.99:0/2227244790 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7f79ebb80 10 client.0 _ll_register_callbacks cb 0x7fd7cc099d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:00.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.246+0000 7fd7e1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef53ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48776/0 (socket says 172.21.15.99:48776) 2024-03-30T00:37:00.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.247+0000 7fd7e1400700 1 -- 172.21.15.99:0/2227244790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45fd0 msgr2=0x55e9aef54f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.247+0000 7fd7e1400700 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45fd0 0x55e9aef54f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.247+0000 7fd7e1400700 1 -- 172.21.15.99:0/2227244790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45130 msgr2=0x55e9aef54000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.247+0000 7fd7e1400700 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45130 0x55e9aef54000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.247+0000 7fd7e1400700 1 -- 172.21.15.99:0/2227244790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9aecc3b60 con 0x55e9aef447d0 2024-03-30T00:37:00.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.247+0000 7fd7e0a00700 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45130 0x55e9aef54000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:00.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.247+0000 7fd7e1e00700 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45fd0 0x55e9aef54f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:00.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.248+0000 7fd7e1400700 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef53ac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0964b0 tx=0x7fd7cc0b8220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af36e9f3bf9a3d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.248+0000 7fd7bea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:37:00.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.248+0000 7fd7bea00700 1 -- 172.21.15.99:0/2227244790 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc07b070 con 0x55e9aef447d0 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.248+0000 7fd7bea00700 1 -- 172.21.15.99:0/2227244790 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fd7cc0b2040 con 0x55e9aef447d0 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.248+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2227244790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7cc099fb0 con 0x55e9aef447d0 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.248+0000 7fd7bea00700 1 -- 172.21.15.99:0/2227244790 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7cc0b8040 con 0x55e9aef447d0 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.248+0000 7fd7f79ebb80 10 client.25739 fetch_fsmap learned FSMap version 25 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.248+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2227244790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fd7dc110110 con 0x55e9aef447d0 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.248+0000 7fd7bea00700 1 -- 172.21.15.99:0/2227244790 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7cc099fb0 con 0x55e9aef447d0 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.248+0000 7fd7bea00700 1 -- 172.21.15.99:0/2227244790 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd7cc0a1fa0 con 0x55e9aef447d0 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.249+0000 7fd7f79ebb80 10 client.25739 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.249+0000 7fd7f79ebb80 10 client.25739 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.249+0000 7fd7f79ebb80 20 client.25739 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.249+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55e9aef5a120 0x55e9aefb08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.249+0000 7fd7f79ebb80 4 client.25739 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:37:00.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.249+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2227244790 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e9aece4cf0 con 0x55e9aef5a120 2024-03-30T00:37:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.249+0000 7fd7e0a00700 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55e9aef5a120 0x55e9aefb08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.250+0000 7fd7e0a00700 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55e9aef5a120 0x55e9aefb08d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3fe6bdc38c0cfa7 server_cookie=200f50799a9c13ce in_seq=0 out_seq=0 2024-03-30T00:37:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.250+0000 7fd7bea00700 10 client.25739 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:37:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.251+0000 7fd7bea00700 1 -- 172.21.15.99:0/2227244790 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e9aece4cf0 con 0x55e9aef5a120 2024-03-30T00:37:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.251+0000 7fd7bea00700 10 client.25739 handle_command_reply: tid=0 2024-03-30T00:37:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.251+0000 7fd7f79ebb80 1 client.25739 shutdown 2024-03-30T00:37:00.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.252+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2227244790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 msgr2=0x55e9aef53ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.252+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef53ac0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0964b0 tx=0x7fd7cc0b8220 comp rx=0 tx=0).stop 2024-03-30T00:37:00.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.253+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2227244790 shutdown_connections 2024-03-30T00:37:00.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.253+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x55e9aef5a120 0x55e9aefb08d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.253+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e9aef45fd0 0x55e9aef54f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.253+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e9aef447d0 0x55e9aef53ac0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.253+0000 7fd7f79ebb80 1 --2- 172.21.15.99:0/2227244790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9aef45130 0x55e9aef54000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.253+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2227244790 >> 172.21.15.99:0/2227244790 conn(0x55e9aeea36a0 msgr2=0x55e9aef60ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.253+0000 7fd7bea00700 0 client.25739 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:37:00.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.253+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2227244790 shutdown_connections 2024-03-30T00:37:00.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.253+0000 7fd7f79ebb80 1 -- 172.21.15.99:0/2227244790 wait complete. 2024-03-30T00:37:00.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.254+0000 7fd7f79ebb80 20 client.25739 trim_cache size 0 max 16384 2024-03-30T00:37:00.262 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:37:00.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.255+0000 7fd7e3e00700 1 -- 172.21.15.99:0/1570771318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c40a11a0 msgr2=0x7fd7c40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.255+0000 7fd7e3e00700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c40a11a0 0x7fd7c40a3660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000b580 tx=0x7fd7d001a040 comp rx=0 tx=0).stop 2024-03-30T00:37:00.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.255+0000 7fd7e3e00700 1 -- 172.21.15.99:0/1570771318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7dc107380 msgr2=0x7fd7dc19d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.255+0000 7fd7e3e00700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7dc107380 0x7fd7dc19d260 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00e970 tx=0x7fd7cc00ed30 comp rx=0 tx=0).stop 2024-03-30T00:37:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.256+0000 7fd7e3e00700 1 -- 172.21.15.99:0/1570771318 shutdown_connections 2024-03-30T00:37:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.256+0000 7fd7e3e00700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fd7c40a11a0 0x7fd7c40a3660 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.256+0000 7fd7e3e00700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7dc108f10 0x7fd7dc1a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.256+0000 7fd7e3e00700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7dc107380 0x7fd7dc19d260 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.256+0000 7fd7e3e00700 1 --2- 172.21.15.99:0/1570771318 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc108570 0x7fd7dc19d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.256+0000 7fd7e3e00700 1 -- 172.21.15.99:0/1570771318 >> 172.21.15.99:0/1570771318 conn(0x7fd7dc076320 msgr2=0x7fd7dc10b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.256+0000 7fd7e3e00700 1 -- 172.21.15.99:0/1570771318 shutdown_connections 2024-03-30T00:37:00.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.256+0000 7fd7e3e00700 1 -- 172.21.15.99:0/1570771318 wait complete. 2024-03-30T00:37:00.288 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T00:37:00.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.604+0000 7f1001e00700 1 -- 172.21.15.99:0/3148098629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ffc102910 msgr2=0x7f0ffc1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.604+0000 7f1001e00700 1 --2- 172.21.15.99:0/3148098629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ffc102910 0x7f0ffc1091c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009fc0 tx=0x7f0ff400b330 comp rx=0 tx=0).stop 2024-03-30T00:37:00.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.605+0000 7f1001e00700 1 -- 172.21.15.99:0/3148098629 shutdown_connections 2024-03-30T00:37:00.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.605+0000 7f1001e00700 1 --2- 172.21.15.99:0/3148098629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ffc102910 0x7f0ffc1091c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.605+0000 7f1001e00700 1 --2- 172.21.15.99:0/3148098629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ffc100d80 0x7f0ffc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.605+0000 7f1001e00700 1 --2- 172.21.15.99:0/3148098629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc101f70 0x7f0ffc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.605+0000 7f1001e00700 1 -- 172.21.15.99:0/3148098629 >> 172.21.15.99:0/3148098629 conn(0x7f0ffc0fc2f0 msgr2=0x7f0ffc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.606+0000 7f1001e00700 1 -- 172.21.15.99:0/3148098629 shutdown_connections 2024-03-30T00:37:00.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.606+0000 7f1001e00700 1 -- 172.21.15.99:0/3148098629 wait complete. 2024-03-30T00:37:00.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.607+0000 7f1001e00700 1 Processor -- start 2024-03-30T00:37:00.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.607+0000 7f1001e00700 1 -- start start 2024-03-30T00:37:00.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f1001e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ffc100d80 0x7f0ffc074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f1001e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ffc101f70 0x7f0ffc0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f1001e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc102910 0x7f0ffc0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f1001e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc073e80 con 0x7f0ffc100d80 2024-03-30T00:37:00.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f1001e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc073ff0 con 0x7f0ffc102910 2024-03-30T00:37:00.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f1001e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc1a0d90 con 0x7f0ffc101f70 2024-03-30T00:37:00.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f0ffb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc102910 0x7f0ffc0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f0ffb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc102910 0x7f0ffc0736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48794/0 (socket says 172.21.15.99:48794) 2024-03-30T00:37:00.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f0ffb400700 1 -- 172.21.15.99:0/1023842919 learned_addr learned my addr 172.21.15.99:0/1023842919 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f0ffaa00700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ffc100d80 0x7f0ffc074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f0ffa000700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ffc101f70 0x7f0ffc0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f0ffb400700 1 -- 172.21.15.99:0/1023842919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ffc101f70 msgr2=0x7f0ffc0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f0ffb400700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ffc101f70 0x7f0ffc0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f0ffb400700 1 -- 172.21.15.99:0/1023842919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ffc100d80 msgr2=0x7f0ffc074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f0ffb400700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ffc100d80 0x7f0ffc074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.608+0000 7f0ffb400700 1 -- 172.21.15.99:0/1023842919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe80095f0 con 0x7f0ffc102910 2024-03-30T00:37:00.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.609+0000 7f0ffb400700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc102910 0x7f0ffc0736e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4007a80 tx=0x7f0ff4007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.609+0000 7f1000a00700 1 -- 172.21.15.99:0/1023842919 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff400e040 con 0x7f0ffc102910 2024-03-30T00:37:00.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.609+0000 7f1000a00700 1 -- 172.21.15.99:0/1023842919 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ff4009350 con 0x7f0ffc102910 2024-03-30T00:37:00.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.609+0000 7f1001e00700 1 -- 172.21.15.99:0/1023842919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff4009c40 con 0x7f0ffc102910 2024-03-30T00:37:00.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.609+0000 7f1000a00700 1 -- 172.21.15.99:0/1023842919 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff401b6b0 con 0x7f0ffc102910 2024-03-30T00:37:00.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.610+0000 7f1001e00700 1 -- 172.21.15.99:0/1023842919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ffc1a11f0 con 0x7f0ffc102910 2024-03-30T00:37:00.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.613+0000 7f1000a00700 1 -- 172.21.15.99:0/1023842919 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f0ff401d9a0 con 0x7f0ffc102910 2024-03-30T00:37:00.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.616+0000 7f10158beb80 1 Processor -- start 2024-03-30T00:37:00.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.616+0000 7f1000a00700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0fe00a11a0 0x7f0fe00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.616+0000 7f0ffa000700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0fe00a11a0 0x7f0fe00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.616+0000 7f10158beb80 1 -- start start 2024-03-30T00:37:00.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.617+0000 7f10158beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.617+0000 7f10158beb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.617+0000 7f10158beb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9edc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.617+0000 7f10158beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e93b990 con 0x55ae6e9ebf80 2024-03-30T00:37:00.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.617+0000 7f10158beb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e939350 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.617+0000 7f10158beb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e934c40 con 0x55ae6e9ed7d0 2024-03-30T00:37:00.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.617+0000 7f0ffaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.617+0000 7f0ffaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9ec3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44198/0 (socket says 172.21.15.99:44198) 2024-03-30T00:37:00.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.617+0000 7f0ffaa00700 1 -- 172.21.15.99:0/2513424848 learned_addr learned my addr 172.21.15.99:0/2513424848 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.617+0000 7f1000a00700 1 -- 172.21.15.99:0/1023842919 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f0ff40ce200 con 0x7f0ffc102910 2024-03-30T00:37:00.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.618+0000 7f0ffaa00700 1 -- 172.21.15.99:0/2513424848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 msgr2=0x55ae6e9edc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.618+0000 7f0ffb400700 1 --2- 172.21.15.99:0/2513424848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.618+0000 7f0ffaa00700 1 --2- 172.21.15.99:0/2513424848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9edc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.618+0000 7f0ffaa00700 1 -- 172.21.15.99:0/2513424848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 msgr2=0x55ae6e9ed1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:37:00.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.618+0000 7f0ffaa00700 1 --2- 172.21.15.99:0/2513424848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9ed1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.618+0000 7f0ffaa00700 1 -- 172.21.15.99:0/2513424848 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6e7b63b0 con 0x55ae6e9ebf80 2024-03-30T00:37:00.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.618+0000 7f0ffb400700 1 --2- 172.21.15.99:0/2513424848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9edc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:37:00.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.618+0000 7f0ffaa00700 1 --2- 172.21.15.99:0/2513424848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9ec3a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009d30 tx=0x7f0fec011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54a8b916cfb89e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.618+0000 7f0ffa000700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0fe00a11a0 0x7f0fe00a3660 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8005fd0 tx=0x7f0fe8009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.618+0000 7f0fe7e00700 1 -- 172.21.15.99:0/2513424848 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec004220 con 0x55ae6e9ebf80 2024-03-30T00:37:00.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.619+0000 7f0fe7e00700 1 -- 172.21.15.99:0/2513424848 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fec004380 con 0x55ae6e9ebf80 2024-03-30T00:37:00.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.619+0000 7f0fe7e00700 1 -- 172.21.15.99:0/2513424848 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0197a0 con 0x55ae6e9ebf80 2024-03-30T00:37:00.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.619+0000 7f10158beb80 1 -- 172.21.15.99:0/2513424848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 msgr2=0x55ae6e9ec3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.619+0000 7f10158beb80 1 --2- 172.21.15.99:0/2513424848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9ec3a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009d30 tx=0x7f0fec011720 comp rx=0 tx=0).stop 2024-03-30T00:37:00.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.620+0000 7f10158beb80 1 -- 172.21.15.99:0/2513424848 shutdown_connections 2024-03-30T00:37:00.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.620+0000 7f10158beb80 1 --2- 172.21.15.99:0/2513424848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9edc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.620+0000 7f10158beb80 1 --2- 172.21.15.99:0/2513424848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9ec3a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.620+0000 7f10158beb80 1 --2- 172.21.15.99:0/2513424848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9ed1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.620+0000 7f10158beb80 1 -- 172.21.15.99:0/2513424848 >> 172.21.15.99:0/2513424848 conn(0x55ae6e9426a0 msgr2=0x55ae6e9de880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.621+0000 7f10158beb80 1 -- 172.21.15.99:0/2513424848 shutdown_connections 2024-03-30T00:37:00.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.621+0000 7f10158beb80 1 -- 172.21.15.99:0/2513424848 wait complete. 2024-03-30T00:37:00.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.621+0000 7f10158beb80 1 -- start start 2024-03-30T00:37:00.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.622+0000 7f10158beb80 1 Processor -- start 2024-03-30T00:37:00.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.622+0000 7f10158beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.622+0000 7f10158beb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.622+0000 7f10158beb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.622+0000 7f10158beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e8d0d80 con 0x55ae6e9ebf80 2024-03-30T00:37:00.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.622+0000 7f10158beb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e8d0a20 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.622+0000 7f10158beb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e7b86a0 con 0x55ae6e9ed7d0 2024-03-30T00:37:00.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.622+0000 7f10158beb80 10 client.0 _ll_register_callbacks cb 0x55ae6e8da470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:00.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.623+0000 7f0ffaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.623+0000 7f0ffaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44212/0 (socket says 172.21.15.99:44212) 2024-03-30T00:37:00.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.623+0000 7f0ffaa00700 1 -- 172.21.15.99:0/3858609704 learned_addr learned my addr 172.21.15.99:0/3858609704 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.623+0000 7f0ffa000700 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.623+0000 7f0ffb400700 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.623+0000 7f0ffaa00700 1 -- 172.21.15.99:0/3858609704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 msgr2=0x55ae6e9fd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.623+0000 7f0ffaa00700 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9fd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.623+0000 7f0ffaa00700 1 -- 172.21.15.99:0/3858609704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 msgr2=0x55ae6e9f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.623+0000 7f0ffaa00700 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9f63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.623+0000 7f0ffaa00700 1 -- 172.21.15.99:0/3858609704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec009960 con 0x55ae6e9ebf80 2024-03-30T00:37:00.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.624+0000 7f0ffaa00700 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9f5e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0fec000c00 tx=0x7f0fec005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929a2ddbc60b3784 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.624+0000 7f0fe4c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:37:00.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.625+0000 7f0fe4c00700 1 -- 172.21.15.99:0/3858609704 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec005040 con 0x55ae6e9ebf80 2024-03-30T00:37:00.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.625+0000 7f0fe4c00700 1 -- 172.21.15.99:0/3858609704 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f0fec0051a0 con 0x55ae6e9ebf80 2024-03-30T00:37:00.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.625+0000 7f10158beb80 1 -- 172.21.15.99:0/3858609704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae6e8f5380 con 0x55ae6e9ebf80 2024-03-30T00:37:00.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.625+0000 7f0fe4c00700 1 -- 172.21.15.99:0/3858609704 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fec005360 con 0x55ae6e9ebf80 2024-03-30T00:37:00.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.626+0000 7f10158beb80 10 client.16056 fetch_fsmap learned FSMap version 25 2024-03-30T00:37:00.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.626+0000 7f10158beb80 1 -- 172.21.15.99:0/3858609704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ae6e762b60 con 0x55ae6e9ebf80 2024-03-30T00:37:00.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.626+0000 7f0fe4c00700 1 -- 172.21.15.99:0/3858609704 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ae6e8f5380 con 0x55ae6e9ebf80 2024-03-30T00:37:00.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.626+0000 7f0fe4c00700 1 -- 172.21.15.99:0/3858609704 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0fec021040 con 0x55ae6e9ebf80 2024-03-30T00:37:00.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.626+0000 7f10158beb80 10 client.16056 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:37:00.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.626+0000 7f10158beb80 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:00.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.626+0000 7f10158beb80 20 client.16056 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:00.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.626+0000 7f10158beb80 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ae6ea01b00 0x55ae6ea03f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.626+0000 7f10158beb80 4 client.16056 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:37:00.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.626+0000 7f10158beb80 1 -- 172.21.15.99:0/3858609704 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae6e92d700 con 0x55ae6ea01b00 2024-03-30T00:37:00.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.627+0000 7f0ffa000700 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ae6ea01b00 0x55ae6ea03f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.627+0000 7f0ffa000700 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ae6ea01b00 0x55ae6ea03f20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce22e8657320eb9 server_cookie=55bfe6937c777fb1 in_seq=0 out_seq=0 2024-03-30T00:37:00.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.627+0000 7f0fe4c00700 10 client.16056 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:00.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.628+0000 7f0fe4c00700 1 -- 172.21.15.99:0/3858609704 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae6e92d700 con 0x55ae6ea01b00 2024-03-30T00:37:00.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.628+0000 7f0fe4c00700 10 client.16056 handle_command_reply: tid=0 2024-03-30T00:37:00.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.628+0000 7f10158beb80 1 client.16056 shutdown 2024-03-30T00:37:00.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.629+0000 7f10158beb80 1 -- 172.21.15.99:0/3858609704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 msgr2=0x55ae6e9f5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.629+0000 7f10158beb80 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9f5e60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0fec000c00 tx=0x7f0fec005d50 comp rx=0 tx=0).stop 2024-03-30T00:37:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.629+0000 7f10158beb80 1 -- 172.21.15.99:0/3858609704 shutdown_connections 2024-03-30T00:37:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.629+0000 7f10158beb80 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ae6ea01b00 0x55ae6ea03f20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.629+0000 7f10158beb80 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.629+0000 7f10158beb80 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9f5e60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.629+0000 7f10158beb80 1 --2- 172.21.15.99:0/3858609704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.629+0000 7f10158beb80 1 -- 172.21.15.99:0/3858609704 >> 172.21.15.99:0/3858609704 conn(0x55ae6e9426a0 msgr2=0x55ae6e9ea980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.629+0000 7f0fe4c00700 0 client.16056 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.629+0000 7f10158beb80 1 -- 172.21.15.99:0/3858609704 shutdown_connections 2024-03-30T00:37:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.629+0000 7f10158beb80 1 -- 172.21.15.99:0/3858609704 wait complete. 2024-03-30T00:37:00.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.630+0000 7f10158beb80 20 client.16056 trim_cache size 0 max 16384 2024-03-30T00:37:00.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.651+0000 7f10158beb80 1 Processor -- start 2024-03-30T00:37:00.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.651+0000 7f10158beb80 1 -- start start 2024-03-30T00:37:00.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.651+0000 7f10158beb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9fda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.651+0000 7f10158beb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.651+0000 7f10158beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9d7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.651+0000 7f10158beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e9df410 con 0x55ae6e9ed7d0 2024-03-30T00:37:00.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.651+0000 7f10158beb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e9ecfe0 con 0x55ae6e9ebf80 2024-03-30T00:37:00.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.651+0000 7f10158beb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e9f3960 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.652+0000 7f0ffa000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.652+0000 7f0ffa000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:36716/0 (socket says 172.21.15.99:36716) 2024-03-30T00:37:00.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.652+0000 7f0ffa000700 1 -- 172.21.15.99:0/2323942087 learned_addr learned my addr 172.21.15.99:0/2323942087 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.652+0000 7f0ffaa00700 1 --2- 172.21.15.99:0/2323942087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9fda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.652+0000 7f0ffb400700 1 --2- 172.21.15.99:0/2323942087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9d7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.652+0000 7f0ffa000700 1 -- 172.21.15.99:0/2323942087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ebf80 msgr2=0x55ae6e9fda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.652+0000 7f0ffa000700 1 --2- 172.21.15.99:0/2323942087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9fda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.652+0000 7f0ffa000700 1 -- 172.21.15.99:0/2323942087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ed7d0 msgr2=0x55ae6e9d7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.652+0000 7f0ffa000700 1 --2- 172.21.15.99:0/2323942087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9d7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.652+0000 7f0ffa000700 1 -- 172.21.15.99:0/2323942087 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6e8faea0 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.652+0000 7f0ffb400700 1 --2- 172.21.15.99:0/2323942087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9d7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:00.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.653+0000 7f0ffa000700 1 --2- 172.21.15.99:0/2323942087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9fd3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8012480 tx=0x7f0fe8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dae32f3e06dbd2ef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.653+0000 7f0ff1200700 1 -- 172.21.15.99:0/2323942087 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8026070 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.653+0000 7f0ff1200700 1 -- 172.21.15.99:0/2323942087 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fe8016070 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.653+0000 7f0ff1200700 1 -- 172.21.15.99:0/2323942087 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8026de0 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.654+0000 7f10158beb80 1 -- 172.21.15.99:0/2323942087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ec8e0 msgr2=0x55ae6e9fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.654+0000 7f10158beb80 1 --2- 172.21.15.99:0/2323942087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9fd3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8012480 tx=0x7f0fe8014000 comp rx=0 tx=0).stop 2024-03-30T00:37:00.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.654+0000 7f10158beb80 1 -- 172.21.15.99:0/2323942087 shutdown_connections 2024-03-30T00:37:00.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.654+0000 7f10158beb80 1 --2- 172.21.15.99:0/2323942087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9d7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.654+0000 7f10158beb80 1 --2- 172.21.15.99:0/2323942087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9fda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.654+0000 7f10158beb80 1 --2- 172.21.15.99:0/2323942087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9fd3a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.654+0000 7f10158beb80 1 -- 172.21.15.99:0/2323942087 >> 172.21.15.99:0/2323942087 conn(0x55ae6e9426a0 msgr2=0x55ae6e9d75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.655+0000 7f10158beb80 1 -- 172.21.15.99:0/2323942087 shutdown_connections 2024-03-30T00:37:00.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.655+0000 7f10158beb80 1 -- 172.21.15.99:0/2323942087 wait complete. 2024-03-30T00:37:00.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.655+0000 7f10158beb80 1 -- start start 2024-03-30T00:37:00.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.656+0000 7f10158beb80 1 Processor -- start 2024-03-30T00:37:00.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f10158beb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9facb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f10158beb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9fb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f10158beb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9fc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f10158beb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e9fc650 con 0x55ae6e9ebf80 2024-03-30T00:37:00.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f10158beb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e9fc7c0 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f10158beb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6e9fc980 con 0x55ae6e9ed7d0 2024-03-30T00:37:00.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f10158beb80 10 client.0 _ll_register_callbacks cb 0x55ae6e8da470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:00.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f0ffb400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f0ffb400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:36726/0 (socket says 172.21.15.99:36726) 2024-03-30T00:37:00.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f0ffb400700 1 -- 172.21.15.99:0/1786187630 learned_addr learned my addr 172.21.15.99:0/1786187630 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:00.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f0ffa000700 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9fb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.657+0000 7f0ffaa00700 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.658+0000 7f0ffa000700 1 -- 172.21.15.99:0/1786187630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 msgr2=0x55ae6e9fc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.658+0000 7f0ffa000700 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9fc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.658+0000 7f0ffa000700 1 -- 172.21.15.99:0/1786187630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 msgr2=0x55ae6e9facb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:00.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.658+0000 7f0ffa000700 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9facb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.658+0000 7f0ffa000700 1 -- 172.21.15.99:0/1786187630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe8011d10 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.658+0000 7f0ffa000700 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9fb1f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8012450 tx=0x7f0fe8027c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=973b65719c16cdfa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:00.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.658+0000 7f0fe4c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:37:00.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.659+0000 7f0fe4c00700 1 -- 172.21.15.99:0/1786187630 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8012e10 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.659+0000 7f0fe4c00700 1 -- 172.21.15.99:0/1786187630 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f0fe8018070 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.659+0000 7f10158beb80 1 -- 172.21.15.99:0/1786187630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fec004380 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.659+0000 7f0fe4c00700 1 -- 172.21.15.99:0/1786187630 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe8026290 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.659+0000 7f10158beb80 10 client.25754 fetch_fsmap learned FSMap version 25 2024-03-30T00:37:00.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.659+0000 7f10158beb80 1 -- 172.21.15.99:0/1786187630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ae6e7bacd0 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.660+0000 7f0fe4c00700 1 -- 172.21.15.99:0/1786187630 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fec004380 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.660+0000 7f0fe4c00700 1 -- 172.21.15.99:0/1786187630 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0fe8036020 con 0x55ae6e9ec8e0 2024-03-30T00:37:00.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.660+0000 7f10158beb80 10 client.25754 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:37:00.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.660+0000 7f10158beb80 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:00.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.660+0000 7f10158beb80 20 client.25754 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:00.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.660+0000 7f10158beb80 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ae6ea56270 0x55ae6ea58690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:00.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.660+0000 7f10158beb80 4 client.25754 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:37:00.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.660+0000 7f10158beb80 1 -- 172.21.15.99:0/1786187630 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae6e783cf0 con 0x55ae6ea56270 2024-03-30T00:37:00.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.661+0000 7f0ffaa00700 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ae6ea56270 0x55ae6ea58690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:00.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.662+0000 7f0ffaa00700 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ae6ea56270 0x55ae6ea58690 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b082a7a0618e5824 server_cookie=44405fceaba3a9cc in_seq=0 out_seq=0 2024-03-30T00:37:00.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.662+0000 7f0fe4c00700 10 client.25754 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:00.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.662+0000 7f0fe4c00700 1 -- 172.21.15.99:0/1786187630 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae6e783cf0 con 0x55ae6ea56270 2024-03-30T00:37:00.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.662+0000 7f0fe4c00700 10 client.25754 handle_command_reply: tid=0 2024-03-30T00:37:00.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.662+0000 7f10158beb80 1 client.25754 shutdown 2024-03-30T00:37:00.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.663+0000 7f10158beb80 1 -- 172.21.15.99:0/1786187630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 msgr2=0x55ae6e9fb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.663+0000 7f10158beb80 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9fb1f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8012450 tx=0x7f0fe8027c80 comp rx=0 tx=0).stop 2024-03-30T00:37:00.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.663+0000 7f10158beb80 1 -- 172.21.15.99:0/1786187630 shutdown_connections 2024-03-30T00:37:00.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.663+0000 7f10158beb80 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55ae6ea56270 0x55ae6ea58690 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.663+0000 7f10158beb80 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae6e9ed7d0 0x55ae6e9fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.663+0000 7f10158beb80 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae6e9ebf80 0x55ae6e9facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.663+0000 7f10158beb80 1 --2- 172.21.15.99:0/1786187630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ae6e9ec8e0 0x55ae6e9fb1f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.663+0000 7f10158beb80 1 -- 172.21.15.99:0/1786187630 >> 172.21.15.99:0/1786187630 conn(0x55ae6e9426a0 msgr2=0x55ae6e9d7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.663+0000 7f0fe4c00700 0 client.25754 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:00.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.663+0000 7f10158beb80 1 -- 172.21.15.99:0/1786187630 shutdown_connections 2024-03-30T00:37:00.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.664+0000 7f10158beb80 1 -- 172.21.15.99:0/1786187630 wait complete. 2024-03-30T00:37:00.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.664+0000 7f10158beb80 20 client.25754 trim_cache size 0 max 16384 2024-03-30T00:37:00.672 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:37:00.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.664+0000 7f1001e00700 1 -- 172.21.15.99:0/1023842919 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0fe00a11a0 msgr2=0x7f0fe00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.664+0000 7f1001e00700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0fe00a11a0 0x7f0fe00a3660 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8005fd0 tx=0x7f0fe8009310 comp rx=0 tx=0).stop 2024-03-30T00:37:00.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.664+0000 7f1001e00700 1 -- 172.21.15.99:0/1023842919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc102910 msgr2=0x7f0ffc0736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:00.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.664+0000 7f1001e00700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc102910 0x7f0ffc0736e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4007a80 tx=0x7f0ff4007d50 comp rx=0 tx=0).stop 2024-03-30T00:37:00.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.665+0000 7f1001e00700 1 -- 172.21.15.99:0/1023842919 shutdown_connections 2024-03-30T00:37:00.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.665+0000 7f1001e00700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f0fe00a11a0 0x7f0fe00a3660 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.665+0000 7f1001e00700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc102910 0x7f0ffc0736e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.665+0000 7f1001e00700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ffc100d80 0x7f0ffc074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.665+0000 7f1001e00700 1 --2- 172.21.15.99:0/1023842919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ffc101f70 0x7f0ffc0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:00.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.665+0000 7f1001e00700 1 -- 172.21.15.99:0/1023842919 >> 172.21.15.99:0/1023842919 conn(0x7f0ffc0fc2f0 msgr2=0x7f0ffc105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:00.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.665+0000 7f1001e00700 1 -- 172.21.15.99:0/1023842919 shutdown_connections 2024-03-30T00:37:00.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:00.665+0000 7f1001e00700 1 -- 172.21.15.99:0/1023842919 wait complete. 2024-03-30T00:37:00.694 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:00 smithi099 ceph-mon[38079]: pgmap v372: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 14 MiB/s wr, 3.01k op/s 2024-03-30T00:37:00.694 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:00 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/698046900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:37:00.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:00 smithi161 ceph-mon[42221]: pgmap v372: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 14 MiB/s wr, 3.01k op/s 2024-03-30T00:37:00.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:00 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/698046900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:37:00.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:00 smithi142 ceph-mon[41779]: pgmap v372: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 14 MiB/s wr, 3.01k op/s 2024-03-30T00:37:00.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:00 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/698046900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:37:01.697 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T00:37:01.698 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T00:37:01.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.971+0000 7fc360e00700 1 -- 172.21.15.99:0/943522503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc35c104110 msgr2=0x7fc35c106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:01.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.971+0000 7fc360e00700 1 --2- 172.21.15.99:0/943522503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc35c104110 0x7fc35c106500 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009990 tx=0x7fc34c009ca0 comp rx=0 tx=0).stop 2024-03-30T00:37:01.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.971+0000 7fc360e00700 1 -- 172.21.15.99:0/943522503 shutdown_connections 2024-03-30T00:37:01.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.971+0000 7fc360e00700 1 --2- 172.21.15.99:0/943522503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c106a40 0x7fc35c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:01.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.971+0000 7fc360e00700 1 --2- 172.21.15.99:0/943522503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c1017e0 0x7fc35c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:01.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.971+0000 7fc360e00700 1 --2- 172.21.15.99:0/943522503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc35c104110 0x7fc35c106500 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:01.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.971+0000 7fc360e00700 1 -- 172.21.15.99:0/943522503 >> 172.21.15.99:0/943522503 conn(0x7fc35c0fb1d0 msgr2=0x7fc35c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:01.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.971+0000 7fc360e00700 1 -- 172.21.15.99:0/943522503 shutdown_connections 2024-03-30T00:37:01.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.971+0000 7fc360e00700 1 -- 172.21.15.99:0/943522503 wait complete. 2024-03-30T00:37:01.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc360e00700 1 Processor -- start 2024-03-30T00:37:01.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc360e00700 1 -- start start 2024-03-30T00:37:01.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc360e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c1017e0 0x7fc35c196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:01.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc360e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc35c104110 0x7fc35c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:01.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc360e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c106a40 0x7fc35c19e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:01.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc360e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c19e7c0 con 0x7fc35c1017e0 2024-03-30T00:37:01.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc360e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c19e930 con 0x7fc35c104110 2024-03-30T00:37:01.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc360e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c19eaf0 con 0x7fc35c106a40 2024-03-30T00:37:01.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc359e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c1017e0 0x7fc35c196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:01.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc359400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc35c104110 0x7fc35c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:01.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc359e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c1017e0 0x7fc35c196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44246/0 (socket says 172.21.15.99:44246) 2024-03-30T00:37:01.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc359e00700 1 -- 172.21.15.99:0/690471804 learned_addr learned my addr 172.21.15.99:0/690471804 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:01.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.973+0000 7fc35a800700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c106a40 0x7fc35c19e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:01.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.974+0000 7fc359400700 1 -- 172.21.15.99:0/690471804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c106a40 msgr2=0x7fc35c19e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:01.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.974+0000 7fc359400700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c106a40 0x7fc35c19e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:01.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.974+0000 7fc359400700 1 -- 172.21.15.99:0/690471804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c1017e0 msgr2=0x7fc35c196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:01.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.974+0000 7fc359400700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c1017e0 0x7fc35c196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:01.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.974+0000 7fc359400700 1 -- 172.21.15.99:0/690471804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c0095f0 con 0x7fc35c104110 2024-03-30T00:37:01.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.974+0000 7fc359400700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc35c104110 0x7fc35c197210 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc34c006010 tx=0x7fc34c003980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:01.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.975+0000 7fc352a00700 1 -- 172.21.15.99:0/690471804 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c01d070 con 0x7fc35c104110 2024-03-30T00:37:01.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.975+0000 7fc352a00700 1 -- 172.21.15.99:0/690471804 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc34c00be50 con 0x7fc35c104110 2024-03-30T00:37:01.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.975+0000 7fc352a00700 1 -- 172.21.15.99:0/690471804 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c022e20 con 0x7fc35c104110 2024-03-30T00:37:01.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.975+0000 7fc360e00700 1 -- 172.21.15.99:0/690471804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c19ed70 con 0x7fc35c104110 2024-03-30T00:37:01.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.975+0000 7fc360e00700 1 -- 172.21.15.99:0/690471804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c19f140 con 0x7fc35c104110 2024-03-30T00:37:01.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.978+0000 7fc352a00700 1 -- 172.21.15.99:0/690471804 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc34c00f460 con 0x7fc35c104110 2024-03-30T00:37:01.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.981+0000 7fc3749e6b80 1 Processor -- start 2024-03-30T00:37:01.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.981+0000 7fc3749e6b80 1 -- start start 2024-03-30T00:37:01.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.981+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557275388850 0x557275388c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:01.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.981+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x557275389a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:01.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.981+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55727538a050 0x55727538a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:01.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.981+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572752e0a10 con 0x5572753891b0 2024-03-30T00:37:01.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.981+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572752de3d0 con 0x557275388850 2024-03-30T00:37:01.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.981+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572752d9cc0 con 0x55727538a050 2024-03-30T00:37:01.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.981+0000 7fc352a00700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3400a1120 0x7fc3400a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:01.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.982+0000 7fc359e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x557275389a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:01.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.982+0000 7fc359e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x557275389a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44252/0 (socket says 172.21.15.99:44252) 2024-03-30T00:37:01.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.982+0000 7fc359e00700 1 -- 172.21.15.99:0/3753769579 learned_addr learned my addr 172.21.15.99:0/3753769579 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:01.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.982+0000 7fc352a00700 1 -- 172.21.15.99:0/690471804 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fc34c0cde00 con 0x7fc35c104110 2024-03-30T00:37:01.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.982+0000 7fc359400700 1 --2- 172.21.15.99:0/3753769579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55727538a050 0x55727538a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:01.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.982+0000 7fc359e00700 1 -- 172.21.15.99:0/3753769579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55727538a050 msgr2=0x55727538a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:01.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.982+0000 7fc359e00700 1 --2- 172.21.15.99:0/3753769579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55727538a050 0x55727538a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:01.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.982+0000 7fc359e00700 1 -- 172.21.15.99:0/3753769579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557275388850 msgr2=0x557275388c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T00:37:01.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.982+0000 7fc35a800700 1 --2- 172.21.15.99:0/3753769579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557275388850 0x557275388c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:01.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.982+0000 7fc359e00700 1 --2- 172.21.15.99:0/3753769579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557275388850 0x557275388c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:01.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.982+0000 7fc359e00700 1 -- 172.21.15.99:0/3753769579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55727515b430 con 0x5572753891b0 2024-03-30T00:37:01.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.983+0000 7fc35a800700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3400a1120 0x7fc3400a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:01.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.983+0000 7fc359e00700 1 --2- 172.21.15.99:0/3753769579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x557275389a70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc348009b80 tx=0x7fc348009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa3df9f8ed0b35d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:01.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.983+0000 7fc347400700 1 -- 172.21.15.99:0/3753769579 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34800e990 con 0x5572753891b0 2024-03-30T00:37:01.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.983+0000 7fc347400700 1 -- 172.21.15.99:0/3753769579 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc348013470 con 0x5572753891b0 2024-03-30T00:37:01.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.983+0000 7fc347400700 1 -- 172.21.15.99:0/3753769579 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348012630 con 0x5572753891b0 2024-03-30T00:37:02.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.984+0000 7fc3749e6b80 1 -- 172.21.15.99:0/3753769579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 msgr2=0x557275389a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.984+0000 7fc35a800700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3400a1120 0x7fc3400a35e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc354009ec0 tx=0x7fc354004ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:02.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.984+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/3753769579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x557275389a70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc348009b80 tx=0x7fc348009e90 comp rx=0 tx=0).stop 2024-03-30T00:37:02.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.984+0000 7fc3749e6b80 1 -- 172.21.15.99:0/3753769579 shutdown_connections 2024-03-30T00:37:02.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.985+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/3753769579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55727538a050 0x55727538a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.985+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/3753769579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557275388850 0x557275388c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.985+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/3753769579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x557275389a70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.985+0000 7fc3749e6b80 1 -- 172.21.15.99:0/3753769579 >> 172.21.15.99:0/3753769579 conn(0x5572752e7720 msgr2=0x557275383a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:02.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.985+0000 7fc3749e6b80 1 -- 172.21.15.99:0/3753769579 shutdown_connections 2024-03-30T00:37:02.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.985+0000 7fc3749e6b80 1 -- 172.21.15.99:0/3753769579 wait complete. 2024-03-30T00:37:02.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.985+0000 7fc3749e6b80 1 -- start start 2024-03-30T00:37:02.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.985+0000 7fc3749e6b80 1 Processor -- start 2024-03-30T00:37:02.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.986+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557275388850 0x557275394830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.986+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x557275394d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.986+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55727538a050 0x5572753913a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.986+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x557275275e00 con 0x557275388850 2024-03-30T00:37:02.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.986+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557275275aa0 con 0x5572753891b0 2024-03-30T00:37:02.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.986+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55727515d720 con 0x55727538a050 2024-03-30T00:37:02.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.986+0000 7fc3749e6b80 10 client.0 _ll_register_callbacks cb 0x55727527f4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:02.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc359400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x557275394d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc359e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557275388850 0x557275394830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc359400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x557275394d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48872/0 (socket says 172.21.15.99:48872) 2024-03-30T00:37:02.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc359400700 1 -- 172.21.15.99:0/1061206846 learned_addr learned my addr 172.21.15.99:0/1061206846 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:02.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc35a800700 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55727538a050 0x5572753913a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc359400700 1 -- 172.21.15.99:0/1061206846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55727538a050 msgr2=0x5572753913a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc359400700 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55727538a050 0x5572753913a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc359400700 1 -- 172.21.15.99:0/1061206846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557275388850 msgr2=0x557275394830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc359400700 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557275388850 0x557275394830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc359400700 1 -- 172.21.15.99:0/1061206846 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3480097b0 con 0x5572753891b0 2024-03-30T00:37:02.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc359400700 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x557275394d70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc34c000c00 tx=0x7fc34c097670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668e30fde82dd4dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:02.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.987+0000 7fc33be00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:37:02.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc33be00700 1 -- 172.21.15.99:0/1061206846 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c022770 con 0x5572753891b0 2024-03-30T00:37:02.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc33be00700 1 -- 172.21.15.99:0/1061206846 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fc34c09b900 con 0x5572753891b0 2024-03-30T00:37:02.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc3749e6b80 1 -- 172.21.15.99:0/1061206846 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55727529a400 con 0x5572753891b0 2024-03-30T00:37:02.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc33be00700 1 -- 172.21.15.99:0/1061206846 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc34c0b3d20 con 0x5572753891b0 2024-03-30T00:37:02.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc3749e6b80 10 client.25763 fetch_fsmap learned FSMap version 25 2024-03-30T00:37:02.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc3749e6b80 1 -- 172.21.15.99:0/1061206846 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557275107be0 con 0x5572753891b0 2024-03-30T00:37:02.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc33be00700 1 -- 172.21.15.99:0/1061206846 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55727529a400 con 0x5572753891b0 2024-03-30T00:37:02.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc33be00700 1 -- 172.21.15.99:0/1061206846 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc34c0a10b0 con 0x5572753891b0 2024-03-30T00:37:02.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc3749e6b80 10 client.25763 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:37:02.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc3749e6b80 10 client.25763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:02.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc3749e6b80 20 client.25763 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:02.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5572753a6970 0x5572753a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc3749e6b80 4 client.25763 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:37:02.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.988+0000 7fc3749e6b80 1 -- 172.21.15.99:0/1061206846 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572752d2780 con 0x5572753a6970 2024-03-30T00:37:02.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.989+0000 7fc359e00700 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5572753a6970 0x5572753a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.989+0000 7fc359e00700 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5572753a6970 0x5572753a8d90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88335bcf26d5ce16 server_cookie=4a73e8c3d59ea5fa in_seq=0 out_seq=0 2024-03-30T00:37:02.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.989+0000 7fc33be00700 10 client.25763 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:02.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.990+0000 7fc33be00700 1 -- 172.21.15.99:0/1061206846 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5572752d2780 con 0x5572753a6970 2024-03-30T00:37:02.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.990+0000 7fc33be00700 10 client.25763 handle_command_reply: tid=0 2024-03-30T00:37:02.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.990+0000 7fc3749e6b80 1 client.25763 shutdown 2024-03-30T00:37:02.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.991+0000 7fc3749e6b80 1 -- 172.21.15.99:0/1061206846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 msgr2=0x557275394d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.991+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x557275394d70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc34c000c00 tx=0x7fc34c097670 comp rx=0 tx=0).stop 2024-03-30T00:37:02.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.991+0000 7fc3749e6b80 1 -- 172.21.15.99:0/1061206846 shutdown_connections 2024-03-30T00:37:02.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.991+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5572753a6970 0x5572753a8d90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.991+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55727538a050 0x5572753913a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.991+0000 7fc33be00700 0 client.25763 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:02.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.991+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557275388850 0x557275394830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.991+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/1061206846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x557275394d70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.991+0000 7fc3749e6b80 1 -- 172.21.15.99:0/1061206846 >> 172.21.15.99:0/1061206846 conn(0x5572752e7720 msgr2=0x5572753871f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:02.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.992+0000 7fc3749e6b80 1 -- 172.21.15.99:0/1061206846 shutdown_connections 2024-03-30T00:37:02.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.993+0000 7fc3749e6b80 1 -- 172.21.15.99:0/1061206846 wait complete. 2024-03-30T00:37:02.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:01.995+0000 7fc3749e6b80 20 client.25763 trim_cache size 0 max 16384 2024-03-30T00:37:02.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.024+0000 7fc3749e6b80 1 Processor -- start 2024-03-30T00:37:02.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.025+0000 7fc3749e6b80 1 -- start start 2024-03-30T00:37:02.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.025+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557275388850 0x557275396000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.025+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x55727538a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.025+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55727538a050 0x5572753b6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.025+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572753a2f50 con 0x5572753891b0 2024-03-30T00:37:02.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.025+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572752d2780 con 0x55727538a050 2024-03-30T00:37:02.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.025+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572753921d0 con 0x557275388850 2024-03-30T00:37:02.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.025+0000 7fc359400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x55727538a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.025+0000 7fc359400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x55727538a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44274/0 (socket says 172.21.15.99:44274) 2024-03-30T00:37:02.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.025+0000 7fc359400700 1 -- 172.21.15.99:0/2692457186 learned_addr learned my addr 172.21.15.99:0/2692457186 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:02.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.025+0000 7fc359e00700 1 --2- 172.21.15.99:0/2692457186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557275388850 0x557275396000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.026+0000 7fc35a800700 1 --2- 172.21.15.99:0/2692457186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55727538a050 0x5572753b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.026+0000 7fc359400700 1 -- 172.21.15.99:0/2692457186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557275388850 msgr2=0x557275396000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.026+0000 7fc359400700 1 --2- 172.21.15.99:0/2692457186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557275388850 0x557275396000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.026+0000 7fc359400700 1 -- 172.21.15.99:0/2692457186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55727538a050 msgr2=0x5572753b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.026+0000 7fc359400700 1 --2- 172.21.15.99:0/2692457186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55727538a050 0x5572753b6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.026+0000 7fc359400700 1 -- 172.21.15.99:0/2692457186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55727529ff20 con 0x5572753891b0 2024-03-30T00:37:02.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.026+0000 7fc359400700 1 --2- 172.21.15.99:0/2692457186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x55727538a430 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0098e0 tx=0x7fc34c09ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff484e094ad9433d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:02.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.026+0000 7fc347e00700 1 -- 172.21.15.99:0/2692457186 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c0a3640 con 0x5572753891b0 2024-03-30T00:37:02.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.026+0000 7fc347e00700 1 -- 172.21.15.99:0/2692457186 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc34c09b900 con 0x5572753891b0 2024-03-30T00:37:02.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.026+0000 7fc347e00700 1 -- 172.21.15.99:0/2692457186 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c09f030 con 0x5572753891b0 2024-03-30T00:37:02.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.027+0000 7fc3749e6b80 1 -- 172.21.15.99:0/2692457186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 msgr2=0x55727538a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.027+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/2692457186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x55727538a430 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0098e0 tx=0x7fc34c09ac10 comp rx=0 tx=0).stop 2024-03-30T00:37:02.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.027+0000 7fc3749e6b80 1 -- 172.21.15.99:0/2692457186 shutdown_connections 2024-03-30T00:37:02.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.027+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/2692457186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55727538a050 0x5572753b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.027+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/2692457186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557275388850 0x557275396000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.027+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/2692457186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5572753891b0 0x55727538a430 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.027+0000 7fc3749e6b80 1 -- 172.21.15.99:0/2692457186 >> 172.21.15.99:0/2692457186 conn(0x5572752e7720 msgr2=0x55727537c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:02.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.028+0000 7fc3749e6b80 1 -- 172.21.15.99:0/2692457186 shutdown_connections 2024-03-30T00:37:02.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.028+0000 7fc3749e6b80 1 -- 172.21.15.99:0/2692457186 wait complete. 2024-03-30T00:37:02.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.028+0000 7fc3749e6b80 1 -- start start 2024-03-30T00:37:02.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.028+0000 7fc3749e6b80 1 Processor -- start 2024-03-30T00:37:02.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557275388850 0x5572753f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x5572753f3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc3749e6b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727538a050 0x5572753fa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572753fa540 con 0x55727538a050 2024-03-30T00:37:02.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572753fa6b0 con 0x5572753891b0 2024-03-30T00:37:02.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc3749e6b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572753fa870 con 0x557275388850 2024-03-30T00:37:02.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc3749e6b80 10 client.0 _ll_register_callbacks cb 0x55727527f4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:02.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc359400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x5572753f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc359400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x5572753f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:48894/0 (socket says 172.21.15.99:48894) 2024-03-30T00:37:02.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc359400700 1 -- 172.21.15.99:0/455704553 learned_addr learned my addr 172.21.15.99:0/455704553 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:02.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc35a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727538a050 0x5572753fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.029+0000 7fc359e00700 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557275388850 0x5572753f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.030+0000 7fc35a800700 1 -- 172.21.15.99:0/455704553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557275388850 msgr2=0x5572753f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.030+0000 7fc35a800700 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557275388850 0x5572753f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.030+0000 7fc35a800700 1 -- 172.21.15.99:0/455704553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 msgr2=0x5572753f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.030+0000 7fc35a800700 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x5572753f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.030+0000 7fc35a800700 1 -- 172.21.15.99:0/455704553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557275107be0 con 0x55727538a050 2024-03-30T00:37:02.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.030+0000 7fc359400700 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x5572753f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:02.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.030+0000 7fc35a800700 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727538a050 0x5572753fa000 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc354009fd0 tx=0x7fc35400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ab321ab2231665 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:02.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.030+0000 7fc33be00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:37:02.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.030+0000 7fc33be00700 1 -- 172.21.15.99:0/455704553 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35400e040 con 0x55727538a050 2024-03-30T00:37:02.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.030+0000 7fc33be00700 1 -- 172.21.15.99:0/455704553 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fc35401b040 con 0x55727538a050 2024-03-30T00:37:02.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.031+0000 7fc3749e6b80 1 -- 172.21.15.99:0/455704553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc348013470 con 0x55727538a050 2024-03-30T00:37:02.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.031+0000 7fc33be00700 1 -- 172.21.15.99:0/455704553 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc354015830 con 0x55727538a050 2024-03-30T00:37:02.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.031+0000 7fc3749e6b80 10 client.16080 fetch_fsmap learned FSMap version 25 2024-03-30T00:37:02.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.031+0000 7fc3749e6b80 1 -- 172.21.15.99:0/455704553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55727515fd50 con 0x55727538a050 2024-03-30T00:37:02.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.031+0000 7fc33be00700 1 -- 172.21.15.99:0/455704553 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc348013470 con 0x55727538a050 2024-03-30T00:37:02.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.032+0000 7fc33be00700 1 -- 172.21.15.99:0/455704553 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fc35402da10 con 0x55727538a050 2024-03-30T00:37:02.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.032+0000 7fc3749e6b80 10 client.16080 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:37:02.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.032+0000 7fc3749e6b80 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.032+0000 7fc3749e6b80 20 client.16080 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.032+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5572753a6970 0x5572753fe420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.032+0000 7fc3749e6b80 4 client.16080 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T00:37:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.032+0000 7fc3749e6b80 1 -- 172.21.15.99:0/455704553 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557275128d70 con 0x5572753a6970 2024-03-30T00:37:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.032+0000 7fc359e00700 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5572753a6970 0x5572753fe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.033+0000 7fc33be00700 10 client.16080 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:02.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.033+0000 7fc359e00700 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5572753a6970 0x5572753fe420 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa1730da28e8fde server_cookie=6a15463b8d1874c2 in_seq=0 out_seq=0 2024-03-30T00:37:02.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc33be00700 1 -- 172.21.15.99:0/455704553 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557275128d70 con 0x5572753a6970 2024-03-30T00:37:02.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc33be00700 10 client.16080 handle_command_reply: tid=0 2024-03-30T00:37:02.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc3749e6b80 1 client.16080 shutdown 2024-03-30T00:37:02.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc3749e6b80 1 -- 172.21.15.99:0/455704553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727538a050 msgr2=0x5572753fa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727538a050 0x5572753fa000 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc354009fd0 tx=0x7fc35400b040 comp rx=0 tx=0).stop 2024-03-30T00:37:02.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc3749e6b80 1 -- 172.21.15.99:0/455704553 shutdown_connections 2024-03-30T00:37:02.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5572753a6970 0x5572753fe420 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55727538a050 0x5572753fa000 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557275388850 0x5572753f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc3749e6b80 1 --2- 172.21.15.99:0/455704553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572753891b0 0x5572753f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc3749e6b80 1 -- 172.21.15.99:0/455704553 >> 172.21.15.99:0/455704553 conn(0x5572752e7720 msgr2=0x55727538d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:02.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.035+0000 7fc33be00700 0 client.16080 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:02.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.036+0000 7fc3749e6b80 1 -- 172.21.15.99:0/455704553 shutdown_connections 2024-03-30T00:37:02.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.036+0000 7fc3749e6b80 1 -- 172.21.15.99:0/455704553 wait complete. 2024-03-30T00:37:02.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.036+0000 7fc3749e6b80 20 client.16080 trim_cache size 0 max 16384 2024-03-30T00:37:02.057 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:37:02.057 INFO:teuthology.orchestra.run.smithi099.stdout: "return_code": 0, 2024-03-30T00:37:02.058 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_tag": "c680f250-fbaa-47c3-924b-3cb95519f6a8", 2024-03-30T00:37:02.058 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": "asynchronous" 2024-03-30T00:37:02.058 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:37:02.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 -- 172.21.15.99:0/690471804 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3400a1120 msgr2=0x7fc3400a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3400a1120 0x7fc3400a35e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc354009ec0 tx=0x7fc354004ad0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 -- 172.21.15.99:0/690471804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc35c104110 msgr2=0x7fc35c197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc35c104110 0x7fc35c197210 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc34c006010 tx=0x7fc34c003980 comp rx=0 tx=0).stop 2024-03-30T00:37:02.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 -- 172.21.15.99:0/690471804 shutdown_connections 2024-03-30T00:37:02.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc35c106a40 0x7fc35c19e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c1017e0 0x7fc35c196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fc3400a1120 0x7fc3400a35e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 --2- 172.21.15.99:0/690471804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc35c104110 0x7fc35c197210 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 -- 172.21.15.99:0/690471804 >> 172.21.15.99:0/690471804 conn(0x7fc35c0fb1d0 msgr2=0x7fc35c103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:02.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 -- 172.21.15.99:0/690471804 shutdown_connections 2024-03-30T00:37:02.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.037+0000 7fc360e00700 1 -- 172.21.15.99:0/690471804 wait complete. 2024-03-30T00:37:02.062 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:37:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-a[88085]: 2024-03-30T00:37:02.037+0000 7f3256800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000201 [2,head] /volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester auth v10 ap=1 s=26432 n(v0 rc2024-03-30T00:36:12.284517+0000 b26432 1=1+0) (ifile excl) (iversion lock) cr={15339=0-4194304@1} caps={15339=pAsLsXsFsxcrwb/pFscr@1},l=15339 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x558529388580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000201:[<0x10000000006/fsync-tester v10>,<0x10000000005/tmp v33>,<0x10000000003/client.0 v18>,<0x10000000002/53f81470-29f1-4e8a-82e0-10b602b921af v32>,<0x10000000001/sv_0 v57>,<0x10000000000/qa v45>,<0x1/volumes v43>]//[]","memoryvalue":"(3)0x10000000201:[<0x10000000006/fsync-tester v10>,<0x10000000005/tmp v64>,<0x10000000003/client.0 v48>,<0x10000000002/53f81470-29f1-4e8a-82e0-10b602b921af v61>,<0x10000000001/sv_0 v83>,<0x10000000000/qa v29>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:37:02.063 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:37:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-a[88085]: 2024-03-30T00:37:02.037+0000 7f3256800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [2,head] /volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.tst-file auth v104 ap=1 s=1048576 n(v0 rc2024-03-30T00:37:01.861148+0000 b1048576 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15339=0-4194304@1} caps={15339=pAsxLsXsxFsxcrwb/pAsxXsxFxcwb@1},l=15339 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x558529388b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000203:[<0x10000000006/fsync-tester.tst-file v46>,<0x10000000005/tmp v49>,<0x10000000003/client.0 v33>,<0x10000000002/53f81470-29f1-4e8a-82e0-10b602b921af v48>,<0x10000000001/sv_0 v73>,<0x10000000000/qa v61>,<0x1/volumes v59>]//[]","memoryvalue":"(3)0x10000000203:[<0x10000000006/fsync-tester.tst-file v104>,<0x10000000005/tmp v64>,<0x10000000003/client.0 v48>,<0x10000000002/53f81470-29f1-4e8a-82e0-10b602b921af v61>,<0x10000000001/sv_0 v83>,<0x10000000000/qa v29>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:37:02.063 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:37:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-a[88085]: 2024-03-30T00:37:02.037+0000 7f3256800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [2,head] /volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.c auth v6 ap=1 s=3024 n(v0 rc2024-03-30T00:36:12.132004+0000 b3024 1=1+0) (iversion lock) caps={15339=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x558529389080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000007:[<0x10000000006/fsync-tester.c v6>,<0x10000000005/tmp v33>,<0x10000000003/client.0 v18>,<0x10000000002/53f81470-29f1-4e8a-82e0-10b602b921af v32>,<0x10000000001/sv_0 v57>,<0x10000000000/qa v45>,<0x1/volumes v43>]//[]","memoryvalue":"(3)0x10000000007:[<0x10000000006/fsync-tester.c v6>,<0x10000000005/tmp v64>,<0x10000000003/client.0 v48>,<0x10000000002/53f81470-29f1-4e8a-82e0-10b602b921af v61>,<0x10000000001/sv_0 v83>,<0x10000000000/qa v29>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:37:02.063 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:37:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-a[88085]: 2024-03-30T00:37:02.037+0000 7f3256800700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000202 [2,head] /volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.rnd-file auth v32 ap=1 s=194220032 n(v0 rc2024-03-30T00:36:23.387548+0000 b194220032 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15339=0-390070272@1} caps={15339=pAsxLsXsxFsxcrwb/pAsxXsxFxcwb@1},l=15339 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x558529389600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000202:[<0x10000000006/fsync-tester.rnd-file v30>,<0x10000000005/tmp v33>,<0x10000000003/client.0 v18>,<0x10000000002/53f81470-29f1-4e8a-82e0-10b602b921af v32>,<0x10000000001/sv_0 v57>,<0x10000000000/qa v45>,<0x1/volumes v43>]//[]","memoryvalue":"(3)0x10000000202:[<0x10000000006/fsync-tester.rnd-file v32>,<0x10000000005/tmp v64>,<0x10000000003/client.0 v48>,<0x10000000002/53f81470-29f1-4e8a-82e0-10b602b921af v61>,<0x10000000001/sv_0 v83>,<0x10000000000/qa v29>,<0x1/volumes v73>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-30T00:37:02.069 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T00:37:02.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.298+0000 7f4634a00700 1 -- 172.21.15.99:0/3034053297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630074df0 msgr2=0x7f4630073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.298+0000 7f4634a00700 1 --2- 172.21.15.99:0/3034053297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630074df0 0x7f4630073240 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4620009990 tx=0x7f4620009ca0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.299+0000 7f4634a00700 1 -- 172.21.15.99:0/3034053297 shutdown_connections 2024-03-30T00:37:02.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.299+0000 7f4634a00700 1 --2- 172.21.15.99:0/3034053297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4630108c40 0x7f463010b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.299+0000 7f4634a00700 1 --2- 172.21.15.99:0/3034053297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630074df0 0x7f4630073240 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.299+0000 7f4634a00700 1 --2- 172.21.15.99:0/3034053297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4630073810 0x7f4630073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.299+0000 7f4634a00700 1 -- 172.21.15.99:0/3034053297 >> 172.21.15.99:0/3034053297 conn(0x7f46300fc2f0 msgr2=0x7f46300fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:02.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.299+0000 7f4634a00700 1 -- 172.21.15.99:0/3034053297 shutdown_connections 2024-03-30T00:37:02.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.299+0000 7f4634a00700 1 -- 172.21.15.99:0/3034053297 wait complete. 2024-03-30T00:37:02.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.300+0000 7f4634a00700 1 Processor -- start 2024-03-30T00:37:02.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.300+0000 7f4634a00700 1 -- start start 2024-03-30T00:37:02.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.300+0000 7f4634a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4630073810 0x7f4630198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.300+0000 7f4634a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4630074df0 0x7f4630199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.300+0000 7f4634a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630108c40 0x7f46301a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.300+0000 7f4634a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46301a0950 con 0x7f4630108c40 2024-03-30T00:37:02.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.300+0000 7f4634a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46301a0a90 con 0x7f4630074df0 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.300+0000 7f4634a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46301a0c20 con 0x7f4630073810 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.300+0000 7f462e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630108c40 0x7f46301a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.300+0000 7f462e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630108c40 0x7f46301a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44312/0 (socket says 172.21.15.99:44312) 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.300+0000 7f462e800700 1 -- 172.21.15.99:0/3131274537 learned_addr learned my addr 172.21.15.99:0/3131274537 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f462d400700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4630074df0 0x7f4630199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f462de00700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4630073810 0x7f4630198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f462e800700 1 -- 172.21.15.99:0/3131274537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4630073810 msgr2=0x7f4630198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f462e800700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4630073810 0x7f4630198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f462e800700 1 -- 172.21.15.99:0/3131274537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4630074df0 msgr2=0x7f4630199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f462e800700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4630074df0 0x7f4630199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f462e800700 1 -- 172.21.15.99:0/3131274537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46200095f0 con 0x7f4630108c40 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f462de00700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4630073810 0x7f4630198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f462e800700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630108c40 0x7f46301a0380 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f462800ef30 tx=0x7f462800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:02.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f4626a00700 1 -- 172.21.15.99:0/3131274537 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4628018070 con 0x7f4630108c40 2024-03-30T00:37:02.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f4626a00700 1 -- 172.21.15.99:0/3131274537 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f462800f040 con 0x7f4630108c40 2024-03-30T00:37:02.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f4626a00700 1 -- 172.21.15.99:0/3131274537 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462800cc70 con 0x7f4630108c40 2024-03-30T00:37:02.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.301+0000 7f4634a00700 1 -- 172.21.15.99:0/3131274537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46301a0ea0 con 0x7f4630108c40 2024-03-30T00:37:02.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.302+0000 7f4634a00700 1 -- 172.21.15.99:0/3131274537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46301a12d0 con 0x7f4630108c40 2024-03-30T00:37:02.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.304+0000 7f4626a00700 1 -- 172.21.15.99:0/3131274537 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f4628007500 con 0x7f4630108c40 2024-03-30T00:37:02.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.305+0000 7f46484fdb80 1 Processor -- start 2024-03-30T00:37:02.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f46484fdb80 1 -- start start 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f4626a00700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f46140a1120 0x7f46140a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f07bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08fd0 0x560962f09450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f46484fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962e5f990 con 0x560962f077d0 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f46484fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962e5d350 con 0x560962f08fd0 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f46484fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962e58c40 con 0x560962f08130 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f4626a00700 1 -- 172.21.15.99:0/3131274537 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f46280cc500 con 0x7f4630108c40 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f462d400700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f46140a1120 0x7f46140a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f462de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f07bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f462de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f07bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44326/0 (socket says 172.21.15.99:44326) 2024-03-30T00:37:02.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f462de00700 1 -- 172.21.15.99:0/910708932 learned_addr learned my addr 172.21.15.99:0/910708932 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.306+0000 7f462d400700 1 --2- 172.21.15.99:0/910708932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.307+0000 7f462e800700 1 --2- 172.21.15.99:0/910708932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08fd0 0x560962f09450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.307+0000 7f462de00700 1 -- 172.21.15.99:0/910708932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 msgr2=0x560962f089f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.307+0000 7f462de00700 1 --2- 172.21.15.99:0/910708932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f089f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.307+0000 7f462de00700 1 -- 172.21.15.99:0/910708932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08fd0 msgr2=0x560962f09450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.307+0000 7f462de00700 1 --2- 172.21.15.99:0/910708932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08fd0 0x560962f09450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.307+0000 7f462de00700 1 -- 172.21.15.99:0/910708932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560962cda3b0 con 0x560962f077d0 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.307+0000 7f462d400700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f46140a1120 0x7f46140a35e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4630074b10 tx=0x7f461c009b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.307+0000 7f462de00700 1 --2- 172.21.15.99:0/910708932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f07bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4620005b40 tx=0x7f4620005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c613c41f5f15e9a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.307+0000 7f461b000700 1 -- 172.21.15.99:0/910708932 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620005240 con 0x560962f077d0 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.307+0000 7f461b000700 1 -- 172.21.15.99:0/910708932 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f462001e040 con 0x560962f077d0 2024-03-30T00:37:02.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.307+0000 7f461b000700 1 -- 172.21.15.99:0/910708932 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000b970 con 0x560962f077d0 2024-03-30T00:37:02.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.308+0000 7f46484fdb80 1 -- 172.21.15.99:0/910708932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 msgr2=0x560962f07bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.308+0000 7f46484fdb80 1 --2- 172.21.15.99:0/910708932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f07bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4620005b40 tx=0x7f4620005c00 comp rx=0 tx=0).stop 2024-03-30T00:37:02.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.308+0000 7f46484fdb80 1 -- 172.21.15.99:0/910708932 shutdown_connections 2024-03-30T00:37:02.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.308+0000 7f46484fdb80 1 --2- 172.21.15.99:0/910708932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08fd0 0x560962f09450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.308+0000 7f46484fdb80 1 --2- 172.21.15.99:0/910708932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f07bf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.308+0000 7f46484fdb80 1 --2- 172.21.15.99:0/910708932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.308+0000 7f46484fdb80 1 -- 172.21.15.99:0/910708932 >> 172.21.15.99:0/910708932 conn(0x560962e666a0 msgr2=0x560962f029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:02.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.308+0000 7f46484fdb80 1 -- 172.21.15.99:0/910708932 shutdown_connections 2024-03-30T00:37:02.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.308+0000 7f46484fdb80 1 -- 172.21.15.99:0/910708932 wait complete. 2024-03-30T00:37:02.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.309+0000 7f46484fdb80 1 -- start start 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.309+0000 7f46484fdb80 1 Processor -- start 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f115a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f11ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08fd0 0x560962f18b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f46484fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962df4d80 con 0x560962f077d0 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f46484fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962df4a20 con 0x560962f08fd0 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f46484fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962cdc6a0 con 0x560962f08130 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f46484fdb80 10 client.0 _ll_register_callbacks cb 0x560962dfe470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f462de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f115a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f462de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f115a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44328/0 (socket says 172.21.15.99:44328) 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f462de00700 1 -- 172.21.15.99:0/1408050866 learned_addr learned my addr 172.21.15.99:0/1408050866 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f462d400700 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f11ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f462de00700 1 -- 172.21.15.99:0/1408050866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 msgr2=0x560962f11ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f462de00700 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f11ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f462e800700 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08fd0 0x560962f18b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f462de00700 1 -- 172.21.15.99:0/1408050866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08fd0 msgr2=0x560962f18b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.310+0000 7f462de00700 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08fd0 0x560962f18b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.311+0000 7f462de00700 1 -- 172.21.15.99:0/1408050866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463010bcf0 con 0x560962f077d0 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.311+0000 7f462e800700 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08fd0 0x560962f18b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.311+0000 7f462de00700 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f115a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4620005910 tx=0x7f4620005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2d9651d91abe71 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.311+0000 7f460f400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.311+0000 7f460f400700 1 -- 172.21.15.99:0/1408050866 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000b710 con 0x560962f077d0 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.311+0000 7f460f400700 1 -- 172.21.15.99:0/1408050866 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f462000b870 con 0x560962f077d0 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.311+0000 7f46484fdb80 1 -- 172.21.15.99:0/1408050866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560962e19380 con 0x560962f077d0 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.311+0000 7f460f400700 1 -- 172.21.15.99:0/1408050866 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f462000ba30 con 0x560962f077d0 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.311+0000 7f46484fdb80 10 client.16092 fetch_fsmap learned FSMap version 25 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.311+0000 7f46484fdb80 1 -- 172.21.15.99:0/1408050866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560962c86b60 con 0x560962f077d0 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.312+0000 7f460f400700 1 -- 172.21.15.99:0/1408050866 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f462003a020 con 0x560962f077d0 2024-03-30T00:37:02.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.312+0000 7f460f400700 1 -- 172.21.15.99:0/1408050866 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4620004b00 con 0x560962f077d0 2024-03-30T00:37:02.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.312+0000 7f46484fdb80 10 client.16092 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:37:02.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.312+0000 7f46484fdb80 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:02.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.312+0000 7f46484fdb80 20 client.16092 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:02.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.312+0000 7f46484fdb80 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x560962f1cef0 0x560962f1f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.312+0000 7f46484fdb80 4 client.16092 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:37:02.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.312+0000 7f46484fdb80 1 -- 172.21.15.99:0/1408050866 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560962e51700 con 0x560962f1cef0 2024-03-30T00:37:02.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.312+0000 7f462d400700 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x560962f1cef0 0x560962f1f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.313+0000 7f462d400700 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x560962f1cef0 0x560962f1f310 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f67de623ad2cc162 server_cookie=7db460e423bab8bd in_seq=0 out_seq=0 2024-03-30T00:37:02.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.313+0000 7f460f400700 10 client.16092 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:02.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.313+0000 7f460f400700 1 -- 172.21.15.99:0/1408050866 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560962e51700 con 0x560962f1cef0 2024-03-30T00:37:02.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.313+0000 7f460f400700 10 client.16092 handle_command_reply: tid=0 2024-03-30T00:37:02.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.313+0000 7f46484fdb80 1 client.16092 shutdown 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f46484fdb80 1 -- 172.21.15.99:0/1408050866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 msgr2=0x560962f115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f46484fdb80 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f115a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4620005910 tx=0x7f4620005a00 comp rx=0 tx=0).stop 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f46484fdb80 1 -- 172.21.15.99:0/1408050866 shutdown_connections 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f46484fdb80 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x560962f1cef0 0x560962f1f310 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f46484fdb80 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08fd0 0x560962f18b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f46484fdb80 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f077d0 0x560962f115a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f46484fdb80 1 --2- 172.21.15.99:0/1408050866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f11ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f460f400700 0 client.16092 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f46484fdb80 1 -- 172.21.15.99:0/1408050866 >> 172.21.15.99:0/1408050866 conn(0x560962e666a0 msgr2=0x560962f06170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f46484fdb80 1 -- 172.21.15.99:0/1408050866 shutdown_connections 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f46484fdb80 1 -- 172.21.15.99:0/1408050866 wait complete. 2024-03-30T00:37:02.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.314+0000 7f46484fdb80 20 client.16092 trim_cache size 0 max 16384 2024-03-30T00:37:02.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.331+0000 7f46484fdb80 1 Processor -- start 2024-03-30T00:37:02.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.331+0000 7f46484fdb80 1 -- start start 2024-03-30T00:37:02.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f077d0 0x560962f06170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08130 0x560962f18ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f46484fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962e51700 con 0x560962f08fd0 2024-03-30T00:37:02.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f46484fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962f034b0 con 0x560962f08130 2024-03-30T00:37:02.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f46484fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962f08e50 con 0x560962f077d0 2024-03-30T00:37:02.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f462e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f462e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f066b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44330/0 (socket says 172.21.15.99:44330) 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f462e800700 1 -- 172.21.15.99:0/962193729 learned_addr learned my addr 172.21.15.99:0/962193729 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f462de00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f077d0 0x560962f06170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f462d400700 1 --2- 172.21.15.99:0/962193729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08130 0x560962f18ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f462e800700 1 -- 172.21.15.99:0/962193729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f077d0 msgr2=0x560962f06170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f462e800700 1 --2- 172.21.15.99:0/962193729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f077d0 0x560962f06170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f462e800700 1 -- 172.21.15.99:0/962193729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08130 msgr2=0x560962f18ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f462e800700 1 --2- 172.21.15.99:0/962193729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08130 0x560962f18ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.332+0000 7f462e800700 1 -- 172.21.15.99:0/962193729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560962e1eea0 con 0x560962f08fd0 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f462e800700 1 --2- 172.21.15.99:0/962193729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f066b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f46280c9f20 tx=0x7f4628099380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762549bdbb854c5b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f461ba00700 1 -- 172.21.15.99:0/962193729 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462809a000 con 0x560962f08fd0 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f461ba00700 1 -- 172.21.15.99:0/962193729 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4628099900 con 0x560962f08fd0 2024-03-30T00:37:02.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f461ba00700 1 -- 172.21.15.99:0/962193729 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4628099dc0 con 0x560962f08fd0 2024-03-30T00:37:02.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f46484fdb80 1 -- 172.21.15.99:0/962193729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 msgr2=0x560962f066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f46484fdb80 1 --2- 172.21.15.99:0/962193729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f066b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f46280c9f20 tx=0x7f4628099380 comp rx=0 tx=0).stop 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f46484fdb80 1 -- 172.21.15.99:0/962193729 shutdown_connections 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f46484fdb80 1 --2- 172.21.15.99:0/962193729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f066b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f46484fdb80 1 --2- 172.21.15.99:0/962193729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f077d0 0x560962f06170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f46484fdb80 1 --2- 172.21.15.99:0/962193729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f08130 0x560962f18ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f46484fdb80 1 -- 172.21.15.99:0/962193729 >> 172.21.15.99:0/962193729 conn(0x560962e666a0 msgr2=0x560962f10ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f46484fdb80 1 -- 172.21.15.99:0/962193729 shutdown_connections 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.333+0000 7f46484fdb80 1 -- 172.21.15.99:0/962193729 wait complete. 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.334+0000 7f46484fdb80 1 -- start start 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.334+0000 7f46484fdb80 1 Processor -- start 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.334+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f077d0 0x560962f17290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.334+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f177d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.334+0000 7f46484fdb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f17d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.334+0000 7f46484fdb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962f18e20 con 0x560962f08fd0 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.334+0000 7f46484fdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962f70cf0 con 0x560962f077d0 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.334+0000 7f46484fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560962f70e30 con 0x560962f08130 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f46484fdb80 10 client.0 _ll_register_callbacks cb 0x560962dfe470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f17d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462d400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f17d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44332/0 (socket says 172.21.15.99:44332) 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462e800700 1 -- 172.21.15.99:0/2322403827 learned_addr learned my addr 172.21.15.99:0/2322403827 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462de00700 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f077d0 0x560962f17290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462e800700 1 -- 172.21.15.99:0/2322403827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 msgr2=0x560962f177d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462e800700 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f177d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462e800700 1 -- 172.21.15.99:0/2322403827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f077d0 msgr2=0x560962f17290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462e800700 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f077d0 0x560962f17290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462e800700 1 -- 172.21.15.99:0/2322403827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46301a12d0 con 0x560962f08fd0 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462d400700 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.335+0000 7f462de00700 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f077d0 0x560962f17290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:02.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.336+0000 7f462e800700 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f17d10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4628096370 tx=0x7f4628095630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05107306e1ca791 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:02.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.336+0000 7f460f400700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:37:02.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.336+0000 7f460f400700 1 -- 172.21.15.99:0/2322403827 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462809a000 con 0x560962f08fd0 2024-03-30T00:37:02.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.336+0000 7f460f400700 1 -- 172.21.15.99:0/2322403827 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f462809d730 con 0x560962f08fd0 2024-03-30T00:37:02.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.336+0000 7f46484fdb80 1 -- 172.21.15.99:0/2322403827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560962f105d0 con 0x560962f08fd0 2024-03-30T00:37:02.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.339+0000 7f460f400700 1 -- 172.21.15.99:0/2322403827 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4628099b60 con 0x560962f08fd0 2024-03-30T00:37:02.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.339+0000 7f46484fdb80 10 client.16098 fetch_fsmap learned FSMap version 25 2024-03-30T00:37:02.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.339+0000 7f46484fdb80 1 -- 172.21.15.99:0/2322403827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560962cdecd0 con 0x560962f08fd0 2024-03-30T00:37:02.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.339+0000 7f460f400700 1 -- 172.21.15.99:0/2322403827 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f462807c080 con 0x560962f08fd0 2024-03-30T00:37:02.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.339+0000 7f460f400700 1 -- 172.21.15.99:0/2322403827 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f46280a1960 con 0x560962f08fd0 2024-03-30T00:37:02.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.339+0000 7f46484fdb80 10 client.16098 fetch_fsmap finished waiting for FSMap version 25 2024-03-30T00:37:02.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.339+0000 7f46484fdb80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:02.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.340+0000 7f46484fdb80 20 client.16098 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:02.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.340+0000 7f46484fdb80 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x560962f72520 0x560962f74940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:02.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.340+0000 7f46484fdb80 4 client.16098 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:37:02.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.340+0000 7f46484fdb80 1 -- 172.21.15.99:0/2322403827 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f462001e040 con 0x560962f72520 2024-03-30T00:37:02.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.340+0000 7f462de00700 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x560962f72520 0x560962f74940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:02.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.340+0000 7f460f400700 10 client.16098 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:02.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.340+0000 7f462de00700 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x560962f72520 0x560962f74940 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81514292f6908609 server_cookie=f20dfd06a08941cd in_seq=0 out_seq=0 2024-03-30T00:37:02.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f460f400700 1 -- 172.21.15.99:0/2322403827 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f462001e040 con 0x560962f72520 2024-03-30T00:37:02.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f460f400700 10 client.16098 handle_command_reply: tid=0 2024-03-30T00:37:02.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 1 client.16098 shutdown 2024-03-30T00:37:02.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 1 -- 172.21.15.99:0/2322403827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 msgr2=0x560962f17d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f17d10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4628096370 tx=0x7f4628095630 comp rx=0 tx=0).stop 2024-03-30T00:37:02.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 1 -- 172.21.15.99:0/2322403827 shutdown_connections 2024-03-30T00:37:02.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x560962f72520 0x560962f74940 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560962f08fd0 0x560962f17d10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560962f077d0 0x560962f17290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f460f400700 0 client.16098 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:02.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 1 --2- 172.21.15.99:0/2322403827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560962f08130 0x560962f177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 1 -- 172.21.15.99:0/2322403827 >> 172.21.15.99:0/2322403827 conn(0x560962e666a0 msgr2=0x560962f06170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:02.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 1 -- 172.21.15.99:0/2322403827 shutdown_connections 2024-03-30T00:37:02.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 1 -- 172.21.15.99:0/2322403827 wait complete. 2024-03-30T00:37:02.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.341+0000 7f46484fdb80 20 client.16098 trim_cache size 0 max 16384 2024-03-30T00:37:02.350 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:37:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T00:37:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": { 2024-03-30T00:37:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: "c680f250-fbaa-47c3-924b-3cb95519f6a8": { 2024-03-30T00:37:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/", 2024-03-30T00:37:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: "tag": "c680f250-fbaa-47c3-924b-3cb95519f6a8", 2024-03-30T00:37:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: "options": "recursive,force" 2024-03-30T00:37:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:02.351 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 -- 172.21.15.99:0/3131274537 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f46140a1120 msgr2=0x7f46140a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f46140a1120 0x7f46140a35e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4630074b10 tx=0x7f461c009b60 comp rx=0 tx=0).stop 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 -- 172.21.15.99:0/3131274537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630108c40 msgr2=0x7f46301a0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630108c40 0x7f46301a0380 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f462800ef30 tx=0x7f462800c3a0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 -- 172.21.15.99:0/3131274537 shutdown_connections 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4630108c40 0x7f46301a0380 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4630073810 0x7f4630198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f46140a1120 0x7f46140a35e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 --2- 172.21.15.99:0/3131274537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4630074df0 0x7f4630199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 -- 172.21.15.99:0/3131274537 >> 172.21.15.99:0/3131274537 conn(0x7f46300fc2f0 msgr2=0x7f4630104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 -- 172.21.15.99:0/3131274537 shutdown_connections 2024-03-30T00:37:02.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:02.342+0000 7f4634a00700 1 -- 172.21.15.99:0/3131274537 wait complete. 2024-03-30T00:37:02.367 INFO:tasks.cephfs.filesystem:scrub status for tag:c680f250-fbaa-47c3-924b-3cb95519f6a8 - {'path': '/', 'tag': 'c680f250-fbaa-47c3-924b-3cb95519f6a8', 'options': 'recursive,force'} 2024-03-30T00:37:02.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:02 smithi161 ceph-mon[42221]: pgmap v373: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 2.77k op/s 2024-03-30T00:37:02.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:02 smithi161 ceph-mon[42221]: scrub summary: active paths [/] 2024-03-30T00:37:02.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:02 smithi161 ceph-mon[42221]: Scrub error on inode 0x10000000201 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:02 smithi161 ceph-mon[42221]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.tst-file) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:02 smithi161 ceph-mon[42221]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.c) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:02 smithi161 ceph-mon[42221]: Scrub error on inode 0x10000000202 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.rnd-file) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.720 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:02 smithi161 ceph-mon[42221]: scrub summary: idle+waiting paths [/] 2024-03-30T00:37:02.738 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:02 smithi099 ceph-mon[38079]: pgmap v373: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 2.77k op/s 2024-03-30T00:37:02.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:02 smithi099 ceph-mon[38079]: scrub summary: active paths [/] 2024-03-30T00:37:02.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:02 smithi099 ceph-mon[38079]: Scrub error on inode 0x10000000201 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:02 smithi099 ceph-mon[38079]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.tst-file) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:02 smithi099 ceph-mon[38079]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.c) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:02 smithi099 ceph-mon[38079]: Scrub error on inode 0x10000000202 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.rnd-file) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.739 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:02 smithi099 ceph-mon[38079]: scrub summary: idle+waiting paths [/] 2024-03-30T00:37:02.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:02 smithi142 ceph-mon[41779]: pgmap v373: 97 pgs: 97 active+clean; 258 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 12 MiB/s wr, 2.77k op/s 2024-03-30T00:37:02.817 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:02 smithi142 ceph-mon[41779]: scrub summary: active paths [/] 2024-03-30T00:37:02.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:02 smithi142 ceph-mon[41779]: Scrub error on inode 0x10000000201 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:02 smithi142 ceph-mon[41779]: Scrub error on inode 0x10000000203 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.tst-file) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:02 smithi142 ceph-mon[41779]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.c) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:02 smithi142 ceph-mon[41779]: Scrub error on inode 0x10000000202 (/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.rnd-file) see mds.a log and `damage ls` output for details 2024-03-30T00:37:02.818 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:02 smithi142 ceph-mon[41779]: scrub summary: idle+waiting paths [/] 2024-03-30T00:37:03.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:03 smithi161 ceph-mon[42221]: scrub queued for path: / 2024-03-30T00:37:03.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:03 smithi161 ceph-mon[42221]: scrub summary: idle+waiting paths [/] 2024-03-30T00:37:03.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:03 smithi161 ceph-mon[42221]: scrub summary: active paths [/] 2024-03-30T00:37:03.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:03 smithi161 ceph-mon[42221]: scrub summary: idle+waiting paths [/] 2024-03-30T00:37:03.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:03 smithi099 ceph-mon[38079]: scrub queued for path: / 2024-03-30T00:37:03.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:03 smithi099 ceph-mon[38079]: scrub summary: idle+waiting paths [/] 2024-03-30T00:37:03.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:03 smithi099 ceph-mon[38079]: scrub summary: active paths [/] 2024-03-30T00:37:03.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:03 smithi099 ceph-mon[38079]: scrub summary: idle+waiting paths [/] 2024-03-30T00:37:03.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:03 smithi142 ceph-mon[41779]: scrub queued for path: / 2024-03-30T00:37:03.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:03 smithi142 ceph-mon[41779]: scrub summary: idle+waiting paths [/] 2024-03-30T00:37:03.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:03 smithi142 ceph-mon[41779]: scrub summary: active paths [/] 2024-03-30T00:37:03.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:03 smithi142 ceph-mon[41779]: scrub summary: idle+waiting paths [/] 2024-03-30T00:37:04.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:04 smithi161 ceph-mon[42221]: pgmap v374: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 MiB/s wr, 2.76k op/s 2024-03-30T00:37:04.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:04 smithi099 ceph-mon[38079]: pgmap v374: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 MiB/s wr, 2.76k op/s 2024-03-30T00:37:04.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:04 smithi142 ceph-mon[41779]: pgmap v374: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 MiB/s wr, 2.76k op/s 2024-03-30T00:37:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:06 smithi161 ceph-mon[42221]: pgmap v375: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 13 MiB/s wr, 2.88k op/s 2024-03-30T00:37:06.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:06 smithi099 ceph-mon[38079]: pgmap v375: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 13 MiB/s wr, 2.88k op/s 2024-03-30T00:37:06.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:06 smithi142 ceph-mon[41779]: pgmap v375: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 13 MiB/s wr, 2.88k op/s 2024-03-30T00:37:07.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:07 smithi161 ceph-mon[42221]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-30T00:37:07.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:07 smithi161 ceph-mon[42221]: mds.? [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] up:active 2024-03-30T00:37:07.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:07 smithi161 ceph-mon[42221]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:37:07.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:07 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-a[38056]: 2024-03-30T00:37:07.350+0000 7f14fb600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-30T00:37:07.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:07 smithi099 ceph-mon[38079]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-30T00:37:07.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:07 smithi099 ceph-mon[38079]: mds.? [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] up:active 2024-03-30T00:37:07.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:07 smithi099 ceph-mon[38079]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:37:07.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:07 smithi142 ceph-mon[41779]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-30T00:37:07.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:07 smithi142 ceph-mon[41779]: mds.? [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] up:active 2024-03-30T00:37:07.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:07 smithi142 ceph-mon[41779]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:37:08.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:08 smithi161 ceph-mon[42221]: pgmap v376: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 12 MiB/s wr, 2.62k op/s 2024-03-30T00:37:08.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:08 smithi099 ceph-mon[38079]: pgmap v376: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 12 MiB/s wr, 2.62k op/s 2024-03-30T00:37:08.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:08 smithi142 ceph-mon[41779]: pgmap v376: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 12 MiB/s wr, 2.62k op/s 2024-03-30T00:37:09.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:09 smithi161 ceph-mon[42221]: scrub summary: idle 2024-03-30T00:37:09.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:09 smithi161 ceph-mon[42221]: scrub summary: idle 2024-03-30T00:37:09.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:09 smithi099 ceph-mon[38079]: scrub summary: idle 2024-03-30T00:37:09.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:09 smithi099 ceph-mon[38079]: scrub summary: idle 2024-03-30T00:37:09.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:09 smithi142 ceph-mon[41779]: scrub summary: idle 2024-03-30T00:37:09.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:09 smithi142 ceph-mon[41779]: scrub summary: idle 2024-03-30T00:37:10.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:10 smithi161 ceph-mon[42221]: pgmap v377: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 13 MiB/s wr, 2.87k op/s 2024-03-30T00:37:10.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:10 smithi099 ceph-mon[38079]: pgmap v377: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 13 MiB/s wr, 2.87k op/s 2024-03-30T00:37:10.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:10 smithi142 ceph-mon[41779]: pgmap v377: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 13 MiB/s wr, 2.87k op/s 2024-03-30T00:37:12.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:12 smithi161 ceph-mon[42221]: pgmap v378: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 MiB/s wr, 2.64k op/s 2024-03-30T00:37:12.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:12 smithi099 ceph-mon[38079]: pgmap v378: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 MiB/s wr, 2.64k op/s 2024-03-30T00:37:12.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:12 smithi142 ceph-mon[41779]: pgmap v378: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 MiB/s wr, 2.64k op/s 2024-03-30T00:37:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.353+0000 7f6542c00700 1 -- 172.21.15.99:0/4213813043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f653c1028a0 msgr2=0x7f653c109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.353+0000 7f6542c00700 1 --2- 172.21.15.99:0/4213813043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f653c1028a0 0x7f653c109150 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f653400b0d0 tx=0x7f653400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.354+0000 7f6542c00700 1 -- 172.21.15.99:0/4213813043 shutdown_connections 2024-03-30T00:37:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.354+0000 7f6542c00700 1 --2- 172.21.15.99:0/4213813043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f653c1028a0 0x7f653c109150 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.354+0000 7f6542c00700 1 --2- 172.21.15.99:0/4213813043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c100d10 0x7f653c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.354+0000 7f6542c00700 1 --2- 172.21.15.99:0/4213813043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653c101f00 0x7f653c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.354+0000 7f6542c00700 1 -- 172.21.15.99:0/4213813043 >> 172.21.15.99:0/4213813043 conn(0x7f653c0fc280 msgr2=0x7f653c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:13.361 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.354+0000 7f6542c00700 1 -- 172.21.15.99:0/4213813043 shutdown_connections 2024-03-30T00:37:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.354+0000 7f6542c00700 1 -- 172.21.15.99:0/4213813043 wait complete. 2024-03-30T00:37:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.355+0000 7f6542c00700 1 Processor -- start 2024-03-30T00:37:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.356+0000 7f6542c00700 1 -- start start 2024-03-30T00:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.356+0000 7f6542c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c100d10 0x7f653c06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.356+0000 7f653be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c100d10 0x7f653c06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.356+0000 7f653be00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c100d10 0x7f653c06f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:60224/0 (socket says 172.21.15.99:60224) 2024-03-30T00:37:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.357+0000 7f6542c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f653c101f00 0x7f653c06fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.357+0000 7f6542c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653c1028a0 0x7f653c1a8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.357+0000 7f6542c00700 1 -- 172.21.15.99:0/1629580258 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c070340 con 0x7f653c101f00 2024-03-30T00:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.357+0000 7f653be00700 1 -- 172.21.15.99:0/1629580258 learned_addr learned my addr 172.21.15.99:0/1629580258 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.357+0000 7f6542c00700 1 -- 172.21.15.99:0/1629580258 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c1a9350 con 0x7f653c1028a0 2024-03-30T00:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.357+0000 7f6540c00700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653c1028a0 0x7f653c1a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.357+0000 7f6542c00700 1 -- 172.21.15.99:0/1629580258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c1a9490 con 0x7f653c100d10 2024-03-30T00:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.358+0000 7f653b400700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f653c101f00 0x7f653c06fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.358+0000 7f653be00700 1 -- 172.21.15.99:0/1629580258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653c1028a0 msgr2=0x7f653c1a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.358+0000 7f653be00700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653c1028a0 0x7f653c1a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.358+0000 7f653be00700 1 -- 172.21.15.99:0/1629580258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f653c101f00 msgr2=0x7f653c06fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.358+0000 7f653be00700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f653c101f00 0x7f653c06fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.358+0000 7f653be00700 1 -- 172.21.15.99:0/1629580258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6528009520 con 0x7f653c100d10 2024-03-30T00:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.358+0000 7f653b400700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f653c101f00 0x7f653c06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.358+0000 7f653be00700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c100d10 0x7f653c06f7e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f652c00c990 tx=0x7f652c00cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.358+0000 7f6538c00700 1 -- 172.21.15.99:0/1629580258 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c007b00 con 0x7f653c100d10 2024-03-30T00:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.359+0000 7f6538c00700 1 -- 172.21.15.99:0/1629580258 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f652c007c60 con 0x7f653c100d10 2024-03-30T00:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.359+0000 7f6538c00700 1 -- 172.21.15.99:0/1629580258 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c018660 con 0x7f653c100d10 2024-03-30T00:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.359+0000 7f6542c00700 1 -- 172.21.15.99:0/1629580258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6534009d00 con 0x7f653c100d10 2024-03-30T00:37:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.359+0000 7f6542c00700 1 -- 172.21.15.99:0/1629580258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c1a9a20 con 0x7f653c100d10 2024-03-30T00:37:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.365+0000 7f6538c00700 1 -- 172.21.15.99:0/1629580258 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f652c00f450 con 0x7f653c100d10 2024-03-30T00:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.367+0000 7f6538c00700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f65200a1270 0x7f65200a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.367+0000 7f6538c00700 1 -- 172.21.15.99:0/1629580258 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f652c0cc800 con 0x7f653c100d10 2024-03-30T00:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.368+0000 7f6556753b80 1 Processor -- start 2024-03-30T00:37:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.368+0000 7f653b400700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f65200a1270 0x7f65200a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.368+0000 7f6556753b80 1 -- start start 2024-03-30T00:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.368+0000 7f6556753b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 0x559d32829620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.369+0000 7f6556753b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d32829b60 0x559d3282a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.369+0000 7f6556753b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3282aa60 0x559d3282aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.369+0000 7f653b400700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f65200a1270 0x7f65200a3730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6528009d70 tx=0x7f6528009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.369+0000 7f653be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 0x559d32829620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.369+0000 7f6556753b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d32781930 con 0x559d3282aa60 2024-03-30T00:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.369+0000 7f653b400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d32829b60 0x559d3282a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.369+0000 7f6556753b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3277f2f0 con 0x559d32829200 2024-03-30T00:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.369+0000 7f6556753b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d3277abe0 con 0x559d32829b60 2024-03-30T00:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.369+0000 7f6540c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3282aa60 0x559d3282aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.369+0000 7f653b400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d32829b60 0x559d3282a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:60232/0 (socket says 172.21.15.99:60232) 2024-03-30T00:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.369+0000 7f653b400700 1 -- 172.21.15.99:0/3375483207 learned_addr learned my addr 172.21.15.99:0/3375483207 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.370+0000 7f653be00700 1 -- 172.21.15.99:0/3375483207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d32829b60 msgr2=0x559d3282a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.370+0000 7f653be00700 1 --2- 172.21.15.99:0/3375483207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d32829b60 0x559d3282a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.370+0000 7f653be00700 1 -- 172.21.15.99:0/3375483207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3282aa60 msgr2=0x559d3282aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.370+0000 7f653be00700 1 --2- 172.21.15.99:0/3375483207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3282aa60 0x559d3282aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.370+0000 7f653be00700 1 -- 172.21.15.99:0/3375483207 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d325fc350 con 0x559d32829200 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.370+0000 7f653b400700 1 --2- 172.21.15.99:0/3375483207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d32829b60 0x559d3282a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.370+0000 7f653be00700 1 --2- 172.21.15.99:0/3375483207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 0x559d32829620 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f652c0074a0 tx=0x7f652c099790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fccc956c439d8c4f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.370+0000 7f6531200700 1 -- 172.21.15.99:0/3375483207 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c007dd0 con 0x559d32829200 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.370+0000 7f6531200700 1 -- 172.21.15.99:0/3375483207 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f652c09a350 con 0x559d32829200 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.370+0000 7f6531200700 1 -- 172.21.15.99:0/3375483207 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c00f700 con 0x559d32829200 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.371+0000 7f6556753b80 1 -- 172.21.15.99:0/3375483207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 msgr2=0x559d32829620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.371+0000 7f6556753b80 1 --2- 172.21.15.99:0/3375483207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 0x559d32829620 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f652c0074a0 tx=0x7f652c099790 comp rx=0 tx=0).stop 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.371+0000 7f6556753b80 1 -- 172.21.15.99:0/3375483207 shutdown_connections 2024-03-30T00:37:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.371+0000 7f6556753b80 1 --2- 172.21.15.99:0/3375483207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d3282aa60 0x559d3282aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.371+0000 7f6556753b80 1 --2- 172.21.15.99:0/3375483207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 0x559d32829620 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.371+0000 7f6556753b80 1 --2- 172.21.15.99:0/3375483207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d32829b60 0x559d3282a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.371+0000 7f6556753b80 1 -- 172.21.15.99:0/3375483207 >> 172.21.15.99:0/3375483207 conn(0x559d32788640 msgr2=0x559d32824050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.371+0000 7f6556753b80 1 -- 172.21.15.99:0/3375483207 shutdown_connections 2024-03-30T00:37:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.371+0000 7f6556753b80 1 -- 172.21.15.99:0/3375483207 wait complete. 2024-03-30T00:37:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.372+0000 7f6556753b80 1 -- start start 2024-03-30T00:37:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.372+0000 7f6556753b80 1 Processor -- start 2024-03-30T00:37:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.373+0000 7f6556753b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 0x559d328331f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.373+0000 7f6556753b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829b60 0x559d32833730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.373+0000 7f6556753b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d3283a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.373+0000 7f6556753b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d32716d20 con 0x559d32829b60 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.373+0000 7f6556753b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d327169c0 con 0x559d32829200 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.373+0000 7f6556753b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d325fe640 con 0x559d3282aa60 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.374+0000 7f6556753b80 10 client.0 _ll_register_callbacks cb 0x559d32720410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.374+0000 7f653be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 0x559d328331f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.374+0000 7f653be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 0x559d328331f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:59900/0 (socket says 172.21.15.99:59900) 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.374+0000 7f653be00700 1 -- 172.21.15.99:0/3718051172 learned_addr learned my addr 172.21.15.99:0/3718051172 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.374+0000 7f6540c00700 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d3283a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.375+0000 7f653b400700 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829b60 0x559d32833730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.375+0000 7f653be00700 1 -- 172.21.15.99:0/3718051172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 msgr2=0x559d3283a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.375+0000 7f653be00700 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d3283a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.375+0000 7f653be00700 1 -- 172.21.15.99:0/3718051172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829b60 msgr2=0x559d32833730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.375+0000 7f653be00700 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829b60 0x559d32833730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.375+0000 7f653be00700 1 -- 172.21.15.99:0/3718051172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6528010bf0 con 0x559d32829200 2024-03-30T00:37:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.375+0000 7f653b400700 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829b60 0x559d32833730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.375+0000 7f653be00700 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 0x559d328331f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f652c0ca5f0 tx=0x7f652c016fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=271e1f6eb8cb4cdd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.375+0000 7f6525600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.376+0000 7f6525600700 1 -- 172.21.15.99:0/3718051172 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c09da80 con 0x559d32829200 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.376+0000 7f6525600700 1 -- 172.21.15.99:0/3718051172 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f652c0a20d0 con 0x559d32829200 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.376+0000 7f6556753b80 1 -- 172.21.15.99:0/3718051172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d3273b320 con 0x559d32829200 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.376+0000 7f6525600700 1 -- 172.21.15.99:0/3718051172 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f652c099140 con 0x559d32829200 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.376+0000 7f6556753b80 10 client.25796 fetch_fsmap learned FSMap version 26 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.376+0000 7f6556753b80 1 -- 172.21.15.99:0/3718051172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f653c1a9a20 con 0x559d32829200 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.377+0000 7f6525600700 1 -- 172.21.15.99:0/3718051172 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559d3273b320 con 0x559d32829200 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.377+0000 7f6525600700 1 -- 172.21.15.99:0/3718051172 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f652c095a40 con 0x559d32829200 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.377+0000 7f6556753b80 10 client.25796 fetch_fsmap finished waiting for FSMap version 26 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.377+0000 7f6556753b80 10 client.25796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.377+0000 7f6556753b80 20 client.25796 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.377+0000 7f6556753b80 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x559d3283d280 0x559d3283f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.377+0000 7f6556753b80 4 client.25796 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:37:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.377+0000 7f6556753b80 1 -- 172.21.15.99:0/3718051172 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d327736a0 con 0x559d3283d280 2024-03-30T00:37:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.377+0000 7f653b400700 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x559d3283d280 0x559d3283f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.378+0000 7f653b400700 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x559d3283d280 0x559d3283f6a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=381507bbf90a2d2b server_cookie=ee5963cc23ec0397 in_seq=0 out_seq=0 2024-03-30T00:37:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.378+0000 7f6525600700 10 client.25796 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.379+0000 7f6525600700 1 -- 172.21.15.99:0/3718051172 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559d327736a0 con 0x559d3283d280 2024-03-30T00:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.379+0000 7f6525600700 10 client.25796 handle_command_reply: tid=0 2024-03-30T00:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.379+0000 7f6556753b80 1 client.25796 shutdown 2024-03-30T00:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.379+0000 7f6556753b80 1 -- 172.21.15.99:0/3718051172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 msgr2=0x559d328331f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.379+0000 7f6556753b80 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 0x559d328331f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f652c0ca5f0 tx=0x7f652c016fa0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.380+0000 7f6556753b80 1 -- 172.21.15.99:0/3718051172 shutdown_connections 2024-03-30T00:37:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.380+0000 7f6556753b80 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x559d3283d280 0x559d3283f6a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.380+0000 7f6556753b80 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d3283a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.380+0000 7f6556753b80 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829200 0x559d328331f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.380+0000 7f6556753b80 1 --2- 172.21.15.99:0/3718051172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829b60 0x559d32833730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.380+0000 7f6556753b80 1 -- 172.21.15.99:0/3718051172 >> 172.21.15.99:0/3718051172 conn(0x559d32788640 msgr2=0x559d32827fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.380+0000 7f6525600700 0 client.25796 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.380+0000 7f6556753b80 1 -- 172.21.15.99:0/3718051172 shutdown_connections 2024-03-30T00:37:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.380+0000 7f6556753b80 1 -- 172.21.15.99:0/3718051172 wait complete. 2024-03-30T00:37:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.380+0000 7f6556753b80 20 client.25796 trim_cache size 0 max 16384 2024-03-30T00:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.403+0000 7f6556753b80 1 Processor -- start 2024-03-30T00:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.403+0000 7f6556753b80 1 -- start start 2024-03-30T00:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.404+0000 7f6556753b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829200 0x559d32842fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.404+0000 7f6556753b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 0x559d3281e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.404+0000 7f6556753b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d32848d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.404+0000 7f6556753b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d327ef780 con 0x559d32829200 2024-03-30T00:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.404+0000 7f6556753b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d32828ea0 con 0x559d32829b60 2024-03-30T00:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.404+0000 7f6556753b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d32820c80 con 0x559d3282aa60 2024-03-30T00:37:13.411 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.404+0000 7f653be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829200 0x559d32842fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.404+0000 7f653be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829200 0x559d32842fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60988/0 (socket says 172.21.15.99:60988) 2024-03-30T00:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.404+0000 7f653be00700 1 -- 172.21.15.99:0/1003827599 learned_addr learned my addr 172.21.15.99:0/1003827599 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.404+0000 7f653b400700 1 --2- 172.21.15.99:0/1003827599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 0x559d3281e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.405+0000 7f6540c00700 1 --2- 172.21.15.99:0/1003827599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d32848d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.405+0000 7f653be00700 1 -- 172.21.15.99:0/1003827599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 msgr2=0x559d32848d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.405+0000 7f653be00700 1 --2- 172.21.15.99:0/1003827599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d32848d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.405+0000 7f653be00700 1 -- 172.21.15.99:0/1003827599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 msgr2=0x559d3281e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.405+0000 7f653be00700 1 --2- 172.21.15.99:0/1003827599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 0x559d3281e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.405+0000 7f653be00700 1 -- 172.21.15.99:0/1003827599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d325a8b00 con 0x559d32829200 2024-03-30T00:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.405+0000 7f653be00700 1 --2- 172.21.15.99:0/1003827599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829200 0x559d32842fc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f652c099900 tx=0x7f652c098200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7b313d2fd131c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.405+0000 7f6531c00700 1 -- 172.21.15.99:0/1003827599 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c007dd0 con 0x559d32829200 2024-03-30T00:37:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.405+0000 7f6531c00700 1 -- 172.21.15.99:0/1003827599 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f652c016de0 con 0x559d32829200 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.405+0000 7f6531c00700 1 -- 172.21.15.99:0/1003827599 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c095a40 con 0x559d32829200 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.406+0000 7f653b400700 1 --2- 172.21.15.99:0/1003827599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 0x559d3281e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.406+0000 7f6556753b80 1 -- 172.21.15.99:0/1003827599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829200 msgr2=0x559d32842fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.406+0000 7f6556753b80 1 --2- 172.21.15.99:0/1003827599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829200 0x559d32842fc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f652c099900 tx=0x7f652c098200 comp rx=0 tx=0).stop 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.406+0000 7f6556753b80 1 -- 172.21.15.99:0/1003827599 shutdown_connections 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.406+0000 7f6556753b80 1 --2- 172.21.15.99:0/1003827599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d32848d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.406+0000 7f6556753b80 1 --2- 172.21.15.99:0/1003827599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d32829200 0x559d32842fc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.406+0000 7f6556753b80 1 --2- 172.21.15.99:0/1003827599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 0x559d3281e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.406+0000 7f6556753b80 1 -- 172.21.15.99:0/1003827599 >> 172.21.15.99:0/1003827599 conn(0x559d32788640 msgr2=0x559d32824020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.407+0000 7f6556753b80 1 -- 172.21.15.99:0/1003827599 shutdown_connections 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.407+0000 7f6556753b80 1 -- 172.21.15.99:0/1003827599 wait complete. 2024-03-30T00:37:13.414 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.407+0000 7f6556753b80 1 -- start start 2024-03-30T00:37:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.407+0000 7f6556753b80 1 Processor -- start 2024-03-30T00:37:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f6556753b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 0x559d32848d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f6556753b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d32847470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f6556753b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d328479b0 0x559d32847e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f6556753b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d328493b0 con 0x559d328479b0 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f6556753b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d32891ea0 con 0x559d32829b60 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f6556753b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d32892060 con 0x559d3282aa60 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f6556753b80 10 client.0 _ll_register_callbacks cb 0x559d32720410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f653b400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d32847470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f6540c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d328479b0 0x559d32847e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f653b400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d32847470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:60264/0 (socket says 172.21.15.99:60264) 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f653b400700 1 -- 172.21.15.99:0/1749358204 learned_addr learned my addr 172.21.15.99:0/1749358204 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.408+0000 7f653be00700 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 0x559d32848d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.409+0000 7f653b400700 1 -- 172.21.15.99:0/1749358204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 msgr2=0x559d32848d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.409+0000 7f653b400700 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 0x559d32848d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.409+0000 7f653b400700 1 -- 172.21.15.99:0/1749358204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d328479b0 msgr2=0x559d32847e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.409+0000 7f653b400700 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d328479b0 0x559d32847e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.409+0000 7f653b400700 1 -- 172.21.15.99:0/1749358204 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c1a9a20 con 0x559d3282aa60 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.409+0000 7f6540c00700 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d328479b0 0x559d32847e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.409+0000 7f653be00700 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 0x559d32848d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.409+0000 7f653b400700 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d32847470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6528009f80 tx=0x7f65280047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fba982e27d131bc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.409+0000 7f6525600700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.410+0000 7f6525600700 1 -- 172.21.15.99:0/1749358204 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528026070 con 0x559d3282aa60 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.410+0000 7f6525600700 1 -- 172.21.15.99:0/1749358204 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f6528004020 con 0x559d3282aa60 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.410+0000 7f6556753b80 1 -- 172.21.15.99:0/1749358204 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f652c09a350 con 0x559d3282aa60 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.410+0000 7f6525600700 1 -- 172.21.15.99:0/1749358204 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65280041e0 con 0x559d3282aa60 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.410+0000 7f6556753b80 10 client.25819 fetch_fsmap learned FSMap version 26 2024-03-30T00:37:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.410+0000 7f6556753b80 1 -- 172.21.15.99:0/1749358204 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559d32740e40 con 0x559d3282aa60 2024-03-30T00:37:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.411+0000 7f6525600700 1 -- 172.21.15.99:0/1749358204 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f652c09a350 con 0x559d3282aa60 2024-03-30T00:37:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.411+0000 7f6525600700 1 -- 172.21.15.99:0/1749358204 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6528005200 con 0x559d3282aa60 2024-03-30T00:37:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.411+0000 7f6556753b80 10 client.25819 fetch_fsmap finished waiting for FSMap version 26 2024-03-30T00:37:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.411+0000 7f6556753b80 10 client.25819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.411+0000 7f6556753b80 20 client.25819 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.411+0000 7f6556753b80 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x559d32893b80 0x559d32895fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.411+0000 7f6556753b80 4 client.25819 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:37:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.411+0000 7f6556753b80 1 -- 172.21.15.99:0/1749358204 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559d325c9c90 con 0x559d32893b80 2024-03-30T00:37:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.411+0000 7f653be00700 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x559d32893b80 0x559d32895fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.412+0000 7f6525600700 10 client.25819 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.412+0000 7f653be00700 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x559d32893b80 0x559d32895fa0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b10c16b85d3a1a4 server_cookie=37b1ef1b26caece1 in_seq=0 out_seq=0 2024-03-30T00:37:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.421+0000 7f6525600700 1 -- 172.21.15.99:0/1749358204 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559d325c9c90 con 0x559d32893b80 2024-03-30T00:37:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.421+0000 7f6525600700 10 client.25819 handle_command_reply: tid=0 2024-03-30T00:37:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.421+0000 7f6556753b80 1 client.25819 shutdown 2024-03-30T00:37:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.422+0000 7f6556753b80 1 -- 172.21.15.99:0/1749358204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 msgr2=0x559d32847470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.422+0000 7f6556753b80 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d32847470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6528009f80 tx=0x7f65280047c0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.422+0000 7f6556753b80 1 -- 172.21.15.99:0/1749358204 shutdown_connections 2024-03-30T00:37:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.422+0000 7f6556753b80 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x559d32893b80 0x559d32895fa0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.422+0000 7f6556753b80 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d328479b0 0x559d32847e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.422+0000 7f6556753b80 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559d32829b60 0x559d32848d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.422+0000 7f6556753b80 1 --2- 172.21.15.99:0/1749358204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559d3282aa60 0x559d32847470 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.422+0000 7f6556753b80 1 -- 172.21.15.99:0/1749358204 >> 172.21.15.99:0/1749358204 conn(0x559d32788640 msgr2=0x559d32832700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.422+0000 7f6525600700 0 client.25819 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.422+0000 7f6556753b80 1 -- 172.21.15.99:0/1749358204 shutdown_connections 2024-03-30T00:37:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.422+0000 7f6556753b80 1 -- 172.21.15.99:0/1749358204 wait complete. 2024-03-30T00:37:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.423+0000 7f6556753b80 20 client.25819 trim_cache size 0 max 16384 2024-03-30T00:37:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:37:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:37:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:37:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:37:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.424+0000 7f6542c00700 1 -- 172.21.15.99:0/1629580258 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f65200a1270 msgr2=0x7f65200a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.424+0000 7f6542c00700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f65200a1270 0x7f65200a3730 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6528009d70 tx=0x7f6528009260 comp rx=0 tx=0).stop 2024-03-30T00:37:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.424+0000 7f6542c00700 1 -- 172.21.15.99:0/1629580258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c100d10 msgr2=0x7f653c06f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.424+0000 7f6542c00700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c100d10 0x7f653c06f7e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f652c00c990 tx=0x7f652c00cca0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.424+0000 7f6542c00700 1 -- 172.21.15.99:0/1629580258 shutdown_connections 2024-03-30T00:37:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.424+0000 7f6542c00700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f65200a1270 0x7f65200a3730 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.424+0000 7f6542c00700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f653c1028a0 0x7f653c1a8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.424+0000 7f6542c00700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f653c100d10 0x7f653c06f7e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.424+0000 7f6542c00700 1 --2- 172.21.15.99:0/1629580258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f653c101f00 0x7f653c06fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.424+0000 7f6542c00700 1 -- 172.21.15.99:0/1629580258 >> 172.21.15.99:0/1629580258 conn(0x7f653c0fc280 msgr2=0x7f653c105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.425+0000 7f6542c00700 1 -- 172.21.15.99:0/1629580258 shutdown_connections 2024-03-30T00:37:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:13.425+0000 7f6542c00700 1 -- 172.21.15.99:0/1629580258 wait complete. 2024-03-30T00:37:14.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:14 smithi161 ceph-mon[42221]: pgmap v379: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-03-30T00:37:14.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:14 smithi099 ceph-mon[38079]: pgmap v379: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-03-30T00:37:14.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:14 smithi142 ceph-mon[41779]: pgmap v379: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-03-30T00:37:16.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:16 smithi161 ceph-mon[42221]: pgmap v380: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 2.87k op/s 2024-03-30T00:37:16.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:16 smithi099 ceph-mon[38079]: pgmap v380: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 2.87k op/s 2024-03-30T00:37:16.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:16 smithi142 ceph-mon[41779]: pgmap v380: 97 pgs: 97 active+clean; 258 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 2.87k op/s 2024-03-30T00:37:18.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:18 smithi161 ceph-mon[42221]: pgmap v381: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 MiB/s wr, 2.69k op/s 2024-03-30T00:37:18.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:18 smithi099 ceph-mon[38079]: pgmap v381: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 MiB/s wr, 2.69k op/s 2024-03-30T00:37:18.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:18 smithi142 ceph-mon[41779]: pgmap v381: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 MiB/s wr, 2.69k op/s 2024-03-30T00:37:20.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:20 smithi161 ceph-mon[42221]: pgmap v382: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 14 MiB/s wr, 2.98k op/s 2024-03-30T00:37:20.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:20 smithi099 ceph-mon[38079]: pgmap v382: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 14 MiB/s wr, 2.98k op/s 2024-03-30T00:37:20.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:20 smithi142 ceph-mon[41779]: pgmap v382: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 14 MiB/s wr, 2.98k op/s 2024-03-30T00:37:22.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:22 smithi161 ceph-mon[42221]: pgmap v383: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 MiB/s wr, 2.75k op/s 2024-03-30T00:37:22.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:22 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:37:22.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:22 smithi099 ceph-mon[38079]: pgmap v383: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 MiB/s wr, 2.75k op/s 2024-03-30T00:37:22.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:22 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:37:22.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:22 smithi142 ceph-mon[41779]: pgmap v383: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 MiB/s wr, 2.75k op/s 2024-03-30T00:37:22.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:22 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T00:37:24.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:23 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:37:24.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:23 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:37:24.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:23 smithi161 ceph-mon[42221]: pgmap v384: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 MiB/s wr, 2.76k op/s 2024-03-30T00:37:24.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:23 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:37:24.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:23 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:37:24.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:23 smithi099 ceph-mon[38079]: pgmap v384: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 MiB/s wr, 2.76k op/s 2024-03-30T00:37:24.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:23 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:37:24.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:23 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:37:24.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:23 smithi142 ceph-mon[41779]: pgmap v384: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 MiB/s wr, 2.76k op/s 2024-03-30T00:37:25.218 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:24 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:37:25.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:24 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:37:25.219 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:24 smithi161 ceph-mon[42221]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:37:25.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:24 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:37:25.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:24 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:37:25.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:24 smithi099 ceph-mon[38079]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:37:25.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:24 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T00:37:25.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:24 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T00:37:25.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:24 smithi142 ceph-mon[41779]: from='mgr.14150 172.21.15.99:0/667609488' entity='mgr.x' 2024-03-30T00:37:26.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:26 smithi161 ceph-mon[42221]: pgmap v385: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 2.92k op/s 2024-03-30T00:37:26.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:26 smithi099 ceph-mon[38079]: pgmap v385: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 2.92k op/s 2024-03-30T00:37:26.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:26 smithi142 ceph-mon[41779]: pgmap v385: 97 pgs: 97 active+clean; 258 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 2.92k op/s 2024-03-30T00:37:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.638+0000 7f28db600700 1 -- 172.21.15.99:0/778251298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4101f00 msgr2=0x7f28d4102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.638+0000 7f28db600700 1 --2- 172.21.15.99:0/778251298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4101f00 0x7f28d4102360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009920 tx=0x7f28c0009c30 comp rx=0 tx=0).stop 2024-03-30T00:37:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.639+0000 7f28db600700 1 -- 172.21.15.99:0/778251298 shutdown_connections 2024-03-30T00:37:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.639+0000 7f28db600700 1 --2- 172.21.15.99:0/778251298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d41028a0 0x7f28d4109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.639+0000 7f28db600700 1 --2- 172.21.15.99:0/778251298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28d4100d10 0x7f28d4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.639+0000 7f28db600700 1 --2- 172.21.15.99:0/778251298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4101f00 0x7f28d4102360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.639+0000 7f28db600700 1 -- 172.21.15.99:0/778251298 >> 172.21.15.99:0/778251298 conn(0x7f28d40fc2a0 msgr2=0x7f28d40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.639+0000 7f28db600700 1 -- 172.21.15.99:0/778251298 shutdown_connections 2024-03-30T00:37:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.639+0000 7f28db600700 1 -- 172.21.15.99:0/778251298 wait complete. 2024-03-30T00:37:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.641+0000 7f28db600700 1 Processor -- start 2024-03-30T00:37:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.641+0000 7f28db600700 1 -- start start 2024-03-30T00:37:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.641+0000 7f28db600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100d10 0x7f28d4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.641+0000 7f28db600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4101f00 0x7f28d41993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.641+0000 7f28db600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28d41028a0 0x7f28d41a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.641+0000 7f28db600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41a09c0 con 0x7f28d41028a0 2024-03-30T00:37:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.641+0000 7f28db600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41a0b00 con 0x7f28d4101f00 2024-03-30T00:37:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.641+0000 7f28db600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d41a0c90 con 0x7f28d4100d10 2024-03-30T00:37:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d8c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100d10 0x7f28d4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d8c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100d10 0x7f28d4198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:44260/0 (socket says 172.21.15.99:44260) 2024-03-30T00:37:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d8c00700 1 -- 172.21.15.99:0/1546666010 learned_addr learned my addr 172.21.15.99:0/1546666010 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d3e00700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4101f00 0x7f28d41993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d9600700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28d41028a0 0x7f28d41a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d8c00700 1 -- 172.21.15.99:0/1546666010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4101f00 msgr2=0x7f28d41993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d8c00700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4101f00 0x7f28d41993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d8c00700 1 -- 172.21.15.99:0/1546666010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28d41028a0 msgr2=0x7f28d41a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d8c00700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28d41028a0 0x7f28d41a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d8c00700 1 -- 172.21.15.99:0/1546666010 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80095f0 con 0x7f28d4100d10 2024-03-30T00:37:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d9600700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28d41028a0 0x7f28d41a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.642+0000 7f28d8c00700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100d10 0x7f28d4198e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f28c800ebf0 tx=0x7f28c800ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.643+0000 7f28d1600700 1 -- 172.21.15.99:0/1546666010 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8009a10 con 0x7f28d4100d10 2024-03-30T00:37:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.643+0000 7f28d1600700 1 -- 172.21.15.99:0/1546666010 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c800cd40 con 0x7f28d4100d10 2024-03-30T00:37:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.643+0000 7f28d1600700 1 -- 172.21.15.99:0/1546666010 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8018700 con 0x7f28d4100d10 2024-03-30T00:37:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.643+0000 7f28db600700 1 -- 172.21.15.99:0/1546666010 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c0009520 con 0x7f28d4100d10 2024-03-30T00:37:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.643+0000 7f28db600700 1 -- 172.21.15.99:0/1546666010 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d41a11b0 con 0x7f28d4100d10 2024-03-30T00:37:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.647+0000 7f28d1600700 1 -- 172.21.15.99:0/1546666010 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f28c8010430 con 0x7f28d4100d10 2024-03-30T00:37:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.649+0000 7f28ef080b80 1 Processor -- start 2024-03-30T00:37:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.649+0000 7f28ef080b80 1 -- start start 2024-03-30T00:37:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28d1600700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f28bc0a11a0 0x7f28bc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28d1600700 1 -- 172.21.15.99:0/1546666010 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f28c8014070 con 0x7f28d4100d10 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff04e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff05370 0x5646dff05ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff06700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28d9600700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f28bc0a11a0 0x7f28bc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28ef080b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dfe54930 con 0x5646dff05370 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28ef080b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dfe522f0 con 0x5646dff06270 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28ef080b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dfe4dbe0 con 0x5646dff04a10 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28d3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff05370 0x5646dff05ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28d3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff05370 0x5646dff05ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57938/0 (socket says 172.21.15.99:57938) 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.650+0000 7f28d3e00700 1 -- 172.21.15.99:0/3225815485 learned_addr learned my addr 172.21.15.99:0/3225815485 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.651+0000 7f28d9600700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f28bc0a11a0 0x7f28bc0a3660 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009400 tx=0x7f28cc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.651+0000 7f28d9600700 1 --2- 172.21.15.99:0/3225815485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff06700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.651+0000 7f28d8c00700 1 --2- 172.21.15.99:0/3225815485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff04e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.651+0000 7f28d3e00700 1 -- 172.21.15.99:0/3225815485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 msgr2=0x5646dff04e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.651+0000 7f28d3e00700 1 --2- 172.21.15.99:0/3225815485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff04e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.651+0000 7f28d3e00700 1 -- 172.21.15.99:0/3225815485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 msgr2=0x5646dff06700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.651+0000 7f28d3e00700 1 --2- 172.21.15.99:0/3225815485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff06700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.651+0000 7f28d3e00700 1 -- 172.21.15.99:0/3225815485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646dfccf350 con 0x5646dff05370 2024-03-30T00:37:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.652+0000 7f28d3e00700 1 --2- 172.21.15.99:0/3225815485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff05370 0x5646dff05ce0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f28c0000c00 tx=0x7f28c0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf124f6b9a0dbc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.652+0000 7f28c5c00700 1 -- 172.21.15.99:0/3225815485 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c000fda0 con 0x5646dff05370 2024-03-30T00:37:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.652+0000 7f28c5c00700 1 -- 172.21.15.99:0/3225815485 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c001d040 con 0x5646dff05370 2024-03-30T00:37:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.652+0000 7f28c5c00700 1 -- 172.21.15.99:0/3225815485 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c00052a0 con 0x5646dff05370 2024-03-30T00:37:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.653+0000 7f28ef080b80 1 -- 172.21.15.99:0/3225815485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff05370 msgr2=0x5646dff05ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.653+0000 7f28ef080b80 1 --2- 172.21.15.99:0/3225815485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff05370 0x5646dff05ce0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f28c0000c00 tx=0x7f28c0009f80 comp rx=0 tx=0).stop 2024-03-30T00:37:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.653+0000 7f28ef080b80 1 -- 172.21.15.99:0/3225815485 shutdown_connections 2024-03-30T00:37:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.653+0000 7f28ef080b80 1 --2- 172.21.15.99:0/3225815485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff06700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.653+0000 7f28ef080b80 1 --2- 172.21.15.99:0/3225815485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff04e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.653+0000 7f28ef080b80 1 --2- 172.21.15.99:0/3225815485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff05370 0x5646dff05ce0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.653+0000 7f28ef080b80 1 -- 172.21.15.99:0/3225815485 >> 172.21.15.99:0/3225815485 conn(0x5646dfe5b640 msgr2=0x5646dfef7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.653+0000 7f28ef080b80 1 -- 172.21.15.99:0/3225815485 shutdown_connections 2024-03-30T00:37:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.653+0000 7f28ef080b80 1 -- 172.21.15.99:0/3225815485 wait complete. 2024-03-30T00:37:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.654+0000 7f28ef080b80 1 -- start start 2024-03-30T00:37:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.654+0000 7f28ef080b80 1 Processor -- start 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.655+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff04a10 0x5646dfefd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.655+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff05370 0x5646dfefdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.655+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff06270 0x5646dff02f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.655+0000 7f28ef080b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dfde9d20 con 0x5646dff04a10 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.655+0000 7f28ef080b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dfde99c0 con 0x5646dff05370 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.655+0000 7f28ef080b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dfcd1640 con 0x5646dff06270 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.655+0000 7f28ef080b80 10 client.0 _ll_register_callbacks cb 0x5646dfdf3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.655+0000 7f28d3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff05370 0x5646dfefdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.655+0000 7f28d3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff05370 0x5646dfefdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:53730/0 (socket says 172.21.15.99:53730) 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.655+0000 7f28d3e00700 1 -- 172.21.15.99:0/2806069354 learned_addr learned my addr 172.21.15.99:0/2806069354 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.656+0000 7f28d3e00700 1 -- 172.21.15.99:0/2806069354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff06270 msgr2=0x5646dff02f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.656+0000 7f28d9600700 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff06270 0x5646dff02f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.656+0000 7f28d3e00700 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff06270 0x5646dff02f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.656+0000 7f28d3e00700 1 -- 172.21.15.99:0/2806069354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff04a10 msgr2=0x5646dfefd9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:37:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.656+0000 7f28d8c00700 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff04a10 0x5646dfefd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.656+0000 7f28d3e00700 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff04a10 0x5646dfefd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.656+0000 7f28d3e00700 1 -- 172.21.15.99:0/2806069354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d4109aa0 con 0x5646dff05370 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.657+0000 7f28d9600700 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff06270 0x5646dff02f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.657+0000 7f28d8c00700 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff04a10 0x5646dfefd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.657+0000 7f28d3e00700 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff05370 0x5646dfefdf10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f28c000b990 tx=0x7f28c000ff00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0ddbd2197e49963 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.657+0000 7f28ba000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.657+0000 7f28ba000700 1 -- 172.21.15.99:0/2806069354 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0005040 con 0x5646dff05370 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.657+0000 7f28ef080b80 1 -- 172.21.15.99:0/2806069354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646dfe0e320 con 0x5646dff05370 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.657+0000 7f28ba000700 1 -- 172.21.15.99:0/2806069354 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f28c00051a0 con 0x5646dff05370 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.658+0000 7f28ba000700 1 -- 172.21.15.99:0/2806069354 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28c0005360 con 0x5646dff05370 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.658+0000 7f28ef080b80 10 client.25814 fetch_fsmap learned FSMap version 26 2024-03-30T00:37:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.658+0000 7f28ef080b80 1 -- 172.21.15.99:0/2806069354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5646dfc7bb00 con 0x5646dff05370 2024-03-30T00:37:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.658+0000 7f28ba000700 1 -- 172.21.15.99:0/2806069354 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5646dfe0e320 con 0x5646dff05370 2024-03-30T00:37:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.659+0000 7f28ba000700 1 -- 172.21.15.99:0/2806069354 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f28c0009ca0 con 0x5646dff05370 2024-03-30T00:37:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.659+0000 7f28ef080b80 10 client.25814 fetch_fsmap finished waiting for FSMap version 26 2024-03-30T00:37:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.659+0000 7f28ef080b80 10 client.25814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.659+0000 7f28ef080b80 20 client.25814 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.659+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5646dff22a80 0x5646dff24e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.659+0000 7f28ef080b80 4 client.25814 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:37:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.659+0000 7f28ef080b80 1 -- 172.21.15.99:0/2806069354 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646dfe466a0 con 0x5646dff22a80 2024-03-30T00:37:26.667 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.659+0000 7f28d8c00700 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5646dff22a80 0x5646dff24e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.660+0000 7f28ba000700 10 client.25814 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.660+0000 7f28d8c00700 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5646dff22a80 0x5646dff24e70 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4158a1193e4aa10 server_cookie=c9efb41108721f8a in_seq=0 out_seq=0 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ba000700 1 -- 172.21.15.99:0/2806069354 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5646dfe466a0 con 0x5646dff22a80 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ba000700 10 client.25814 handle_command_reply: tid=0 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ef080b80 1 client.25814 shutdown 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ef080b80 1 -- 172.21.15.99:0/2806069354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff05370 msgr2=0x5646dfefdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff05370 0x5646dfefdf10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f28c000b990 tx=0x7f28c000ff00 comp rx=0 tx=0).stop 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ef080b80 1 -- 172.21.15.99:0/2806069354 shutdown_connections 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5646dff22a80 0x5646dff24e70 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff06270 0x5646dff02f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dff04a10 0x5646dfefd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2806069354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff05370 0x5646dfefdf10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ef080b80 1 -- 172.21.15.99:0/2806069354 >> 172.21.15.99:0/2806069354 conn(0x5646dfe5b640 msgr2=0x5646dfefafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ba000700 0 client.25814 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ef080b80 1 -- 172.21.15.99:0/2806069354 shutdown_connections 2024-03-30T00:37:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.661+0000 7f28ef080b80 1 -- 172.21.15.99:0/2806069354 wait complete. 2024-03-30T00:37:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.662+0000 7f28ef080b80 20 client.25814 trim_cache size 0 max 16384 2024-03-30T00:37:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.687+0000 7f28ef080b80 1 Processor -- start 2024-03-30T00:37:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.687+0000 7f28ef080b80 1 -- start start 2024-03-30T00:37:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.687+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff09620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.687+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff035f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.687+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dfefcee0 0x5646dfef1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.687+0000 7f28ef080b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dfefe680 con 0x5646dfefcee0 2024-03-30T00:37:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.687+0000 7f28ef080b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dff06650 con 0x5646dff06270 2024-03-30T00:37:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.687+0000 7f28ef080b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dfefc520 con 0x5646dff04a10 2024-03-30T00:37:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.688+0000 7f28d8c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff09620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.688+0000 7f28d8c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff09620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:44282/0 (socket says 172.21.15.99:44282) 2024-03-30T00:37:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.688+0000 7f28d8c00700 1 -- 172.21.15.99:0/1663710413 learned_addr learned my addr 172.21.15.99:0/1663710413 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.688+0000 7f28d3e00700 1 --2- 172.21.15.99:0/1663710413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.688+0000 7f28d8c00700 1 -- 172.21.15.99:0/1663710413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 msgr2=0x5646dff035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.688+0000 7f28d8c00700 1 --2- 172.21.15.99:0/1663710413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.688+0000 7f28d8c00700 1 -- 172.21.15.99:0/1663710413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dfefcee0 msgr2=0x5646dfef1500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T00:37:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.688+0000 7f28d9600700 1 --2- 172.21.15.99:0/1663710413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dfefcee0 0x5646dfef1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.688+0000 7f28d8c00700 1 --2- 172.21.15.99:0/1663710413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dfefcee0 0x5646dfef1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.688+0000 7f28d8c00700 1 -- 172.21.15.99:0/1663710413 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646dfe13e40 con 0x5646dff04a10 2024-03-30T00:37:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.689+0000 7f28d9600700 1 --2- 172.21.15.99:0/1663710413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dfefcee0 0x5646dfef1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:37:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.689+0000 7f28d8c00700 1 --2- 172.21.15.99:0/1663710413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff09620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f28c80ca8d0 tx=0x7f28c80ca7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be67405c5fcac0bc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.689+0000 7f28c6600700 1 -- 172.21.15.99:0/1663710413 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c809e050 con 0x5646dff04a10 2024-03-30T00:37:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.689+0000 7f28c6600700 1 -- 172.21.15.99:0/1663710413 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c80a26a0 con 0x5646dff04a10 2024-03-30T00:37:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.689+0000 7f28c6600700 1 -- 172.21.15.99:0/1663710413 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c80a2210 con 0x5646dff04a10 2024-03-30T00:37:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.690+0000 7f28ef080b80 1 -- 172.21.15.99:0/1663710413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 msgr2=0x5646dff09620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.690+0000 7f28ef080b80 1 --2- 172.21.15.99:0/1663710413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff09620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f28c80ca8d0 tx=0x7f28c80ca7a0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.690+0000 7f28ef080b80 1 -- 172.21.15.99:0/1663710413 shutdown_connections 2024-03-30T00:37:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.690+0000 7f28ef080b80 1 --2- 172.21.15.99:0/1663710413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dfefcee0 0x5646dfef1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.690+0000 7f28ef080b80 1 --2- 172.21.15.99:0/1663710413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff09620 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.690+0000 7f28ef080b80 1 --2- 172.21.15.99:0/1663710413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.690+0000 7f28ef080b80 1 -- 172.21.15.99:0/1663710413 >> 172.21.15.99:0/1663710413 conn(0x5646dfe5b640 msgr2=0x5646dff08f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.690+0000 7f28ef080b80 1 -- 172.21.15.99:0/1663710413 shutdown_connections 2024-03-30T00:37:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.690+0000 7f28ef080b80 1 -- 172.21.15.99:0/1663710413 wait complete. 2024-03-30T00:37:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.691+0000 7f28ef080b80 1 -- start start 2024-03-30T00:37:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.691+0000 7f28ef080b80 1 Processor -- start 2024-03-30T00:37:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.692+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dfefcee0 0x5646dff2e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.692+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff2ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.692+0000 7f28ef080b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff31960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.692+0000 7f28ef080b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dff31ea0 con 0x5646dfefcee0 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.692+0000 7f28ef080b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dff32010 con 0x5646dff06270 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.692+0000 7f28ef080b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646dff321d0 con 0x5646dff04a10 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.693+0000 7f28d9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff31960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.693+0000 7f28ef080b80 10 client.0 _ll_register_callbacks cb 0x5646dfdf3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.693+0000 7f28d9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff31960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:53750/0 (socket says 172.21.15.99:53750) 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.693+0000 7f28d9600700 1 -- 172.21.15.99:0/2882864203 learned_addr learned my addr 172.21.15.99:0/2882864203 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.693+0000 7f28d3e00700 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff2ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.693+0000 7f28d8c00700 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dfefcee0 0x5646dff2e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.694+0000 7f28d3e00700 1 -- 172.21.15.99:0/2882864203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 msgr2=0x5646dff31960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.694+0000 7f28d3e00700 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff31960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.694+0000 7f28d3e00700 1 -- 172.21.15.99:0/2882864203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dfefcee0 msgr2=0x5646dff2e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.694+0000 7f28d3e00700 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dfefcee0 0x5646dff2e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.702 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.694+0000 7f28d3e00700 1 -- 172.21.15.99:0/2882864203 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d41a11b0 con 0x5646dff04a10 2024-03-30T00:37:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.695+0000 7f28d3e00700 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff2ed50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5646dff05960 tx=0x7f28c002a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f663eaa25c9ac3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.695+0000 7f28ba000700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:37:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.695+0000 7f28ba000700 1 -- 172.21.15.99:0/2882864203 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0009e30 con 0x5646dff04a10 2024-03-30T00:37:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.695+0000 7f28ba000700 1 -- 172.21.15.99:0/2882864203 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f28c002a880 con 0x5646dff04a10 2024-03-30T00:37:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.695+0000 7f28ef080b80 1 -- 172.21.15.99:0/2882864203 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28c001d040 con 0x5646dff04a10 2024-03-30T00:37:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.695+0000 7f28ba000700 1 -- 172.21.15.99:0/2882864203 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28c002aa40 con 0x5646dff04a10 2024-03-30T00:37:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.695+0000 7f28ef080b80 10 client.25834 fetch_fsmap learned FSMap version 26 2024-03-30T00:37:26.703 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.695+0000 7f28ef080b80 1 -- 172.21.15.99:0/2882864203 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5646dfcd3c70 con 0x5646dff04a10 2024-03-30T00:37:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.696+0000 7f28ba000700 1 -- 172.21.15.99:0/2882864203 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c001d040 con 0x5646dff04a10 2024-03-30T00:37:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.696+0000 7f28ba000700 1 -- 172.21.15.99:0/2882864203 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 26) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f28c0045050 con 0x5646dff04a10 2024-03-30T00:37:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.696+0000 7f28ef080b80 10 client.25834 fetch_fsmap finished waiting for FSMap version 26 2024-03-30T00:37:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.696+0000 7f28ef080b80 10 client.25834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.696+0000 7f28ef080b80 20 client.25834 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.696+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5646dff888e0 0x5646dff8ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.697+0000 7f28ef080b80 4 client.25834 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-30T00:37:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.697+0000 7f28ef080b80 1 -- 172.21.15.99:0/2882864203 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5646dfc9cc90 con 0x5646dff888e0 2024-03-30T00:37:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.698+0000 7f28d8c00700 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5646dff888e0 0x5646dff8ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.698+0000 7f28ba000700 10 client.25834 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.698+0000 7f28d8c00700 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5646dff888e0 0x5646dff8ad00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=677040bed795e4e9 server_cookie=cd7dfb14dde9b658 in_seq=0 out_seq=0 2024-03-30T00:37:26.716 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.709+0000 7f28ba000700 1 -- 172.21.15.99:0/2882864203 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5646dfc9cc90 con 0x5646dff888e0 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.709+0000 7f28ba000700 10 client.25834 handle_command_reply: tid=0 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.709+0000 7f28ef080b80 1 client.25834 shutdown 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.709+0000 7f28ef080b80 1 -- 172.21.15.99:0/2882864203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 msgr2=0x5646dff2ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.709+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff2ed50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5646dff05960 tx=0x7f28c002a640 comp rx=0 tx=0).stop 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.710+0000 7f28ef080b80 1 -- 172.21.15.99:0/2882864203 shutdown_connections 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.710+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5646dff888e0 0x5646dff8ad00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.710+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646dff06270 0x5646dff31960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.710+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5646dfefcee0 0x5646dff2e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.710+0000 7f28ba000700 0 client.25834 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.710+0000 7f28ef080b80 1 --2- 172.21.15.99:0/2882864203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dff04a10 0x5646dff2ed50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.710+0000 7f28ef080b80 1 -- 172.21.15.99:0/2882864203 >> 172.21.15.99:0/2882864203 conn(0x5646dfe5b640 msgr2=0x5646dfefd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.710+0000 7f28ef080b80 1 -- 172.21.15.99:0/2882864203 shutdown_connections 2024-03-30T00:37:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.710+0000 7f28ef080b80 1 -- 172.21.15.99:0/2882864203 wait complete. 2024-03-30T00:37:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.711+0000 7f28ef080b80 20 client.25834 trim_cache size 0 max 16384 2024-03-30T00:37:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:{ 2024-03-30T00:37:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "message": "", 2024-03-30T00:37:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout: "return_code": 0 2024-03-30T00:37:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stdout:} 2024-03-30T00:37:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.712+0000 7f28db600700 1 -- 172.21.15.99:0/1546666010 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f28bc0a11a0 msgr2=0x7f28bc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.712+0000 7f28db600700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f28bc0a11a0 0x7f28bc0a3660 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009400 tx=0x7f28cc00b040 comp rx=0 tx=0).stop 2024-03-30T00:37:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.712+0000 7f28db600700 1 -- 172.21.15.99:0/1546666010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100d10 msgr2=0x7f28d4198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.712+0000 7f28db600700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100d10 0x7f28d4198e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f28c800ebf0 tx=0x7f28c800ef00 comp rx=0 tx=0).stop 2024-03-30T00:37:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.712+0000 7f28db600700 1 -- 172.21.15.99:0/1546666010 shutdown_connections 2024-03-30T00:37:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.712+0000 7f28db600700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28d41028a0 0x7f28d41a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.712+0000 7f28db600700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28d4100d10 0x7f28d4198e90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.712+0000 7f28db600700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f28bc0a11a0 0x7f28bc0a3660 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.712+0000 7f28db600700 1 --2- 172.21.15.99:0/1546666010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28d4101f00 0x7f28d41993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.712+0000 7f28db600700 1 -- 172.21.15.99:0/1546666010 >> 172.21.15.99:0/1546666010 conn(0x7f28d40fc2a0 msgr2=0x7f28d4107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.712+0000 7f28db600700 1 -- 172.21.15.99:0/1546666010 shutdown_connections 2024-03-30T00:37:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi099.front.sepia.ceph.com.smithi099.stderr:2024-03-30T00:37:26.713+0000 7f28db600700 1 -- 172.21.15.99:0/1546666010 wait complete. 2024-03-30T00:37:28.460 INFO:tasks.workunit.client.0.smithi099.stdout:setting up random write file 2024-03-30T00:37:28.460 INFO:tasks.workunit.client.0.smithi099.stdout:done setting up random write file 2024-03-30T00:37:28.460 INFO:tasks.workunit.client.0.smithi099.stdout:starting fsync run 2024-03-30T00:37:28.460 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0139s fsync time: 0.0184s 2024-03-30T00:37:28.460 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0048s fsync time: 0.0699s 2024-03-30T00:37:28.460 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0038s fsync time: 0.0373s 2024-03-30T00:37:28.460 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0050s fsync time: 0.0205s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0036s fsync time: 0.0214s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0038s fsync time: 0.0114s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0048s fsync time: 0.0171s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0035s fsync time: 0.0212s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0036s fsync time: 0.0145s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0039s fsync time: 0.0148s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0037s fsync time: 0.0156s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0024s fsync time: 0.0177s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0051s fsync time: 0.0120s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0036s fsync time: 0.0214s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0029s fsync time: 0.0198s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0031s fsync time: 0.0239s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0024s fsync time: 0.0303s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0037s fsync time: 0.0216s 2024-03-30T00:37:28.461 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0023s fsync time: 0.0371s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0055s fsync time: 0.0117s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0026s fsync time: 0.0247s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0038s fsync time: 0.0132s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0031s fsync time: 0.0173s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0033s fsync time: 0.0220s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0033s fsync time: 0.0164s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0031s fsync time: 0.0402s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0029s fsync time: 0.0164s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0065s fsync time: 0.0157s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0035s fsync time: 0.0132s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0055s fsync time: 0.0146s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0036s fsync time: 0.0185s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0078s fsync time: 0.0194s 2024-03-30T00:37:28.462 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0035s fsync time: 0.0165s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0045s fsync time: 0.0207s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0029s fsync time: 0.0170s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0027s fsync time: 0.0220s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0050s fsync time: 0.0148s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0029s fsync time: 0.0129s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0052s fsync time: 0.0136s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0033s fsync time: 0.0150s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0036s fsync time: 0.0200s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0036s fsync time: 0.0221s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0054s fsync time: 0.0194s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0054s fsync time: 0.0152s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0026s fsync time: 0.0173s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0040s fsync time: 0.0167s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0027s fsync time: 0.0153s 2024-03-30T00:37:28.463 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0041s fsync time: 0.0132s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0028s fsync time: 0.0198s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0027s fsync time: 0.0306s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0025s fsync time: 0.0219s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0036s fsync time: 0.0139s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0059s fsync time: 0.0210s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0033s fsync time: 0.0143s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0079s fsync time: 0.0155s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0035s fsync time: 0.0188s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0048s fsync time: 0.0169s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0058s fsync time: 0.0179s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0029s fsync time: 0.0229s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:write time: 0.0025s fsync time: 0.0209s 2024-03-30T00:37:28.464 INFO:tasks.workunit.client.0.smithi099.stdout:run done 60 fsyncs total, killing random writer 2024-03-30T00:37:28.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:28 smithi161 ceph-mon[42221]: pgmap v386: 97 pgs: 97 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 12 MiB/s wr, 2.67k op/s 2024-03-30T00:37:28.470 INFO:tasks.workunit.client.0.smithi099.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-03-30T00:37:28.470 INFO:tasks.workunit.client.0.smithi099.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-03-30T00:37:28.471 INFO:tasks.workunit.client.0.smithi099.stderr:+ whereis lsof 2024-03-30T00:37:28.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:28 smithi099 ceph-mon[38079]: pgmap v386: 97 pgs: 97 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 12 MiB/s wr, 2.67k op/s 2024-03-30T00:37:28.528 INFO:tasks.workunit.client.0.smithi099.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-03-30T00:37:28.528 INFO:tasks.workunit.client.0.smithi099.stderr:+ lsof 2024-03-30T00:37:28.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:28 smithi142 ceph-mon[41779]: pgmap v386: 97 pgs: 97 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 12 MiB/s wr, 2.67k op/s 2024-03-30T00:37:28.621 INFO:tasks.workunit.client.0.smithi099.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/dca03cd01cf474c9d0ff124f9def38fb18c65cabe0a1f4178eadfefa6588cc2c/merged 2024-03-30T00:37:28.622 INFO:tasks.workunit.client.0.smithi099.stderr: Output information may be incomplete. 2024-03-30T00:37:28.622 INFO:tasks.workunit.client.0.smithi099.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/fa28250e4f0204586976312a5c2ebeefdeda09da1f3477690eb8f8f000289807/merged 2024-03-30T00:37:28.622 INFO:tasks.workunit.client.0.smithi099.stderr: Output information may be incomplete. 2024-03-30T00:37:28.630 INFO:tasks.workunit.client.0.smithi099.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/564d524d669399834b5ce5030008f1fedd3748369ba489cc10c943a203ffdf32/merged 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr: Output information may be incomplete. 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/867ff4192057d493fc43cebd80aa8b68f0a2e4499dabadcadcaabd7e81d4059a/merged 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr: Output information may be incomplete. 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/23bf7061932560374ea40279cc89903dd71cca9ad38c54784ff20952d2acfe14/merged 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr: Output information may be incomplete. 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/bce47d7b9224a47ec7361c428a6600849bbc414feeafdcdc6a4d5c2e678bd5ef/merged 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr: Output information may be incomplete. 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/6ae0bc3140239dc43af6c2ac0d2d6da7de181859eb240344be2b73427b19e2c1/merged 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr: Output information may be incomplete. 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/a2e78de0127c2f549039afec6660fd79f13a5273e969b74ce4729576bf0c819c/merged 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr: Output information may be incomplete. 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/30454ddc3e9a7cc70d8cb34d83682fc9b776f33ae28196000603b2faeed92eb7/merged 2024-03-30T00:37:28.631 INFO:tasks.workunit.client.0.smithi099.stderr: Output information may be incomplete. 2024-03-30T00:37:28.632 INFO:tasks.workunit.client.0.smithi099.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/ec895ba48edfe81b8a99b0cdd68acbcfa783ece44b9106a6a57b22afe955370b/merged 2024-03-30T00:37:28.632 INFO:tasks.workunit.client.0.smithi099.stderr: Output information may be incomplete. 2024-03-30T00:37:28.766 INFO:tasks.workunit.client.0.smithi099.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-03-30T00:37:28.766 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-03-30T00:37:28.766 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-03-30T00:37:28.766 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-03-30T00:37:28.766 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-03-30T00:37:28.766 INFO:tasks.workunit.client.0.smithi099.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-03-30T00:37:28.766 INFO:tasks.workunit.client.0.smithi099.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-03-30T00:37:28.766 INFO:tasks.workunit.client.0.smithi099.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:pool_work 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:pool_work 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:pool_work 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:pool_work 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-03-30T00:37:28.767 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 7 root cwd unknown /proc/7/cwd (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 7 root rtd unknown /proc/7/root (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 7 root txt unknown /proc/7/exe (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 7 root NOFD /proc/7/fd (opendir: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-03-30T00:37:28.768 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_tasks 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-03-30T00:37:28.769 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_preem 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_preem 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_preem 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:rcu_preem 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:migration 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:migration 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:migration 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:migration 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/0 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/0 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/0 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/0 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-03-30T00:37:28.770 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/1 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-03-30T00:37:28.771 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/1 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-03-30T00:37:28.771 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/1 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-03-30T00:37:28.771 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/1 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-03-30T00:37:28.771 INFO:tasks.workunit.client.0.smithi099.stdout:migration 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-03-30T00:37:28.771 INFO:tasks.workunit.client.0.smithi099.stdout:migration 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-03-30T00:37:28.771 INFO:tasks.workunit.client.0.smithi099.stdout:migration 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-03-30T00:37:28.771 INFO:tasks.workunit.client.0.smithi099.stdout:migration 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-03-30T00:37:28.771 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-03-30T00:37:28.771 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-03-30T00:37:28.771 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-03-30T00:37:28.771 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/2 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/2 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/2 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/2 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:migration 26 root cwd unknown /proc/26/cwd (readlink: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:migration 26 root rtd unknown /proc/26/root (readlink: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:migration 26 root txt unknown /proc/26/exe (readlink: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:migration 26 root NOFD /proc/26/fd (opendir: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-03-30T00:37:28.772 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/3 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/3 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/3 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/3 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:migration 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:migration 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:migration 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:migration 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 32 root cwd unknown /proc/32/cwd (readlink: Permission denied) 2024-03-30T00:37:28.773 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 32 root rtd unknown /proc/32/root (readlink: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 32 root txt unknown /proc/32/exe (readlink: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 32 root NOFD /proc/32/fd (opendir: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/4 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/4 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/4 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/4 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:migration 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:migration 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-03-30T00:37:28.774 INFO:tasks.workunit.client.0.smithi099.stdout:migration 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:migration 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 39 root cwd unknown /proc/39/cwd (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 39 root rtd unknown /proc/39/root (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 39 root txt unknown /proc/39/exe (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 39 root NOFD /proc/39/fd (opendir: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/5 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/5 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/5 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/5 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:migration 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:migration 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-03-30T00:37:28.775 INFO:tasks.workunit.client.0.smithi099.stdout:migration 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:migration 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 44 root cwd unknown /proc/44/cwd (readlink: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 44 root rtd unknown /proc/44/root (readlink: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 44 root txt unknown /proc/44/exe (readlink: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 44 root NOFD /proc/44/fd (opendir: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/6 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/6 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/6 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/6 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:migration 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-03-30T00:37:28.776 INFO:tasks.workunit.client.0.smithi099.stdout:migration 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:migration 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:migration 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/7 50 root cwd unknown /proc/50/cwd (readlink: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/7 50 root rtd unknown /proc/50/root (readlink: Permission denied) 2024-03-30T00:37:28.777 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/7 50 root txt unknown /proc/50/exe (readlink: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:cpuhp/7 50 root NOFD /proc/50/fd (opendir: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:migration 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:migration 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:migration 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:migration 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:ksoftirqd 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-03-30T00:37:28.778 INFO:tasks.workunit.client.0.smithi099.stdout:kdevtmpfs 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:kdevtmpfs 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:kdevtmpfs 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:kdevtmpfs 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 56 root cwd unknown /proc/56/cwd (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 56 root rtd unknown /proc/56/root (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 56 root txt unknown /proc/56/exe (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 56 root NOFD /proc/56/fd (opendir: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:kauditd 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:kauditd 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:kauditd 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:kauditd 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:khungtask 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:khungtask 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:khungtask 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-03-30T00:37:28.779 INFO:tasks.workunit.client.0.smithi099.stdout:khungtask 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:oom_reape 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:oom_reape 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:oom_reape 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:oom_reape 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:kcompactd 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:kcompactd 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:kcompactd 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:kcompactd 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:ksmd 64 root cwd unknown /proc/64/cwd (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:ksmd 64 root rtd unknown /proc/64/root (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:ksmd 64 root txt unknown /proc/64/exe (readlink: Permission denied) 2024-03-30T00:37:28.780 INFO:tasks.workunit.client.0.smithi099.stdout:ksmd 64 root NOFD /proc/64/fd (opendir: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:khugepage 65 root cwd unknown /proc/65/cwd (readlink: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:khugepage 65 root rtd unknown /proc/65/root (readlink: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:khugepage 65 root txt unknown /proc/65/exe (readlink: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:khugepage 65 root NOFD /proc/65/fd (opendir: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 67 root cwd unknown /proc/67/cwd (readlink: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 67 root rtd unknown /proc/67/root (readlink: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 67 root txt unknown /proc/67/exe (readlink: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 67 root NOFD /proc/67/fd (opendir: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-03-30T00:37:28.781 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:irq/9-acp 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:irq/9-acp 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:irq/9-acp 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:irq/9-acp 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-03-30T00:37:28.782 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-03-30T00:37:28.783 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:watchdogd 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:watchdogd 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:watchdogd 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:watchdogd 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kswapd0 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kswapd0 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kswapd0 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-03-30T00:37:28.784 INFO:tasks.workunit.client.0.smithi099.stdout:kswapd0 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 92 root cwd unknown /proc/92/cwd (readlink: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 92 root rtd unknown /proc/92/root (readlink: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 92 root txt unknown /proc/92/exe (readlink: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 92 root NOFD /proc/92/fd (opendir: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_0 93 root cwd unknown /proc/93/cwd (readlink: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_0 93 root rtd unknown /proc/93/root (readlink: Permission denied) 2024-03-30T00:37:28.785 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_0 93 root txt unknown /proc/93/exe (readlink: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_0 93 root NOFD /proc/93/fd (opendir: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_1 95 root cwd unknown /proc/95/cwd (readlink: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_1 95 root rtd unknown /proc/95/root (readlink: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_1 95 root txt unknown /proc/95/exe (readlink: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_1 95 root NOFD /proc/95/fd (opendir: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 96 root cwd unknown /proc/96/cwd (readlink: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 96 root rtd unknown /proc/96/root (readlink: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 96 root txt unknown /proc/96/exe (readlink: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 96 root NOFD /proc/96/fd (opendir: Permission denied) 2024-03-30T00:37:28.786 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_2 97 root cwd unknown /proc/97/cwd (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_2 97 root rtd unknown /proc/97/root (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_2 97 root txt unknown /proc/97/exe (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_2 97 root NOFD /proc/97/fd (opendir: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 98 root cwd unknown /proc/98/cwd (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 98 root rtd unknown /proc/98/root (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 98 root txt unknown /proc/98/exe (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 98 root NOFD /proc/98/fd (opendir: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_3 99 root cwd unknown /proc/99/cwd (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_3 99 root rtd unknown /proc/99/root (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_3 99 root txt unknown /proc/99/exe (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_3 99 root NOFD /proc/99/fd (opendir: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 100 root cwd unknown /proc/100/cwd (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 100 root rtd unknown /proc/100/root (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 100 root txt unknown /proc/100/exe (readlink: Permission denied) 2024-03-30T00:37:28.787 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 100 root NOFD /proc/100/fd (opendir: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 103 root cwd unknown /proc/103/cwd (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 103 root rtd unknown /proc/103/root (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 103 root txt unknown /proc/103/exe (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 103 root NOFD /proc/103/fd (opendir: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_4 104 root cwd unknown /proc/104/cwd (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_4 104 root rtd unknown /proc/104/root (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_4 104 root txt unknown /proc/104/exe (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_4 104 root NOFD /proc/104/fd (opendir: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 105 root cwd unknown /proc/105/cwd (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 105 root rtd unknown /proc/105/root (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 105 root txt unknown /proc/105/exe (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 105 root NOFD /proc/105/fd (opendir: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_5 106 root cwd unknown /proc/106/cwd (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_5 106 root rtd unknown /proc/106/root (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_5 106 root txt unknown /proc/106/exe (readlink: Permission denied) 2024-03-30T00:37:28.788 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_5 106 root NOFD /proc/106/fd (opendir: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 107 root cwd unknown /proc/107/cwd (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 107 root rtd unknown /proc/107/root (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 107 root txt unknown /proc/107/exe (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 107 root NOFD /proc/107/fd (opendir: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_6 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_6 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_6 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_6 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_7 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_7 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_7 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-03-30T00:37:28.789 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_7 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_8 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_8 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_8 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_8 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 113 root cwd unknown /proc/113/cwd (readlink: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 113 root rtd unknown /proc/113/root (readlink: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 113 root txt unknown /proc/113/exe (readlink: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 113 root NOFD /proc/113/fd (opendir: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_9 114 root cwd unknown /proc/114/cwd (readlink: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_9 114 root rtd unknown /proc/114/root (readlink: Permission denied) 2024-03-30T00:37:28.790 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_9 114 root txt unknown /proc/114/exe (readlink: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:scsi_eh_9 114 root NOFD /proc/114/fd (opendir: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 116 root cwd unknown /proc/116/cwd (readlink: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 116 root rtd unknown /proc/116/root (readlink: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 116 root txt unknown /proc/116/exe (readlink: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 116 root NOFD /proc/116/fd (opendir: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 117 root cwd unknown /proc/117/cwd (readlink: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 117 root rtd unknown /proc/117/root (readlink: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 117 root txt unknown /proc/117/exe (readlink: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 117 root NOFD /proc/117/fd (opendir: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 127 root cwd unknown /proc/127/cwd (readlink: Permission denied) 2024-03-30T00:37:28.791 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 127 root rtd unknown /proc/127/root (readlink: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 127 root txt unknown /proc/127/exe (readlink: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 127 root NOFD /proc/127/fd (opendir: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 129 root cwd unknown /proc/129/cwd (readlink: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 129 root rtd unknown /proc/129/root (readlink: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 129 root txt unknown /proc/129/exe (readlink: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 129 root NOFD /proc/129/fd (opendir: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 130 root cwd unknown /proc/130/cwd (readlink: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 130 root rtd unknown /proc/130/root (readlink: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 130 root txt unknown /proc/130/exe (readlink: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 130 root NOFD /proc/130/fd (opendir: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 131 root cwd unknown /proc/131/cwd (readlink: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 131 root rtd unknown /proc/131/root (readlink: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 131 root txt unknown /proc/131/exe (readlink: Permission denied) 2024-03-30T00:37:28.792 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 131 root NOFD /proc/131/fd (opendir: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 132 root cwd unknown /proc/132/cwd (readlink: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 132 root rtd unknown /proc/132/root (readlink: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 132 root txt unknown /proc/132/exe (readlink: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 132 root NOFD /proc/132/fd (opendir: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 140 root cwd unknown /proc/140/cwd (readlink: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 140 root rtd unknown /proc/140/root (readlink: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 140 root txt unknown /proc/140/exe (readlink: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 140 root NOFD /proc/140/fd (opendir: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 210 root cwd unknown /proc/210/cwd (readlink: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 210 root rtd unknown /proc/210/root (readlink: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 210 root txt unknown /proc/210/exe (readlink: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 210 root NOFD /proc/210/fd (opendir: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 244 root cwd unknown /proc/244/cwd (readlink: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 244 root rtd unknown /proc/244/root (readlink: Permission denied) 2024-03-30T00:37:28.793 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 244 root txt unknown /proc/244/exe (readlink: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 244 root NOFD /proc/244/fd (opendir: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 246 root cwd unknown /proc/246/cwd (readlink: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 246 root rtd unknown /proc/246/root (readlink: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 246 root txt unknown /proc/246/exe (readlink: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 246 root NOFD /proc/246/fd (opendir: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 270 root cwd unknown /proc/270/cwd (readlink: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 270 root rtd unknown /proc/270/root (readlink: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 270 root txt unknown /proc/270/exe (readlink: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 270 root NOFD /proc/270/fd (opendir: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 271 root cwd unknown /proc/271/cwd (readlink: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 271 root rtd unknown /proc/271/root (readlink: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 271 root txt unknown /proc/271/exe (readlink: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 271 root NOFD /proc/271/fd (opendir: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 277 root cwd unknown /proc/277/cwd (readlink: Permission denied) 2024-03-30T00:37:28.794 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 277 root rtd unknown /proc/277/root (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 277 root txt unknown /proc/277/exe (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 277 root NOFD /proc/277/fd (opendir: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 325 root cwd unknown /proc/325/cwd (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 325 root rtd unknown /proc/325/root (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 325 root txt unknown /proc/325/exe (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 325 root NOFD /proc/325/fd (opendir: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 399 root cwd unknown /proc/399/cwd (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 399 root rtd unknown /proc/399/root (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 399 root txt unknown /proc/399/exe (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 399 root NOFD /proc/399/fd (opendir: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 402 root cwd unknown /proc/402/cwd (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 402 root rtd unknown /proc/402/root (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 402 root txt unknown /proc/402/exe (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 402 root NOFD /proc/402/fd (opendir: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 407 root cwd unknown /proc/407/cwd (readlink: Permission denied) 2024-03-30T00:37:28.795 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 407 root rtd unknown /proc/407/root (readlink: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 407 root txt unknown /proc/407/exe (readlink: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 407 root NOFD /proc/407/fd (opendir: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 414 root cwd unknown /proc/414/cwd (readlink: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 414 root rtd unknown /proc/414/root (readlink: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 414 root txt unknown /proc/414/exe (readlink: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 414 root NOFD /proc/414/fd (opendir: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:jbd2/sda1 429 root cwd unknown /proc/429/cwd (readlink: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:jbd2/sda1 429 root rtd unknown /proc/429/root (readlink: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:jbd2/sda1 429 root txt unknown /proc/429/exe (readlink: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:jbd2/sda1 429 root NOFD /proc/429/fd (opendir: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 430 root cwd unknown /proc/430/cwd (readlink: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 430 root rtd unknown /proc/430/root (readlink: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 430 root txt unknown /proc/430/exe (readlink: Permission denied) 2024-03-30T00:37:28.796 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 430 root NOFD /proc/430/fd (opendir: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 516 root cwd unknown /proc/516/cwd (readlink: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 516 root rtd unknown /proc/516/root (readlink: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 516 root txt unknown /proc/516/exe (readlink: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 516 root NOFD /proc/516/fd (opendir: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-j 556 root cwd unknown /proc/556/cwd (readlink: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-j 556 root rtd unknown /proc/556/root (readlink: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-j 556 root txt unknown /proc/556/exe (readlink: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-j 556 root NOFD /proc/556/fd (opendir: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:ext4lazyi 572 root cwd unknown /proc/572/cwd (readlink: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:ext4lazyi 572 root rtd unknown /proc/572/root (readlink: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:ext4lazyi 572 root txt unknown /proc/572/exe (readlink: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:ext4lazyi 572 root NOFD /proc/572/fd (opendir: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 579 root cwd unknown /proc/579/cwd (readlink: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 579 root rtd unknown /proc/579/root (readlink: Permission denied) 2024-03-30T00:37:28.797 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 579 root txt unknown /proc/579/exe (readlink: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 579 root NOFD /proc/579/fd (opendir: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 597 root cwd unknown /proc/597/cwd (readlink: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 597 root rtd unknown /proc/597/root (readlink: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 597 root txt unknown /proc/597/exe (readlink: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 597 root NOFD /proc/597/fd (opendir: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-u 605 root cwd unknown /proc/605/cwd (readlink: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-u 605 root rtd unknown /proc/605/root (readlink: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-u 605 root txt unknown /proc/605/exe (readlink: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-u 605 root NOFD /proc/605/fd (opendir: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:rpcbind 655 rpc cwd unknown /proc/655/cwd (readlink: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:rpcbind 655 rpc rtd unknown /proc/655/root (readlink: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:rpcbind 655 rpc txt unknown /proc/655/exe (readlink: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:rpcbind 655 rpc NOFD /proc/655/fd (opendir: Permission denied) 2024-03-30T00:37:28.798 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 root cwd unknown /proc/668/cwd (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 root rtd unknown /proc/668/root (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 root txt unknown /proc/668/exe (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 root NOFD /proc/668/fd (opendir: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 669 auditd root cwd unknown /proc/668/task/669/cwd (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 669 auditd root rtd unknown /proc/668/task/669/root (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 669 auditd root txt unknown /proc/668/task/669/exe (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 669 auditd root NOFD /proc/668/task/669/fd (opendir: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 671 auditd root cwd unknown /proc/668/task/671/cwd (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 671 auditd root rtd unknown /proc/668/task/671/root (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 671 auditd root txt unknown /proc/668/task/671/exe (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:auditd 668 671 auditd root NOFD /proc/668/task/671/fd (opendir: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:sedispatc 670 root cwd unknown /proc/670/cwd (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:sedispatc 670 root rtd unknown /proc/670/root (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:sedispatc 670 root txt unknown /proc/670/exe (readlink: Permission denied) 2024-03-30T00:37:28.799 INFO:tasks.workunit.client.0.smithi099.stdout:sedispatc 670 root NOFD /proc/670/fd (opendir: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 673 root cwd unknown /proc/673/cwd (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 673 root rtd unknown /proc/673/root (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 673 root txt unknown /proc/673/exe (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 673 root NOFD /proc/673/fd (opendir: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 725 root cwd unknown /proc/725/cwd (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 725 root rtd unknown /proc/725/root (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 725 root txt unknown /proc/725/exe (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 725 root NOFD /proc/725/fd (opendir: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 726 root cwd unknown /proc/726/cwd (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 726 root rtd unknown /proc/726/root (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 726 root txt unknown /proc/726/exe (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 726 root NOFD /proc/726/fd (opendir: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kipmi0 748 root cwd unknown /proc/748/cwd (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kipmi0 748 root rtd unknown /proc/748/root (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kipmi0 748 root txt unknown /proc/748/exe (readlink: Permission denied) 2024-03-30T00:37:28.800 INFO:tasks.workunit.client.0.smithi099.stdout:kipmi0 748 root NOFD /proc/748/fd (opendir: Permission denied) 2024-03-30T00:37:28.801 INFO:tasks.workunit.client.0.smithi099.stdout:smartd 758 root cwd unknown /proc/758/cwd (readlink: Permission denied) 2024-03-30T00:37:28.801 INFO:tasks.workunit.client.0.smithi099.stdout:smartd 758 root rtd unknown /proc/758/root (readlink: Permission denied) 2024-03-30T00:37:28.801 INFO:tasks.workunit.client.0.smithi099.stdout:smartd 758 root txt unknown /proc/758/exe (readlink: Permission denied) 2024-03-30T00:37:28.801 INFO:tasks.workunit.client.0.smithi099.stdout:smartd 758 root NOFD /proc/758/fd (opendir: Permission denied) 2024-03-30T00:37:28.801 INFO:tasks.workunit.client.0.smithi099.stdout:sssd 768 root cwd unknown /proc/768/cwd (readlink: Permission denied) 2024-03-30T00:37:28.801 INFO:tasks.workunit.client.0.smithi099.stdout:sssd 768 root rtd unknown /proc/768/root (readlink: Permission denied) 2024-03-30T00:37:28.801 INFO:tasks.workunit.client.0.smithi099.stdout:sssd 768 root txt unknown /proc/768/exe (readlink: Permission denied) 2024-03-30T00:37:28.801 INFO:tasks.workunit.client.0.smithi099.stdout:sssd 768 root NOFD /proc/768/fd (opendir: Permission denied) 2024-03-30T00:37:28.802 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 polkitd cwd unknown /proc/770/cwd (readlink: Permission denied) 2024-03-30T00:37:28.802 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 polkitd rtd unknown /proc/770/root (readlink: Permission denied) 2024-03-30T00:37:28.802 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 polkitd txt unknown /proc/770/exe (readlink: Permission denied) 2024-03-30T00:37:28.802 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 polkitd NOFD /proc/770/fd (opendir: Permission denied) 2024-03-30T00:37:28.802 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 812 gmain polkitd cwd unknown /proc/770/task/812/cwd (readlink: Permission denied) 2024-03-30T00:37:28.802 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 812 gmain polkitd rtd unknown /proc/770/task/812/root (readlink: Permission denied) 2024-03-30T00:37:28.802 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 812 gmain polkitd txt unknown /proc/770/task/812/exe (readlink: Permission denied) 2024-03-30T00:37:28.802 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 812 gmain polkitd NOFD /proc/770/task/812/fd (opendir: Permission denied) 2024-03-30T00:37:28.802 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 813 gdbus polkitd cwd unknown /proc/770/task/813/cwd (readlink: Permission denied) 2024-03-30T00:37:28.803 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 813 gdbus polkitd rtd unknown /proc/770/task/813/root (readlink: Permission denied) 2024-03-30T00:37:28.803 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 813 gdbus polkitd txt unknown /proc/770/task/813/exe (readlink: Permission denied) 2024-03-30T00:37:28.803 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 813 gdbus polkitd NOFD /proc/770/task/813/fd (opendir: Permission denied) 2024-03-30T00:37:28.803 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 816 JS\x20Hel polkitd cwd unknown /proc/770/task/816/cwd (readlink: Permission denied) 2024-03-30T00:37:28.803 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 816 JS\x20Hel polkitd rtd unknown /proc/770/task/816/root (readlink: Permission denied) 2024-03-30T00:37:28.803 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 816 JS\x20Hel polkitd txt unknown /proc/770/task/816/exe (readlink: Permission denied) 2024-03-30T00:37:28.803 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 816 JS\x20Hel polkitd NOFD /proc/770/task/816/fd (opendir: Permission denied) 2024-03-30T00:37:28.803 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 817 JS\x20Hel polkitd cwd unknown /proc/770/task/817/cwd (readlink: Permission denied) 2024-03-30T00:37:28.803 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 817 JS\x20Hel polkitd rtd unknown /proc/770/task/817/root (readlink: Permission denied) 2024-03-30T00:37:28.804 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 817 JS\x20Hel polkitd txt unknown /proc/770/task/817/exe (readlink: Permission denied) 2024-03-30T00:37:28.804 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 817 JS\x20Hel polkitd NOFD /proc/770/task/817/fd (opendir: Permission denied) 2024-03-30T00:37:28.804 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 818 JS\x20Hel polkitd cwd unknown /proc/770/task/818/cwd (readlink: Permission denied) 2024-03-30T00:37:28.804 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 818 JS\x20Hel polkitd rtd unknown /proc/770/task/818/root (readlink: Permission denied) 2024-03-30T00:37:28.804 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 818 JS\x20Hel polkitd txt unknown /proc/770/task/818/exe (readlink: Permission denied) 2024-03-30T00:37:28.804 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 818 JS\x20Hel polkitd NOFD /proc/770/task/818/fd (opendir: Permission denied) 2024-03-30T00:37:28.804 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 819 JS\x20Hel polkitd cwd unknown /proc/770/task/819/cwd (readlink: Permission denied) 2024-03-30T00:37:28.804 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 819 JS\x20Hel polkitd rtd unknown /proc/770/task/819/root (readlink: Permission denied) 2024-03-30T00:37:28.804 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 819 JS\x20Hel polkitd txt unknown /proc/770/task/819/exe (readlink: Permission denied) 2024-03-30T00:37:28.804 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 819 JS\x20Hel polkitd NOFD /proc/770/task/819/fd (opendir: Permission denied) 2024-03-30T00:37:28.805 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 820 JS\x20Hel polkitd cwd unknown /proc/770/task/820/cwd (readlink: Permission denied) 2024-03-30T00:37:28.805 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 820 JS\x20Hel polkitd rtd unknown /proc/770/task/820/root (readlink: Permission denied) 2024-03-30T00:37:28.805 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 820 JS\x20Hel polkitd txt unknown /proc/770/task/820/exe (readlink: Permission denied) 2024-03-30T00:37:28.805 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 820 JS\x20Hel polkitd NOFD /proc/770/task/820/fd (opendir: Permission denied) 2024-03-30T00:37:28.805 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 821 JS\x20Hel polkitd cwd unknown /proc/770/task/821/cwd (readlink: Permission denied) 2024-03-30T00:37:28.805 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 821 JS\x20Hel polkitd rtd unknown /proc/770/task/821/root (readlink: Permission denied) 2024-03-30T00:37:28.805 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 821 JS\x20Hel polkitd txt unknown /proc/770/task/821/exe (readlink: Permission denied) 2024-03-30T00:37:28.805 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 821 JS\x20Hel polkitd NOFD /proc/770/task/821/fd (opendir: Permission denied) 2024-03-30T00:37:28.805 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 822 JS\x20Hel polkitd cwd unknown /proc/770/task/822/cwd (readlink: Permission denied) 2024-03-30T00:37:28.805 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 822 JS\x20Hel polkitd rtd unknown /proc/770/task/822/root (readlink: Permission denied) 2024-03-30T00:37:28.806 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 822 JS\x20Hel polkitd txt unknown /proc/770/task/822/exe (readlink: Permission denied) 2024-03-30T00:37:28.806 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 822 JS\x20Hel polkitd NOFD /proc/770/task/822/fd (opendir: Permission denied) 2024-03-30T00:37:28.806 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 823 JS\x20Hel polkitd cwd unknown /proc/770/task/823/cwd (readlink: Permission denied) 2024-03-30T00:37:28.806 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 823 JS\x20Hel polkitd rtd unknown /proc/770/task/823/root (readlink: Permission denied) 2024-03-30T00:37:28.806 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 823 JS\x20Hel polkitd txt unknown /proc/770/task/823/exe (readlink: Permission denied) 2024-03-30T00:37:28.806 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 823 JS\x20Hel polkitd NOFD /proc/770/task/823/fd (opendir: Permission denied) 2024-03-30T00:37:28.806 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 833 polkitd polkitd cwd unknown /proc/770/task/833/cwd (readlink: Permission denied) 2024-03-30T00:37:28.806 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 833 polkitd polkitd rtd unknown /proc/770/task/833/root (readlink: Permission denied) 2024-03-30T00:37:28.806 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 833 polkitd polkitd txt unknown /proc/770/task/833/exe (readlink: Permission denied) 2024-03-30T00:37:28.806 INFO:tasks.workunit.client.0.smithi099.stdout:polkitd 770 833 polkitd polkitd NOFD /proc/770/task/833/fd (opendir: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:irqbalanc 772 root cwd unknown /proc/772/cwd (readlink: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:irqbalanc 772 root rtd unknown /proc/772/root (readlink: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:irqbalanc 772 root txt unknown /proc/772/exe (readlink: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:irqbalanc 772 root NOFD /proc/772/fd (opendir: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:irqbalanc 772 783 gmain root cwd unknown /proc/772/task/783/cwd (readlink: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:irqbalanc 772 783 gmain root rtd unknown /proc/772/task/783/root (readlink: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:irqbalanc 772 783 gmain root txt unknown /proc/772/task/783/exe (readlink: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:irqbalanc 772 783 gmain root NOFD /proc/772/task/783/fd (opendir: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:lsmd 773 libstoragemgmt cwd unknown /proc/773/cwd (readlink: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:lsmd 773 libstoragemgmt rtd unknown /proc/773/root (readlink: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:lsmd 773 libstoragemgmt txt unknown /proc/773/exe (readlink: Permission denied) 2024-03-30T00:37:28.807 INFO:tasks.workunit.client.0.smithi099.stdout:lsmd 773 libstoragemgmt NOFD /proc/773/fd (opendir: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:dbus-daem 774 dbus cwd unknown /proc/774/cwd (readlink: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:dbus-daem 774 dbus rtd unknown /proc/774/root (readlink: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:dbus-daem 774 dbus txt unknown /proc/774/exe (readlink: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:dbus-daem 774 dbus NOFD /proc/774/fd (opendir: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:dbus-daem 774 791 dbus-daem dbus cwd unknown /proc/774/task/791/cwd (readlink: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:dbus-daem 774 791 dbus-daem dbus rtd unknown /proc/774/task/791/root (readlink: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:dbus-daem 774 791 dbus-daem dbus txt unknown /proc/774/task/791/exe (readlink: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:dbus-daem 774 791 dbus-daem dbus NOFD /proc/774/task/791/fd (opendir: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 root cwd unknown /proc/775/cwd (readlink: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 root rtd unknown /proc/775/root (readlink: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 root txt unknown /proc/775/exe (readlink: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 root NOFD /proc/775/fd (opendir: Permission denied) 2024-03-30T00:37:28.808 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 814 gmain root cwd unknown /proc/775/task/814/cwd (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 814 gmain root rtd unknown /proc/775/task/814/root (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 814 gmain root txt unknown /proc/775/task/814/exe (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 814 gmain root NOFD /proc/775/task/814/fd (opendir: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 815 gdbus root cwd unknown /proc/775/task/815/cwd (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 815 gdbus root rtd unknown /proc/775/task/815/root (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 815 gdbus root txt unknown /proc/775/task/815/exe (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:NetworkMa 775 815 gdbus root NOFD /proc/775/task/815/fd (opendir: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:avahi-dae 777 avahi cwd unknown /proc/777/cwd (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:avahi-dae 777 avahi rtd unknown /proc/777/root (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:avahi-dae 777 avahi txt unknown /proc/777/exe (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:avahi-dae 777 avahi NOFD /proc/777/fd (opendir: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:avahi-dae 793 avahi cwd unknown /proc/793/cwd (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:avahi-dae 793 avahi rtd unknown /proc/793/root (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:avahi-dae 793 avahi txt unknown /proc/793/exe (readlink: Permission denied) 2024-03-30T00:37:28.809 INFO:tasks.workunit.client.0.smithi099.stdout:avahi-dae 793 avahi NOFD /proc/793/fd (opendir: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 825 root cwd unknown /proc/825/cwd (readlink: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 825 root rtd unknown /proc/825/root (readlink: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 825 root txt unknown /proc/825/exe (readlink: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 825 root NOFD /proc/825/fd (opendir: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 root cwd unknown /proc/826/cwd (readlink: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 root rtd unknown /proc/826/root (readlink: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 root txt unknown /proc/826/exe (readlink: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 root NOFD /proc/826/fd (opendir: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1046 tuned root cwd unknown /proc/826/task/1046/cwd (readlink: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1046 tuned root rtd unknown /proc/826/task/1046/root (readlink: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1046 tuned root txt unknown /proc/826/task/1046/exe (readlink: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1046 tuned root NOFD /proc/826/task/1046/fd (opendir: Permission denied) 2024-03-30T00:37:28.810 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1051 tuned root cwd unknown /proc/826/task/1051/cwd (readlink: Permission denied) 2024-03-30T00:37:28.811 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1051 tuned root rtd unknown /proc/826/task/1051/root (readlink: Permission denied) 2024-03-30T00:37:28.811 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1051 tuned root txt unknown /proc/826/task/1051/exe (readlink: Permission denied) 2024-03-30T00:37:28.811 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1051 tuned root NOFD /proc/826/task/1051/fd (opendir: Permission denied) 2024-03-30T00:37:28.811 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1068 tuned root cwd unknown /proc/826/task/1068/cwd (readlink: Permission denied) 2024-03-30T00:37:28.811 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1068 tuned root rtd unknown /proc/826/task/1068/root (readlink: Permission denied) 2024-03-30T00:37:28.811 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1068 tuned root txt unknown /proc/826/task/1068/exe (readlink: Permission denied) 2024-03-30T00:37:28.811 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1068 tuned root NOFD /proc/826/task/1068/fd (opendir: Permission denied) 2024-03-30T00:37:28.811 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1069 tuned root cwd unknown /proc/826/task/1069/cwd (readlink: Permission denied) 2024-03-30T00:37:28.811 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1069 tuned root rtd unknown /proc/826/task/1069/root (readlink: Permission denied) 2024-03-30T00:37:28.811 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1069 tuned root txt unknown /proc/826/task/1069/exe (readlink: Permission denied) 2024-03-30T00:37:28.812 INFO:tasks.workunit.client.0.smithi099.stdout:tuned 826 1069 tuned root NOFD /proc/826/task/1069/fd (opendir: Permission denied) 2024-03-30T00:37:28.812 INFO:tasks.workunit.client.0.smithi099.stdout:rhsmcertd 831 root cwd unknown /proc/831/cwd (readlink: Permission denied) 2024-03-30T00:37:28.812 INFO:tasks.workunit.client.0.smithi099.stdout:rhsmcertd 831 root rtd unknown /proc/831/root (readlink: Permission denied) 2024-03-30T00:37:28.812 INFO:tasks.workunit.client.0.smithi099.stdout:rhsmcertd 831 root txt unknown /proc/831/exe (readlink: Permission denied) 2024-03-30T00:37:28.812 INFO:tasks.workunit.client.0.smithi099.stdout:rhsmcertd 831 root NOFD /proc/831/fd (opendir: Permission denied) 2024-03-30T00:37:28.812 INFO:tasks.workunit.client.0.smithi099.stdout:sssd_be 832 root cwd unknown /proc/832/cwd (readlink: Permission denied) 2024-03-30T00:37:28.812 INFO:tasks.workunit.client.0.smithi099.stdout:sssd_be 832 root rtd unknown /proc/832/root (readlink: Permission denied) 2024-03-30T00:37:28.812 INFO:tasks.workunit.client.0.smithi099.stdout:sssd_be 832 root txt unknown /proc/832/exe (readlink: Permission denied) 2024-03-30T00:37:28.812 INFO:tasks.workunit.client.0.smithi099.stdout:sssd_be 832 root NOFD /proc/832/fd (opendir: Permission denied) 2024-03-30T00:37:28.813 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 root cwd unknown /proc/835/cwd (readlink: Permission denied) 2024-03-30T00:37:28.813 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 root rtd unknown /proc/835/root (readlink: Permission denied) 2024-03-30T00:37:28.813 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 root txt unknown /proc/835/exe (readlink: Permission denied) 2024-03-30T00:37:28.813 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 root NOFD /proc/835/fd (opendir: Permission denied) 2024-03-30T00:37:28.813 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 836 gssproxy root cwd unknown /proc/835/task/836/cwd (readlink: Permission denied) 2024-03-30T00:37:28.813 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 836 gssproxy root rtd unknown /proc/835/task/836/root (readlink: Permission denied) 2024-03-30T00:37:28.813 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 836 gssproxy root txt unknown /proc/835/task/836/exe (readlink: Permission denied) 2024-03-30T00:37:28.813 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 836 gssproxy root NOFD /proc/835/task/836/fd (opendir: Permission denied) 2024-03-30T00:37:28.813 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 837 gssproxy root cwd unknown /proc/835/task/837/cwd (readlink: Permission denied) 2024-03-30T00:37:28.813 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 837 gssproxy root rtd unknown /proc/835/task/837/root (readlink: Permission denied) 2024-03-30T00:37:28.813 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 837 gssproxy root txt unknown /proc/835/task/837/exe (readlink: Permission denied) 2024-03-30T00:37:28.814 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 837 gssproxy root NOFD /proc/835/task/837/fd (opendir: Permission denied) 2024-03-30T00:37:28.814 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 838 gssproxy root cwd unknown /proc/835/task/838/cwd (readlink: Permission denied) 2024-03-30T00:37:28.814 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 838 gssproxy root rtd unknown /proc/835/task/838/root (readlink: Permission denied) 2024-03-30T00:37:28.814 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 838 gssproxy root txt unknown /proc/835/task/838/exe (readlink: Permission denied) 2024-03-30T00:37:28.814 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 838 gssproxy root NOFD /proc/835/task/838/fd (opendir: Permission denied) 2024-03-30T00:37:28.814 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 839 gssproxy root cwd unknown /proc/835/task/839/cwd (readlink: Permission denied) 2024-03-30T00:37:28.814 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 839 gssproxy root rtd unknown /proc/835/task/839/root (readlink: Permission denied) 2024-03-30T00:37:28.814 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 839 gssproxy root txt unknown /proc/835/task/839/exe (readlink: Permission denied) 2024-03-30T00:37:28.814 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 839 gssproxy root NOFD /proc/835/task/839/fd (opendir: Permission denied) 2024-03-30T00:37:28.814 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 840 gssproxy root cwd unknown /proc/835/task/840/cwd (readlink: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 840 gssproxy root rtd unknown /proc/835/task/840/root (readlink: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 840 gssproxy root txt unknown /proc/835/task/840/exe (readlink: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:gssproxy 835 840 gssproxy root NOFD /proc/835/task/840/fd (opendir: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:sssd_nss 852 root cwd unknown /proc/852/cwd (readlink: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:sssd_nss 852 root rtd unknown /proc/852/root (readlink: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:sssd_nss 852 root txt unknown /proc/852/exe (readlink: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:sssd_nss 852 root NOFD /proc/852/fd (opendir: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-l 891 root cwd unknown /proc/891/cwd (readlink: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-l 891 root rtd unknown /proc/891/root (readlink: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-l 891 root txt unknown /proc/891/exe (readlink: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:systemd-l 891 root NOFD /proc/891/fd (opendir: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:cupsd 892 root cwd unknown /proc/892/cwd (readlink: Permission denied) 2024-03-30T00:37:28.815 INFO:tasks.workunit.client.0.smithi099.stdout:cupsd 892 root rtd unknown /proc/892/root (readlink: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:cupsd 892 root txt unknown /proc/892/exe (readlink: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:cupsd 892 root NOFD /proc/892/fd (opendir: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:crond 894 root cwd unknown /proc/894/cwd (readlink: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:crond 894 root rtd unknown /proc/894/root (readlink: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:crond 894 root txt unknown /proc/894/exe (readlink: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:crond 894 root NOFD /proc/894/fd (opendir: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:atd 896 root cwd unknown /proc/896/cwd (readlink: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:atd 896 root rtd unknown /proc/896/root (readlink: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:atd 896 root txt unknown /proc/896/exe (readlink: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:atd 896 root NOFD /proc/896/fd (opendir: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:nrpe 1110 nrpe cwd unknown /proc/1110/cwd (readlink: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:nrpe 1110 nrpe rtd unknown /proc/1110/root (readlink: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:nrpe 1110 nrpe txt unknown /proc/1110/exe (readlink: Permission denied) 2024-03-30T00:37:28.816 INFO:tasks.workunit.client.0.smithi099.stdout:nrpe 1110 nrpe NOFD /proc/1110/fd (opendir: Permission denied) 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu cwd DIR 8,1 4096 2 / 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu rtd DIR 8,1 4096 2 / 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu txt REG 8,1 1601312 8305 /usr/lib/systemd/systemd 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 1598808 5171 /usr/lib64/libm-2.28.so 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 636728 7430 /usr/lib64/libudev.so.1.6.11 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 735192 5316 /usr/lib64/libsepol.so.1 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 1760264 5414 /usr/lib64/libunistring.so.2.1.0 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 144392 5392 /usr/lib64/libgpg-error.so.0.24.2 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 66912 5574 /usr/lib64/libjson-c.so.4.0.0 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 371224 7497 /usr/lib64/libdevmapper.so.1.02 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 26192 5481 /usr/lib64/libattr.so.1.1.2448 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu DEL REG 8,1 7507 /usr/lib64/libcrypto.so.1.1.1k 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu DEL REG 8,1 7509 /usr/lib64/libssl.so.1.1.1k 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 99600 5318 /usr/lib64/libz.so.1.2.11 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 24928 5576 /usr/lib64/libcap-ng.so.0.0.0 2024-03-30T00:37:28.817 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu DEL REG 8,1 5382 /usr/lib64/libuuid.so.1.3.0 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 19584 5169 /usr/lib64/libdl-2.28.so 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 543160 4872 /usr/lib64/libpcre2-8.so.0.7.1 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu DEL REG 8,1 7499 /usr/lib64/libblkid.so.1.1.0 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 119760 5781 /usr/lib64/liblz4.so.1.8.3 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 192016 5359 /usr/lib64/liblzma.so.5.2.4 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 162224 5584 /usr/lib64/libidn2.so.0.3.6 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 1187360 5616 /usr/lib64/libgcrypt.so.20.2.5 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 526360 7535 /usr/lib64/libcryptsetup.so.12.6.0 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 54816 5483 /usr/lib64/libacl.so.1.1.2253 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 33752 5371 /usr/lib64/libcap.so.2.48 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 2088960 5167 /usr/lib64/libc-2.28.so 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 149976 5181 /usr/lib64/libpthread-2.28.so 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 99664 6703 /usr/lib64/libgcc_s-8-20210514.so.1 2024-03-30T00:37:28.818 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 100200 7532 /usr/lib64/libkmod.so.2.3.3 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 135792 5578 /usr/lib64/libaudit.so.1.0.0 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 66504 7674 /usr/lib64/libpam.so.0.84.2 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu DEL REG 8,1 7306 /usr/lib64/libmount.so.1.1.0 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 168536 4876 /usr/lib64/libselinux.so.1 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 128784 6087 /usr/lib64/libseccomp.so.2.5.2 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 42744 5185 /usr/lib64/librt-2.28.so 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 2740672 8300 /usr/lib/systemd/libsystemd-shared-239.so 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu mem REG 8,1 1105744 5160 /usr/lib64/ld-2.28.so 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 1u unix 0x0000000000000000 0t0 8995 type=STREAM 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 2u unix 0x0000000000000000 0t0 8995 type=STREAM 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 3u unix 0x0000000000000000 0t0 9012 type=DGRAM 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 4u a_inode 0,14 0 50 [eventpoll] 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 5u a_inode 0,14 0 50 [signalfd] 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 6r DIR 0,26 0 748 /sys/fs/cgroup/systemd/user.slice/user-1000.slice/user@1000.service 2024-03-30T00:37:28.819 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 7u a_inode 0,14 0 50 [timerfd] 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 8u netlink 0t0 9014 KOBJECT_UEVENT 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 9u a_inode 0,14 0 50 [eventpoll] 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 10r a_inode 0,14 0 50 inotify 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 11u unix 0x0000000000000000 0t0 9030 /run/user/1000/bus type=STREAM 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 12r REG 0,21 0 9019 /proc/1122/mountinfo 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 13r a_inode 0,14 0 50 inotify 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 14r REG 0,21 0 4026532078 /proc/swaps 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 15u unix 0x0000000000000000 0t0 9020 /run/user/1000/systemd/notify type=DGRAM 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 16u unix 0x0000000000000000 0t0 9021 type=DGRAM 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 17u unix 0x0000000000000000 0t0 9022 type=DGRAM 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 18u unix 0x0000000000000000 0t0 9023 /run/user/1000/systemd/private type=STREAM 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 19u unix 0x0000000000000000 0t0 9024 type=STREAM 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 20u a_inode 0,14 0 50 [timerfd] 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 1122 ubuntu 26u netlink 0t0 9032 SELINUX 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:(sd-pam) 1125 ubuntu cwd unknown /proc/1125/cwd (readlink: Permission denied) 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:(sd-pam) 1125 ubuntu rtd unknown /proc/1125/root (readlink: Permission denied) 2024-03-30T00:37:28.820 INFO:tasks.workunit.client.0.smithi099.stdout:(sd-pam) 1125 ubuntu txt unknown /proc/1125/exe (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:(sd-pam) 1125 ubuntu NOFD /proc/1125/fd (opendir: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:agetty 1195 root cwd unknown /proc/1195/cwd (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:agetty 1195 root rtd unknown /proc/1195/root (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:agetty 1195 root txt unknown /proc/1195/exe (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:agetty 1195 root NOFD /proc/1195/fd (opendir: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:agetty 1197 root cwd unknown /proc/1197/cwd (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:agetty 1197 root rtd unknown /proc/1197/root (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:agetty 1197 root txt unknown /proc/1197/exe (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:agetty 1197 root NOFD /proc/1197/fd (opendir: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 root cwd unknown /proc/1483/cwd (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 root rtd unknown /proc/1483/root (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 root txt unknown /proc/1483/exe (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 root NOFD /proc/1483/fd (opendir: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 1485 in:imjour root cwd unknown /proc/1483/task/1485/cwd (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 1485 in:imjour root rtd unknown /proc/1483/task/1485/root (readlink: Permission denied) 2024-03-30T00:37:28.821 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 1485 in:imjour root txt unknown /proc/1483/task/1485/exe (readlink: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 1485 in:imjour root NOFD /proc/1483/task/1485/fd (opendir: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 1486 rs:main root cwd unknown /proc/1483/task/1486/cwd (readlink: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 1486 rs:main root rtd unknown /proc/1483/task/1486/root (readlink: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 1486 rs:main root txt unknown /proc/1483/task/1486/exe (readlink: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:rsyslogd 1483 1486 rs:main root NOFD /proc/1483/task/1486/fd (opendir: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 20511 root cwd unknown /proc/20511/cwd (readlink: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 20511 root rtd unknown /proc/20511/root (readlink: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 20511 root txt unknown /proc/20511/exe (readlink: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 20511 root NOFD /proc/20511/fd (opendir: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:ksmtuned 20520 root cwd unknown /proc/20520/cwd (readlink: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:ksmtuned 20520 root rtd unknown /proc/20520/root (readlink: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:ksmtuned 20520 root txt unknown /proc/20520/exe (readlink: Permission denied) 2024-03-30T00:37:28.822 INFO:tasks.workunit.client.0.smithi099.stdout:ksmtuned 20520 root NOFD /proc/20520/fd (opendir: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22811 root cwd unknown /proc/22811/cwd (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22811 root rtd unknown /proc/22811/root (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22811 root txt unknown /proc/22811/exe (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22811 root NOFD /proc/22811/fd (opendir: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22848 root cwd unknown /proc/22848/cwd (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22848 root rtd unknown /proc/22848/root (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22848 root txt unknown /proc/22848/exe (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22848 root NOFD /proc/22848/fd (opendir: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22885 root cwd unknown /proc/22885/cwd (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22885 root rtd unknown /proc/22885/root (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22885 root txt unknown /proc/22885/exe (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22885 root NOFD /proc/22885/fd (opendir: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22922 root cwd unknown /proc/22922/cwd (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22922 root rtd unknown /proc/22922/root (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22922 root txt unknown /proc/22922/exe (readlink: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22922 root NOFD /proc/22922/fd (opendir: Permission denied) 2024-03-30T00:37:28.823 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22959 root cwd unknown /proc/22959/cwd (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22959 root rtd unknown /proc/22959/root (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22959 root txt unknown /proc/22959/exe (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 22959 root NOFD /proc/22959/fd (opendir: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23179 root cwd unknown /proc/23179/cwd (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23179 root rtd unknown /proc/23179/root (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23179 root txt unknown /proc/23179/exe (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23179 root NOFD /proc/23179/fd (opendir: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23180 root cwd unknown /proc/23180/cwd (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23180 root rtd unknown /proc/23180/root (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23180 root txt unknown /proc/23180/exe (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23180 root NOFD /proc/23180/fd (opendir: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23181 root cwd unknown /proc/23181/cwd (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23181 root rtd unknown /proc/23181/root (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23181 root txt unknown /proc/23181/exe (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23181 root NOFD /proc/23181/fd (opendir: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23183 root cwd unknown /proc/23183/cwd (readlink: Permission denied) 2024-03-30T00:37:28.824 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23183 root rtd unknown /proc/23183/root (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23183 root txt unknown /proc/23183/exe (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23183 root NOFD /proc/23183/fd (opendir: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23184 root cwd unknown /proc/23184/cwd (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23184 root rtd unknown /proc/23184/root (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23184 root txt unknown /proc/23184/exe (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23184 root NOFD /proc/23184/fd (opendir: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23185 root cwd unknown /proc/23185/cwd (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23185 root rtd unknown /proc/23185/root (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23185 root txt unknown /proc/23185/exe (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23185 root NOFD /proc/23185/fd (opendir: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23186 root cwd unknown /proc/23186/cwd (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23186 root rtd unknown /proc/23186/root (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23186 root txt unknown /proc/23186/exe (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23186 root NOFD /proc/23186/fd (opendir: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23187 root cwd unknown /proc/23187/cwd (readlink: Permission denied) 2024-03-30T00:37:28.825 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23187 root rtd unknown /proc/23187/root (readlink: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23187 root txt unknown /proc/23187/exe (readlink: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23187 root NOFD /proc/23187/fd (opendir: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23188 root cwd unknown /proc/23188/cwd (readlink: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23188 root rtd unknown /proc/23188/root (readlink: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23188 root txt unknown /proc/23188/exe (readlink: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/R 23188 root NOFD /proc/23188/fd (opendir: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:xfsaild/d 23189 root cwd unknown /proc/23189/cwd (readlink: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:xfsaild/d 23189 root rtd unknown /proc/23189/root (readlink: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:xfsaild/d 23189 root txt unknown /proc/23189/exe (readlink: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:xfsaild/d 23189 root NOFD /proc/23189/fd (opendir: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 26044 root cwd unknown /proc/26044/cwd (readlink: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 26044 root rtd unknown /proc/26044/root (readlink: Permission denied) 2024-03-30T00:37:28.826 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 26044 root txt unknown /proc/26044/exe (readlink: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 26044 root NOFD /proc/26044/fd (opendir: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 26705 root cwd unknown /proc/26705/cwd (readlink: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 26705 root rtd unknown /proc/26705/root (readlink: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 26705 root txt unknown /proc/26705/exe (readlink: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 26705 root NOFD /proc/26705/fd (opendir: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 26709 ubuntu cwd unknown /proc/26709/cwd (readlink: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 26709 ubuntu rtd unknown /proc/26709/root (readlink: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 26709 ubuntu txt unknown /proc/26709/exe (readlink: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 26709 ubuntu NOFD /proc/26709/fd (opendir: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:chronyd 26779 chrony cwd unknown /proc/26779/cwd (readlink: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:chronyd 26779 chrony rtd unknown /proc/26779/root (readlink: Permission denied) 2024-03-30T00:37:28.827 INFO:tasks.workunit.client.0.smithi099.stdout:chronyd 26779 chrony txt unknown /proc/26779/exe (readlink: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:chronyd 26779 chrony NOFD /proc/26779/fd (opendir: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 28283 root cwd unknown /proc/28283/cwd (readlink: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 28283 root rtd unknown /proc/28283/root (readlink: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 28283 root txt unknown /proc/28283/exe (readlink: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 28283 root NOFD /proc/28283/fd (opendir: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-cras 28614 ceph cwd unknown /proc/28614/cwd (readlink: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-cras 28614 ceph rtd unknown /proc/28614/root (readlink: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-cras 28614 ceph txt unknown /proc/28614/exe (readlink: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-cras 28614 ceph NOFD /proc/28614/fd (opendir: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 36731 root cwd unknown /proc/36731/cwd (readlink: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 36731 root rtd unknown /proc/36731/root (readlink: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 36731 root txt unknown /proc/36731/exe (readlink: Permission denied) 2024-03-30T00:37:28.828 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 36731 root NOFD /proc/36731/fd (opendir: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 36750 root cwd unknown /proc/36750/cwd (readlink: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 36750 root rtd unknown /proc/36750/root (readlink: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 36750 root txt unknown /proc/36750/exe (readlink: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 36750 root NOFD /proc/36750/fd (opendir: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 36758 root cwd unknown /proc/36758/cwd (readlink: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 36758 root rtd unknown /proc/36758/root (readlink: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 36758 root txt unknown /proc/36758/exe (readlink: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 36758 root NOFD /proc/36758/fd (opendir: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 36792 root cwd unknown /proc/36792/cwd (readlink: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 36792 root rtd unknown /proc/36792/root (readlink: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 36792 root txt unknown /proc/36792/exe (readlink: Permission denied) 2024-03-30T00:37:28.829 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 36792 root NOFD /proc/36792/fd (opendir: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37958 root cwd unknown /proc/37958/cwd (readlink: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37958 root rtd unknown /proc/37958/root (readlink: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37958 root txt unknown /proc/37958/exe (readlink: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37958 root NOFD /proc/37958/fd (opendir: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37959 root cwd unknown /proc/37959/cwd (readlink: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37959 root rtd unknown /proc/37959/root (readlink: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37959 root txt unknown /proc/37959/exe (readlink: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37959 root NOFD /proc/37959/fd (opendir: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37960 root cwd unknown /proc/37960/cwd (readlink: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37960 root rtd unknown /proc/37960/root (readlink: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37960 root txt unknown /proc/37960/exe (readlink: Permission denied) 2024-03-30T00:37:28.830 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 37960 root NOFD /proc/37960/fd (opendir: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 37997 root cwd unknown /proc/37997/cwd (readlink: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 37997 root rtd unknown /proc/37997/root (readlink: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 37997 root txt unknown /proc/37997/exe (readlink: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 37997 root NOFD /proc/37997/fd (opendir: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 38056 root cwd unknown /proc/38056/cwd (readlink: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 38056 root rtd unknown /proc/38056/root (readlink: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 38056 root txt unknown /proc/38056/exe (readlink: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 38056 root NOFD /proc/38056/fd (opendir: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 38067 root cwd unknown /proc/38067/cwd (readlink: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 38067 root rtd unknown /proc/38067/root (readlink: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 38067 root txt unknown /proc/38067/exe (readlink: Permission denied) 2024-03-30T00:37:28.831 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 38067 root NOFD /proc/38067/fd (opendir: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 ceph cwd unknown /proc/38079/cwd (readlink: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 ceph rtd unknown /proc/38079/root (readlink: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 ceph txt unknown /proc/38079/exe (readlink: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 ceph NOFD /proc/38079/fd (opendir: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38081 log ceph cwd unknown /proc/38079/task/38081/cwd (readlink: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38081 log ceph rtd unknown /proc/38079/task/38081/root (readlink: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38081 log ceph txt unknown /proc/38079/task/38081/exe (readlink: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38081 log ceph NOFD /proc/38079/task/38081/fd (opendir: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38083 service ceph cwd unknown /proc/38079/task/38083/cwd (readlink: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38083 service ceph rtd unknown /proc/38079/task/38083/root (readlink: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38083 service ceph txt unknown /proc/38079/task/38083/exe (readlink: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38083 service ceph NOFD /proc/38079/task/38083/fd (opendir: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38084 admin_soc ceph cwd unknown /proc/38079/task/38084/cwd (readlink: Permission denied) 2024-03-30T00:37:28.832 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38084 admin_soc ceph rtd unknown /proc/38079/task/38084/root (readlink: Permission denied) 2024-03-30T00:37:28.833 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38084 admin_soc ceph txt unknown /proc/38079/task/38084/exe (readlink: Permission denied) 2024-03-30T00:37:28.833 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38084 admin_soc ceph NOFD /proc/38079/task/38084/fd (opendir: Permission denied) 2024-03-30T00:37:28.833 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38085 signal_ha ceph cwd unknown /proc/38079/task/38085/cwd (readlink: Permission denied) 2024-03-30T00:37:28.833 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38085 signal_ha ceph rtd unknown /proc/38079/task/38085/root (readlink: Permission denied) 2024-03-30T00:37:28.833 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38085 signal_ha ceph txt unknown /proc/38079/task/38085/exe (readlink: Permission denied) 2024-03-30T00:37:28.833 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38085 signal_ha ceph NOFD /proc/38079/task/38085/fd (opendir: Permission denied) 2024-03-30T00:37:28.833 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38086 rocksdb:l ceph cwd unknown /proc/38079/task/38086/cwd (readlink: Permission denied) 2024-03-30T00:37:28.833 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38086 rocksdb:l ceph rtd unknown /proc/38079/task/38086/root (readlink: Permission denied) 2024-03-30T00:37:28.833 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38086 rocksdb:l ceph txt unknown /proc/38079/task/38086/exe (readlink: Permission denied) 2024-03-30T00:37:28.833 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38086 rocksdb:l ceph NOFD /proc/38079/task/38086/fd (opendir: Permission denied) 2024-03-30T00:37:28.833 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38087 rocksdb:h ceph cwd unknown /proc/38079/task/38087/cwd (readlink: Permission denied) 2024-03-30T00:37:28.834 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38087 rocksdb:h ceph rtd unknown /proc/38079/task/38087/root (readlink: Permission denied) 2024-03-30T00:37:28.834 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38087 rocksdb:h ceph txt unknown /proc/38079/task/38087/exe (readlink: Permission denied) 2024-03-30T00:37:28.834 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38087 rocksdb:h ceph NOFD /proc/38079/task/38087/fd (opendir: Permission denied) 2024-03-30T00:37:28.834 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38108 ceph-mon ceph cwd unknown /proc/38079/task/38108/cwd (readlink: Permission denied) 2024-03-30T00:37:28.834 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38108 ceph-mon ceph rtd unknown /proc/38079/task/38108/root (readlink: Permission denied) 2024-03-30T00:37:28.834 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38108 ceph-mon ceph txt unknown /proc/38079/task/38108/exe (readlink: Permission denied) 2024-03-30T00:37:28.834 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38108 ceph-mon ceph NOFD /proc/38079/task/38108/fd (opendir: Permission denied) 2024-03-30T00:37:28.834 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38109 fn_monsto ceph cwd unknown /proc/38079/task/38109/cwd (readlink: Permission denied) 2024-03-30T00:37:28.834 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38109 fn_monsto ceph rtd unknown /proc/38079/task/38109/root (readlink: Permission denied) 2024-03-30T00:37:28.834 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38109 fn_monsto ceph txt unknown /proc/38079/task/38109/exe (readlink: Permission denied) 2024-03-30T00:37:28.834 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38109 fn_monsto ceph NOFD /proc/38079/task/38109/fd (opendir: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38110 msgr-work ceph cwd unknown /proc/38079/task/38110/cwd (readlink: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38110 msgr-work ceph rtd unknown /proc/38079/task/38110/root (readlink: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38110 msgr-work ceph txt unknown /proc/38079/task/38110/exe (readlink: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38110 msgr-work ceph NOFD /proc/38079/task/38110/fd (opendir: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38111 msgr-work ceph cwd unknown /proc/38079/task/38111/cwd (readlink: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38111 msgr-work ceph rtd unknown /proc/38079/task/38111/root (readlink: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38111 msgr-work ceph txt unknown /proc/38079/task/38111/exe (readlink: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38111 msgr-work ceph NOFD /proc/38079/task/38111/fd (opendir: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38112 msgr-work ceph cwd unknown /proc/38079/task/38112/cwd (readlink: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38112 msgr-work ceph rtd unknown /proc/38079/task/38112/root (readlink: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38112 msgr-work ceph txt unknown /proc/38079/task/38112/exe (readlink: Permission denied) 2024-03-30T00:37:28.835 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38112 msgr-work ceph NOFD /proc/38079/task/38112/fd (opendir: Permission denied) 2024-03-30T00:37:28.836 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38113 OpHistory ceph cwd unknown /proc/38079/task/38113/cwd (readlink: Permission denied) 2024-03-30T00:37:28.836 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38113 OpHistory ceph rtd unknown /proc/38079/task/38113/root (readlink: Permission denied) 2024-03-30T00:37:28.836 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38113 OpHistory ceph txt unknown /proc/38079/task/38113/exe (readlink: Permission denied) 2024-03-30T00:37:28.836 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38113 OpHistory ceph NOFD /proc/38079/task/38113/fd (opendir: Permission denied) 2024-03-30T00:37:28.836 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38114 fin ceph cwd unknown /proc/38079/task/38114/cwd (readlink: Permission denied) 2024-03-30T00:37:28.836 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38114 fin ceph rtd unknown /proc/38079/task/38114/root (readlink: Permission denied) 2024-03-30T00:37:28.836 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38114 fin ceph txt unknown /proc/38079/task/38114/exe (readlink: Permission denied) 2024-03-30T00:37:28.836 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38114 fin ceph NOFD /proc/38079/task/38114/fd (opendir: Permission denied) 2024-03-30T00:37:28.836 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38115 safe_time ceph cwd unknown /proc/38079/task/38115/cwd (readlink: Permission denied) 2024-03-30T00:37:28.836 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38115 safe_time ceph rtd unknown /proc/38079/task/38115/root (readlink: Permission denied) 2024-03-30T00:37:28.836 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38115 safe_time ceph txt unknown /proc/38079/task/38115/exe (readlink: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38115 safe_time ceph NOFD /proc/38079/task/38115/fd (opendir: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38116 cpu_tp ceph cwd unknown /proc/38079/task/38116/cwd (readlink: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38116 cpu_tp ceph rtd unknown /proc/38079/task/38116/root (readlink: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38116 cpu_tp ceph txt unknown /proc/38079/task/38116/exe (readlink: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38116 cpu_tp ceph NOFD /proc/38079/task/38116/fd (opendir: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38117 cpu_tp ceph cwd unknown /proc/38079/task/38117/cwd (readlink: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38117 cpu_tp ceph rtd unknown /proc/38079/task/38117/root (readlink: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38117 cpu_tp ceph txt unknown /proc/38079/task/38117/exe (readlink: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38117 cpu_tp ceph NOFD /proc/38079/task/38117/fd (opendir: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38118 cpu_tp ceph cwd unknown /proc/38079/task/38118/cwd (readlink: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38118 cpu_tp ceph rtd unknown /proc/38079/task/38118/root (readlink: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38118 cpu_tp ceph txt unknown /proc/38079/task/38118/exe (readlink: Permission denied) 2024-03-30T00:37:28.837 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38118 cpu_tp ceph NOFD /proc/38079/task/38118/fd (opendir: Permission denied) 2024-03-30T00:37:28.838 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38119 cpu_tp ceph cwd unknown /proc/38079/task/38119/cwd (readlink: Permission denied) 2024-03-30T00:37:28.838 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38119 cpu_tp ceph rtd unknown /proc/38079/task/38119/root (readlink: Permission denied) 2024-03-30T00:37:28.838 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38119 cpu_tp ceph txt unknown /proc/38079/task/38119/exe (readlink: Permission denied) 2024-03-30T00:37:28.838 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38119 cpu_tp ceph NOFD /proc/38079/task/38119/fd (opendir: Permission denied) 2024-03-30T00:37:28.838 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38120 ms_dispat ceph cwd unknown /proc/38079/task/38120/cwd (readlink: Permission denied) 2024-03-30T00:37:28.838 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38120 ms_dispat ceph rtd unknown /proc/38079/task/38120/root (readlink: Permission denied) 2024-03-30T00:37:28.838 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38120 ms_dispat ceph txt unknown /proc/38079/task/38120/exe (readlink: Permission denied) 2024-03-30T00:37:28.838 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38120 ms_dispat ceph NOFD /proc/38079/task/38120/fd (opendir: Permission denied) 2024-03-30T00:37:28.838 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38121 ms_local ceph cwd unknown /proc/38079/task/38121/cwd (readlink: Permission denied) 2024-03-30T00:37:28.838 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38121 ms_local ceph rtd unknown /proc/38079/task/38121/root (readlink: Permission denied) 2024-03-30T00:37:28.839 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38121 ms_local ceph txt unknown /proc/38079/task/38121/exe (readlink: Permission denied) 2024-03-30T00:37:28.839 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38121 ms_local ceph NOFD /proc/38079/task/38121/fd (opendir: Permission denied) 2024-03-30T00:37:28.839 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38122 safe_time ceph cwd unknown /proc/38079/task/38122/cwd (readlink: Permission denied) 2024-03-30T00:37:28.839 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38122 safe_time ceph rtd unknown /proc/38079/task/38122/root (readlink: Permission denied) 2024-03-30T00:37:28.839 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38122 safe_time ceph txt unknown /proc/38079/task/38122/exe (readlink: Permission denied) 2024-03-30T00:37:28.839 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38122 safe_time ceph NOFD /proc/38079/task/38122/fd (opendir: Permission denied) 2024-03-30T00:37:28.839 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38123 ms_dispat ceph cwd unknown /proc/38079/task/38123/cwd (readlink: Permission denied) 2024-03-30T00:37:28.839 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38123 ms_dispat ceph rtd unknown /proc/38079/task/38123/root (readlink: Permission denied) 2024-03-30T00:37:28.839 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38123 ms_dispat ceph txt unknown /proc/38079/task/38123/exe (readlink: Permission denied) 2024-03-30T00:37:28.839 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38123 ms_dispat ceph NOFD /proc/38079/task/38123/fd (opendir: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38124 ms_local ceph cwd unknown /proc/38079/task/38124/cwd (readlink: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38124 ms_local ceph rtd unknown /proc/38079/task/38124/root (readlink: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38124 ms_local ceph txt unknown /proc/38079/task/38124/exe (readlink: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 38124 ms_local ceph NOFD /proc/38079/task/38124/fd (opendir: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 85161 rstore_co ceph cwd unknown /proc/38079/task/85161/cwd (readlink: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 85161 rstore_co ceph rtd unknown /proc/38079/task/85161/root (readlink: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 85161 rstore_co ceph txt unknown /proc/38079/task/85161/exe (readlink: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mon 38079 85161 rstore_co ceph NOFD /proc/38079/task/85161/fd (opendir: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 38332 root cwd unknown /proc/38332/cwd (readlink: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 38332 root rtd unknown /proc/38332/root (readlink: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 38332 root txt unknown /proc/38332/exe (readlink: Permission denied) 2024-03-30T00:37:28.840 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 38332 root NOFD /proc/38332/fd (opendir: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 38343 root cwd unknown /proc/38343/cwd (readlink: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 38343 root rtd unknown /proc/38343/root (readlink: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 38343 root txt unknown /proc/38343/exe (readlink: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 38343 root NOFD /proc/38343/fd (opendir: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 ceph cwd unknown /proc/38355/cwd (readlink: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 ceph rtd unknown /proc/38355/root (readlink: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 ceph txt unknown /proc/38355/exe (readlink: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 ceph NOFD /proc/38355/fd (opendir: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41314 log ceph cwd unknown /proc/38355/task/41314/cwd (readlink: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41314 log ceph rtd unknown /proc/38355/task/41314/root (readlink: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41314 log ceph txt unknown /proc/38355/task/41314/exe (readlink: Permission denied) 2024-03-30T00:37:28.841 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41314 log ceph NOFD /proc/38355/task/41314/fd (opendir: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41316 msgr-work ceph cwd unknown /proc/38355/task/41316/cwd (readlink: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41316 msgr-work ceph rtd unknown /proc/38355/task/41316/root (readlink: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41316 msgr-work ceph txt unknown /proc/38355/task/41316/exe (readlink: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41316 msgr-work ceph NOFD /proc/38355/task/41316/fd (opendir: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41317 msgr-work ceph cwd unknown /proc/38355/task/41317/cwd (readlink: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41317 msgr-work ceph rtd unknown /proc/38355/task/41317/root (readlink: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41317 msgr-work ceph txt unknown /proc/38355/task/41317/exe (readlink: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41317 msgr-work ceph NOFD /proc/38355/task/41317/fd (opendir: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41318 msgr-work ceph cwd unknown /proc/38355/task/41318/cwd (readlink: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41318 msgr-work ceph rtd unknown /proc/38355/task/41318/root (readlink: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41318 msgr-work ceph txt unknown /proc/38355/task/41318/exe (readlink: Permission denied) 2024-03-30T00:37:28.842 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41318 msgr-work ceph NOFD /proc/38355/task/41318/fd (opendir: Permission denied) 2024-03-30T00:37:28.843 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41323 service ceph cwd unknown /proc/38355/task/41323/cwd (readlink: Permission denied) 2024-03-30T00:37:28.843 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41323 service ceph rtd unknown /proc/38355/task/41323/root (readlink: Permission denied) 2024-03-30T00:37:28.843 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41323 service ceph txt unknown /proc/38355/task/41323/exe (readlink: Permission denied) 2024-03-30T00:37:28.843 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41323 service ceph NOFD /proc/38355/task/41323/fd (opendir: Permission denied) 2024-03-30T00:37:28.843 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41324 admin_soc ceph cwd unknown /proc/38355/task/41324/cwd (readlink: Permission denied) 2024-03-30T00:37:28.843 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41324 admin_soc ceph rtd unknown /proc/38355/task/41324/root (readlink: Permission denied) 2024-03-30T00:37:28.843 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41324 admin_soc ceph txt unknown /proc/38355/task/41324/exe (readlink: Permission denied) 2024-03-30T00:37:28.843 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41324 admin_soc ceph NOFD /proc/38355/task/41324/fd (opendir: Permission denied) 2024-03-30T00:37:28.843 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41325 ceph_time ceph cwd unknown /proc/38355/task/41325/cwd (readlink: Permission denied) 2024-03-30T00:37:28.843 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41325 ceph_time ceph rtd unknown /proc/38355/task/41325/root (readlink: Permission denied) 2024-03-30T00:37:28.843 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41325 ceph_time ceph txt unknown /proc/38355/task/41325/exe (readlink: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41325 ceph_time ceph NOFD /proc/38355/task/41325/fd (opendir: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41326 fn_anonym ceph cwd unknown /proc/38355/task/41326/cwd (readlink: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41326 fn_anonym ceph rtd unknown /proc/38355/task/41326/root (readlink: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41326 fn_anonym ceph txt unknown /proc/38355/task/41326/exe (readlink: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41326 fn_anonym ceph NOFD /proc/38355/task/41326/fd (opendir: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41327 signal_ha ceph cwd unknown /proc/38355/task/41327/cwd (readlink: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41327 signal_ha ceph rtd unknown /proc/38355/task/41327/root (readlink: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41327 signal_ha ceph txt unknown /proc/38355/task/41327/exe (readlink: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41327 signal_ha ceph NOFD /proc/38355/task/41327/fd (opendir: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41328 mgrsb-fin ceph cwd unknown /proc/38355/task/41328/cwd (readlink: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41328 mgrsb-fin ceph rtd unknown /proc/38355/task/41328/root (readlink: Permission denied) 2024-03-30T00:37:28.844 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41328 mgrsb-fin ceph txt unknown /proc/38355/task/41328/exe (readlink: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41328 mgrsb-fin ceph NOFD /proc/38355/task/41328/fd (opendir: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41329 ms_dispat ceph cwd unknown /proc/38355/task/41329/cwd (readlink: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41329 ms_dispat ceph rtd unknown /proc/38355/task/41329/root (readlink: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41329 ms_dispat ceph txt unknown /proc/38355/task/41329/exe (readlink: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41329 ms_dispat ceph NOFD /proc/38355/task/41329/fd (opendir: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41330 ms_local ceph cwd unknown /proc/38355/task/41330/cwd (readlink: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41330 ms_local ceph rtd unknown /proc/38355/task/41330/root (readlink: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41330 ms_local ceph txt unknown /proc/38355/task/41330/exe (readlink: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41330 ms_local ceph NOFD /proc/38355/task/41330/fd (opendir: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41331 io_contex ceph cwd unknown /proc/38355/task/41331/cwd (readlink: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41331 io_contex ceph rtd unknown /proc/38355/task/41331/root (readlink: Permission denied) 2024-03-30T00:37:28.845 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41331 io_contex ceph txt unknown /proc/38355/task/41331/exe (readlink: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41331 io_contex ceph NOFD /proc/38355/task/41331/fd (opendir: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41332 io_contex ceph cwd unknown /proc/38355/task/41332/cwd (readlink: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41332 io_contex ceph rtd unknown /proc/38355/task/41332/root (readlink: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41332 io_contex ceph txt unknown /proc/38355/task/41332/exe (readlink: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41332 io_contex ceph NOFD /proc/38355/task/41332/fd (opendir: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41333 safe_time ceph cwd unknown /proc/38355/task/41333/cwd (readlink: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41333 safe_time ceph rtd unknown /proc/38355/task/41333/root (readlink: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41333 safe_time ceph txt unknown /proc/38355/task/41333/exe (readlink: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41333 safe_time ceph NOFD /proc/38355/task/41333/fd (opendir: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41334 safe_time ceph cwd unknown /proc/38355/task/41334/cwd (readlink: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41334 safe_time ceph rtd unknown /proc/38355/task/41334/root (readlink: Permission denied) 2024-03-30T00:37:28.846 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41334 safe_time ceph txt unknown /proc/38355/task/41334/exe (readlink: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41334 safe_time ceph NOFD /proc/38355/task/41334/fd (opendir: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41335 safe_time ceph cwd unknown /proc/38355/task/41335/cwd (readlink: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41335 safe_time ceph rtd unknown /proc/38355/task/41335/root (readlink: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41335 safe_time ceph txt unknown /proc/38355/task/41335/exe (readlink: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41335 safe_time ceph NOFD /proc/38355/task/41335/fd (opendir: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41336 fn_anonym ceph cwd unknown /proc/38355/task/41336/cwd (readlink: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41336 fn_anonym ceph rtd unknown /proc/38355/task/41336/root (readlink: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41336 fn_anonym ceph txt unknown /proc/38355/task/41336/exe (readlink: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41336 fn_anonym ceph NOFD /proc/38355/task/41336/fd (opendir: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41337 flusher ceph cwd unknown /proc/38355/task/41337/cwd (readlink: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41337 flusher ceph rtd unknown /proc/38355/task/41337/root (readlink: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41337 flusher ceph txt unknown /proc/38355/task/41337/exe (readlink: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41337 flusher ceph NOFD /proc/38355/task/41337/fd (opendir: Permission denied) 2024-03-30T00:37:28.847 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41338 safe_time ceph cwd unknown /proc/38355/task/41338/cwd (readlink: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41338 safe_time ceph rtd unknown /proc/38355/task/41338/root (readlink: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41338 safe_time ceph txt unknown /proc/38355/task/41338/exe (readlink: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41338 safe_time ceph NOFD /proc/38355/task/41338/fd (opendir: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41550 mgr-fin ceph cwd unknown /proc/38355/task/41550/cwd (readlink: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41550 mgr-fin ceph rtd unknown /proc/38355/task/41550/root (readlink: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41550 mgr-fin ceph txt unknown /proc/38355/task/41550/exe (readlink: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41550 mgr-fin ceph NOFD /proc/38355/task/41550/fd (opendir: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41551 ms_dispat ceph cwd unknown /proc/38355/task/41551/cwd (readlink: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41551 ms_dispat ceph rtd unknown /proc/38355/task/41551/root (readlink: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41551 ms_dispat ceph txt unknown /proc/38355/task/41551/exe (readlink: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41551 ms_dispat ceph NOFD /proc/38355/task/41551/fd (opendir: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41552 ms_local ceph cwd unknown /proc/38355/task/41552/cwd (readlink: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41552 ms_local ceph rtd unknown /proc/38355/task/41552/root (readlink: Permission denied) 2024-03-30T00:37:28.848 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41552 ms_local ceph txt unknown /proc/38355/task/41552/exe (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41552 ms_local ceph NOFD /proc/38355/task/41552/fd (opendir: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41553 safe_time ceph cwd unknown /proc/38355/task/41553/cwd (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41553 safe_time ceph rtd unknown /proc/38355/task/41553/root (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41553 safe_time ceph txt unknown /proc/38355/task/41553/exe (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41553 safe_time ceph NOFD /proc/38355/task/41553/fd (opendir: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41554 cmdfin ceph cwd unknown /proc/38355/task/41554/cwd (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41554 cmdfin ceph rtd unknown /proc/38355/task/41554/root (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41554 cmdfin ceph txt unknown /proc/38355/task/41554/exe (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41554 cmdfin ceph NOFD /proc/38355/task/41554/fd (opendir: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41558 io_contex ceph cwd unknown /proc/38355/task/41558/cwd (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41558 io_contex ceph rtd unknown /proc/38355/task/41558/root (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41558 io_contex ceph txt unknown /proc/38355/task/41558/exe (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41558 io_contex ceph NOFD /proc/38355/task/41558/fd (opendir: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41559 io_contex ceph cwd unknown /proc/38355/task/41559/cwd (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41559 io_contex ceph rtd unknown /proc/38355/task/41559/root (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41559 io_contex ceph txt unknown /proc/38355/task/41559/exe (readlink: Permission denied) 2024-03-30T00:37:28.849 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41559 io_contex ceph NOFD /proc/38355/task/41559/fd (opendir: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41560 ceph_time ceph cwd unknown /proc/38355/task/41560/cwd (readlink: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41560 ceph_time ceph rtd unknown /proc/38355/task/41560/root (readlink: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41560 ceph_time ceph txt unknown /proc/38355/task/41560/exe (readlink: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41560 ceph_time ceph NOFD /proc/38355/task/41560/fd (opendir: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41561 ms_dispat ceph cwd unknown /proc/38355/task/41561/cwd (readlink: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41561 ms_dispat ceph rtd unknown /proc/38355/task/41561/root (readlink: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41561 ms_dispat ceph txt unknown /proc/38355/task/41561/exe (readlink: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41561 ms_dispat ceph NOFD /proc/38355/task/41561/fd (opendir: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41562 ms_local ceph cwd unknown /proc/38355/task/41562/cwd (readlink: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41562 ms_local ceph rtd unknown /proc/38355/task/41562/root (readlink: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41562 ms_local ceph txt unknown /proc/38355/task/41562/exe (readlink: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41562 ms_local ceph NOFD /proc/38355/task/41562/fd (opendir: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41563 safe_time ceph cwd unknown /proc/38355/task/41563/cwd (readlink: Permission denied) 2024-03-30T00:37:28.850 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41563 safe_time ceph rtd unknown /proc/38355/task/41563/root (readlink: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41563 safe_time ceph txt unknown /proc/38355/task/41563/exe (readlink: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41563 safe_time ceph NOFD /proc/38355/task/41563/fd (opendir: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41564 safe_time ceph cwd unknown /proc/38355/task/41564/cwd (readlink: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41564 safe_time ceph rtd unknown /proc/38355/task/41564/root (readlink: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41564 safe_time ceph txt unknown /proc/38355/task/41564/exe (readlink: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41564 safe_time ceph NOFD /proc/38355/task/41564/fd (opendir: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41565 balancer ceph cwd unknown /proc/38355/task/41565/cwd (readlink: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41565 balancer ceph rtd unknown /proc/38355/task/41565/root (readlink: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41565 balancer ceph txt unknown /proc/38355/task/41565/exe (readlink: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41565 balancer ceph NOFD /proc/38355/task/41565/fd (opendir: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41566 m-fin-bal ceph cwd unknown /proc/38355/task/41566/cwd (readlink: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41566 m-fin-bal ceph rtd unknown /proc/38355/task/41566/root (readlink: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41566 m-fin-bal ceph txt unknown /proc/38355/task/41566/exe (readlink: Permission denied) 2024-03-30T00:37:28.851 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41566 m-fin-bal ceph NOFD /proc/38355/task/41566/fd (opendir: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41567 mgr-fin ceph cwd unknown /proc/38355/task/41567/cwd (readlink: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41567 mgr-fin ceph rtd unknown /proc/38355/task/41567/root (readlink: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41567 mgr-fin ceph txt unknown /proc/38355/task/41567/exe (readlink: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41567 mgr-fin ceph NOFD /proc/38355/task/41567/fd (opendir: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41568 mgr-fin ceph cwd unknown /proc/38355/task/41568/cwd (readlink: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41568 mgr-fin ceph rtd unknown /proc/38355/task/41568/root (readlink: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41568 mgr-fin ceph txt unknown /proc/38355/task/41568/exe (readlink: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41568 mgr-fin ceph NOFD /proc/38355/task/41568/fd (opendir: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41569 mgr-fin ceph cwd unknown /proc/38355/task/41569/cwd (readlink: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41569 mgr-fin ceph rtd unknown /proc/38355/task/41569/root (readlink: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41569 mgr-fin ceph txt unknown /proc/38355/task/41569/exe (readlink: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41569 mgr-fin ceph NOFD /proc/38355/task/41569/fd (opendir: Permission denied) 2024-03-30T00:37:28.852 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41570 mgr-fin ceph cwd unknown /proc/38355/task/41570/cwd (readlink: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41570 mgr-fin ceph rtd unknown /proc/38355/task/41570/root (readlink: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41570 mgr-fin ceph txt unknown /proc/38355/task/41570/exe (readlink: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41570 mgr-fin ceph NOFD /proc/38355/task/41570/fd (opendir: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41571 mgr-fin ceph cwd unknown /proc/38355/task/41571/cwd (readlink: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41571 mgr-fin ceph rtd unknown /proc/38355/task/41571/root (readlink: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41571 mgr-fin ceph txt unknown /proc/38355/task/41571/exe (readlink: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41571 mgr-fin ceph NOFD /proc/38355/task/41571/fd (opendir: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41572 mgr-fin ceph cwd unknown /proc/38355/task/41572/cwd (readlink: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41572 mgr-fin ceph rtd unknown /proc/38355/task/41572/root (readlink: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41572 mgr-fin ceph txt unknown /proc/38355/task/41572/exe (readlink: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41572 mgr-fin ceph NOFD /proc/38355/task/41572/fd (opendir: Permission denied) 2024-03-30T00:37:28.853 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41573 mgr-fin ceph cwd unknown /proc/38355/task/41573/cwd (readlink: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41573 mgr-fin ceph rtd unknown /proc/38355/task/41573/root (readlink: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41573 mgr-fin ceph txt unknown /proc/38355/task/41573/exe (readlink: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41573 mgr-fin ceph NOFD /proc/38355/task/41573/fd (opendir: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41574 mgr-fin ceph cwd unknown /proc/38355/task/41574/cwd (readlink: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41574 mgr-fin ceph rtd unknown /proc/38355/task/41574/root (readlink: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41574 mgr-fin ceph txt unknown /proc/38355/task/41574/exe (readlink: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41574 mgr-fin ceph NOFD /proc/38355/task/41574/fd (opendir: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41575 mgr-fin ceph cwd unknown /proc/38355/task/41575/cwd (readlink: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41575 mgr-fin ceph rtd unknown /proc/38355/task/41575/root (readlink: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41575 mgr-fin ceph txt unknown /proc/38355/task/41575/exe (readlink: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41575 mgr-fin ceph NOFD /proc/38355/task/41575/fd (opendir: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41576 mgr-fin ceph cwd unknown /proc/38355/task/41576/cwd (readlink: Permission denied) 2024-03-30T00:37:28.854 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41576 mgr-fin ceph rtd unknown /proc/38355/task/41576/root (readlink: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41576 mgr-fin ceph txt unknown /proc/38355/task/41576/exe (readlink: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41576 mgr-fin ceph NOFD /proc/38355/task/41576/fd (opendir: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41577 mgr-fin ceph cwd unknown /proc/38355/task/41577/cwd (readlink: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41577 mgr-fin ceph rtd unknown /proc/38355/task/41577/root (readlink: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41577 mgr-fin ceph txt unknown /proc/38355/task/41577/exe (readlink: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41577 mgr-fin ceph NOFD /proc/38355/task/41577/fd (opendir: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41578 mgr-fin ceph cwd unknown /proc/38355/task/41578/cwd (readlink: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41578 mgr-fin ceph rtd unknown /proc/38355/task/41578/root (readlink: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41578 mgr-fin ceph txt unknown /proc/38355/task/41578/exe (readlink: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41578 mgr-fin ceph NOFD /proc/38355/task/41578/fd (opendir: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41579 mgr-fin ceph cwd unknown /proc/38355/task/41579/cwd (readlink: Permission denied) 2024-03-30T00:37:28.855 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41579 mgr-fin ceph rtd unknown /proc/38355/task/41579/root (readlink: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41579 mgr-fin ceph txt unknown /proc/38355/task/41579/exe (readlink: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41579 mgr-fin ceph NOFD /proc/38355/task/41579/fd (opendir: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41580 mgr-fin ceph cwd unknown /proc/38355/task/41580/cwd (readlink: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41580 mgr-fin ceph rtd unknown /proc/38355/task/41580/root (readlink: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41580 mgr-fin ceph txt unknown /proc/38355/task/41580/exe (readlink: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41580 mgr-fin ceph NOFD /proc/38355/task/41580/fd (opendir: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41581 mgr-fin ceph cwd unknown /proc/38355/task/41581/cwd (readlink: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41581 mgr-fin ceph rtd unknown /proc/38355/task/41581/root (readlink: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41581 mgr-fin ceph txt unknown /proc/38355/task/41581/exe (readlink: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41581 mgr-fin ceph NOFD /proc/38355/task/41581/fd (opendir: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41582 mgr-fin ceph cwd unknown /proc/38355/task/41582/cwd (readlink: Permission denied) 2024-03-30T00:37:28.856 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41582 mgr-fin ceph rtd unknown /proc/38355/task/41582/root (readlink: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41582 mgr-fin ceph txt unknown /proc/38355/task/41582/exe (readlink: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41582 mgr-fin ceph NOFD /proc/38355/task/41582/fd (opendir: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41583 cephadm ceph cwd unknown /proc/38355/task/41583/cwd (readlink: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41583 cephadm ceph rtd unknown /proc/38355/task/41583/root (readlink: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41583 cephadm ceph txt unknown /proc/38355/task/41583/exe (readlink: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41583 cephadm ceph NOFD /proc/38355/task/41583/fd (opendir: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41584 m-fin-cep ceph cwd unknown /proc/38355/task/41584/cwd (readlink: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41584 m-fin-cep ceph rtd unknown /proc/38355/task/41584/root (readlink: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41584 m-fin-cep ceph txt unknown /proc/38355/task/41584/exe (readlink: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41584 m-fin-cep ceph NOFD /proc/38355/task/41584/fd (opendir: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41585 mgr-fin ceph cwd unknown /proc/38355/task/41585/cwd (readlink: Permission denied) 2024-03-30T00:37:28.857 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41585 mgr-fin ceph rtd unknown /proc/38355/task/41585/root (readlink: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41585 mgr-fin ceph txt unknown /proc/38355/task/41585/exe (readlink: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41585 mgr-fin ceph NOFD /proc/38355/task/41585/fd (opendir: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41586 crash ceph cwd unknown /proc/38355/task/41586/cwd (readlink: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41586 crash ceph rtd unknown /proc/38355/task/41586/root (readlink: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41586 crash ceph txt unknown /proc/38355/task/41586/exe (readlink: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41586 crash ceph NOFD /proc/38355/task/41586/fd (opendir: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41587 m-fin-cra ceph cwd unknown /proc/38355/task/41587/cwd (readlink: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41587 m-fin-cra ceph rtd unknown /proc/38355/task/41587/root (readlink: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41587 m-fin-cra ceph txt unknown /proc/38355/task/41587/exe (readlink: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41587 m-fin-cra ceph NOFD /proc/38355/task/41587/fd (opendir: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41588 mgr-fin ceph cwd unknown /proc/38355/task/41588/cwd (readlink: Permission denied) 2024-03-30T00:37:28.858 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41588 mgr-fin ceph rtd unknown /proc/38355/task/41588/root (readlink: Permission denied) 2024-03-30T00:37:28.859 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41588 mgr-fin ceph txt unknown /proc/38355/task/41588/exe (readlink: Permission denied) 2024-03-30T00:37:28.859 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41588 mgr-fin ceph NOFD /proc/38355/task/41588/fd (opendir: Permission denied) 2024-03-30T00:37:28.859 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41589 cephadm ceph cwd unknown /proc/38355/task/41589/cwd (readlink: Permission denied) 2024-03-30T00:37:28.859 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41589 cephadm ceph rtd unknown /proc/38355/task/41589/root (readlink: Permission denied) 2024-03-30T00:37:28.859 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41589 cephadm ceph txt unknown /proc/38355/task/41589/exe (readlink: Permission denied) 2024-03-30T00:37:28.859 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41589 cephadm ceph NOFD /proc/38355/task/41589/fd (opendir: Permission denied) 2024-03-30T00:37:28.859 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41590 mgr-fin ceph cwd unknown /proc/38355/task/41590/cwd (readlink: Permission denied) 2024-03-30T00:37:28.859 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41590 mgr-fin ceph rtd unknown /proc/38355/task/41590/root (readlink: Permission denied) 2024-03-30T00:37:28.859 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41590 mgr-fin ceph txt unknown /proc/38355/task/41590/exe (readlink: Permission denied) 2024-03-30T00:37:28.859 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41590 mgr-fin ceph NOFD /proc/38355/task/41590/fd (opendir: Permission denied) 2024-03-30T00:37:28.859 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41591 dashboard ceph cwd unknown /proc/38355/task/41591/cwd (readlink: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41591 dashboard ceph rtd unknown /proc/38355/task/41591/root (readlink: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41591 dashboard ceph txt unknown /proc/38355/task/41591/exe (readlink: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41591 dashboard ceph NOFD /proc/38355/task/41591/fd (opendir: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41592 m-fin-das ceph cwd unknown /proc/38355/task/41592/cwd (readlink: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41592 m-fin-das ceph rtd unknown /proc/38355/task/41592/root (readlink: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41592 m-fin-das ceph txt unknown /proc/38355/task/41592/exe (readlink: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41592 m-fin-das ceph NOFD /proc/38355/task/41592/fd (opendir: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41593 mgr-fin ceph cwd unknown /proc/38355/task/41593/cwd (readlink: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41593 mgr-fin ceph rtd unknown /proc/38355/task/41593/root (readlink: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41593 mgr-fin ceph txt unknown /proc/38355/task/41593/exe (readlink: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41593 mgr-fin ceph NOFD /proc/38355/task/41593/fd (opendir: Permission denied) 2024-03-30T00:37:28.860 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41594 devicehea ceph cwd unknown /proc/38355/task/41594/cwd (readlink: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41594 devicehea ceph rtd unknown /proc/38355/task/41594/root (readlink: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41594 devicehea ceph txt unknown /proc/38355/task/41594/exe (readlink: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41594 devicehea ceph NOFD /proc/38355/task/41594/fd (opendir: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41595 mgr-fin ceph cwd unknown /proc/38355/task/41595/cwd (readlink: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41595 mgr-fin ceph rtd unknown /proc/38355/task/41595/root (readlink: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41595 mgr-fin ceph txt unknown /proc/38355/task/41595/exe (readlink: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41595 mgr-fin ceph NOFD /proc/38355/task/41595/fd (opendir: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41596 m-fin-dev ceph cwd unknown /proc/38355/task/41596/cwd (readlink: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41596 m-fin-dev ceph rtd unknown /proc/38355/task/41596/root (readlink: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41596 m-fin-dev ceph txt unknown /proc/38355/task/41596/exe (readlink: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41596 m-fin-dev ceph NOFD /proc/38355/task/41596/fd (opendir: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41597 mgr-fin ceph cwd unknown /proc/38355/task/41597/cwd (readlink: Permission denied) 2024-03-30T00:37:28.861 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41597 mgr-fin ceph rtd unknown /proc/38355/task/41597/root (readlink: Permission denied) 2024-03-30T00:37:28.862 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41597 mgr-fin ceph txt unknown /proc/38355/task/41597/exe (readlink: Permission denied) 2024-03-30T00:37:28.862 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41597 mgr-fin ceph NOFD /proc/38355/task/41597/fd (opendir: Permission denied) 2024-03-30T00:37:28.862 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41599 mgr-fin ceph cwd unknown /proc/38355/task/41599/cwd (readlink: Permission denied) 2024-03-30T00:37:28.862 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41599 mgr-fin ceph rtd unknown /proc/38355/task/41599/root (readlink: Permission denied) 2024-03-30T00:37:28.862 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41599 mgr-fin ceph txt unknown /proc/38355/task/41599/exe (readlink: Permission denied) 2024-03-30T00:37:28.862 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41599 mgr-fin ceph NOFD /proc/38355/task/41599/fd (opendir: Permission denied) 2024-03-30T00:37:28.862 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41600 m-fin-ios ceph cwd unknown /proc/38355/task/41600/cwd (readlink: Permission denied) 2024-03-30T00:37:28.862 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41600 m-fin-ios ceph rtd unknown /proc/38355/task/41600/root (readlink: Permission denied) 2024-03-30T00:37:28.862 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41600 m-fin-ios ceph txt unknown /proc/38355/task/41600/exe (readlink: Permission denied) 2024-03-30T00:37:28.862 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41600 m-fin-ios ceph NOFD /proc/38355/task/41600/fd (opendir: Permission denied) 2024-03-30T00:37:28.862 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41602 m-fin-nfs ceph cwd unknown /proc/38355/task/41602/cwd (readlink: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41602 m-fin-nfs ceph rtd unknown /proc/38355/task/41602/root (readlink: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41602 m-fin-nfs ceph txt unknown /proc/38355/task/41602/exe (readlink: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41602 m-fin-nfs ceph NOFD /proc/38355/task/41602/fd (opendir: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41604 m-fin-orc ceph cwd unknown /proc/38355/task/41604/cwd (readlink: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41604 m-fin-orc ceph rtd unknown /proc/38355/task/41604/root (readlink: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41604 m-fin-orc ceph txt unknown /proc/38355/task/41604/exe (readlink: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41604 m-fin-orc ceph NOFD /proc/38355/task/41604/fd (opendir: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41605 mgr-fin ceph cwd unknown /proc/38355/task/41605/cwd (readlink: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41605 mgr-fin ceph rtd unknown /proc/38355/task/41605/root (readlink: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41605 mgr-fin ceph txt unknown /proc/38355/task/41605/exe (readlink: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41605 mgr-fin ceph NOFD /proc/38355/task/41605/fd (opendir: Permission denied) 2024-03-30T00:37:28.863 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41606 pg_autosc ceph cwd unknown /proc/38355/task/41606/cwd (readlink: Permission denied) 2024-03-30T00:37:28.864 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41606 pg_autosc ceph rtd unknown /proc/38355/task/41606/root (readlink: Permission denied) 2024-03-30T00:37:28.864 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41606 pg_autosc ceph txt unknown /proc/38355/task/41606/exe (readlink: Permission denied) 2024-03-30T00:37:28.864 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41606 pg_autosc ceph NOFD /proc/38355/task/41606/fd (opendir: Permission denied) 2024-03-30T00:37:28.864 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41607 m-fin-pg_ ceph cwd unknown /proc/38355/task/41607/cwd (readlink: Permission denied) 2024-03-30T00:37:28.864 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41607 m-fin-pg_ ceph rtd unknown /proc/38355/task/41607/root (readlink: Permission denied) 2024-03-30T00:37:28.864 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41607 m-fin-pg_ ceph txt unknown /proc/38355/task/41607/exe (readlink: Permission denied) 2024-03-30T00:37:28.864 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41607 m-fin-pg_ ceph NOFD /proc/38355/task/41607/fd (opendir: Permission denied) 2024-03-30T00:37:28.864 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41608 progress ceph cwd unknown /proc/38355/task/41608/cwd (readlink: Permission denied) 2024-03-30T00:37:28.864 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41608 progress ceph rtd unknown /proc/38355/task/41608/root (readlink: Permission denied) 2024-03-30T00:37:28.864 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41608 progress ceph txt unknown /proc/38355/task/41608/exe (readlink: Permission denied) 2024-03-30T00:37:28.864 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41608 progress ceph NOFD /proc/38355/task/41608/fd (opendir: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41609 m-fin-pro ceph cwd unknown /proc/38355/task/41609/cwd (readlink: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41609 m-fin-pro ceph rtd unknown /proc/38355/task/41609/root (readlink: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41609 m-fin-pro ceph txt unknown /proc/38355/task/41609/exe (readlink: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41609 m-fin-pro ceph NOFD /proc/38355/task/41609/fd (opendir: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41610 mgr-fin ceph cwd unknown /proc/38355/task/41610/cwd (readlink: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41610 mgr-fin ceph rtd unknown /proc/38355/task/41610/root (readlink: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41610 mgr-fin ceph txt unknown /proc/38355/task/41610/exe (readlink: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41610 mgr-fin ceph NOFD /proc/38355/task/41610/fd (opendir: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41611 mgr-fin ceph cwd unknown /proc/38355/task/41611/cwd (readlink: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41611 mgr-fin ceph rtd unknown /proc/38355/task/41611/root (readlink: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41611 mgr-fin ceph txt unknown /proc/38355/task/41611/exe (readlink: Permission denied) 2024-03-30T00:37:28.865 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41611 mgr-fin ceph NOFD /proc/38355/task/41611/fd (opendir: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41612 mgr-fin ceph cwd unknown /proc/38355/task/41612/cwd (readlink: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41612 mgr-fin ceph rtd unknown /proc/38355/task/41612/root (readlink: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41612 mgr-fin ceph txt unknown /proc/38355/task/41612/exe (readlink: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41612 mgr-fin ceph NOFD /proc/38355/task/41612/fd (opendir: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41615 mgr-fin ceph cwd unknown /proc/38355/task/41615/cwd (readlink: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41615 mgr-fin ceph rtd unknown /proc/38355/task/41615/root (readlink: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41615 mgr-fin ceph txt unknown /proc/38355/task/41615/exe (readlink: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41615 mgr-fin ceph NOFD /proc/38355/task/41615/fd (opendir: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41617 io_contex ceph cwd unknown /proc/38355/task/41617/cwd (readlink: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41617 io_contex ceph rtd unknown /proc/38355/task/41617/root (readlink: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41617 io_contex ceph txt unknown /proc/38355/task/41617/exe (readlink: Permission denied) 2024-03-30T00:37:28.866 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41617 io_contex ceph NOFD /proc/38355/task/41617/fd (opendir: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41618 io_contex ceph cwd unknown /proc/38355/task/41618/cwd (readlink: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41618 io_contex ceph rtd unknown /proc/38355/task/41618/root (readlink: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41618 io_contex ceph txt unknown /proc/38355/task/41618/exe (readlink: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41618 io_contex ceph NOFD /proc/38355/task/41618/fd (opendir: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41619 ceph_time ceph cwd unknown /proc/38355/task/41619/cwd (readlink: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41619 ceph_time ceph rtd unknown /proc/38355/task/41619/root (readlink: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41619 ceph_time ceph txt unknown /proc/38355/task/41619/exe (readlink: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41619 ceph_time ceph NOFD /proc/38355/task/41619/fd (opendir: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41620 ms_dispat ceph cwd unknown /proc/38355/task/41620/cwd (readlink: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41620 ms_dispat ceph rtd unknown /proc/38355/task/41620/root (readlink: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41620 ms_dispat ceph txt unknown /proc/38355/task/41620/exe (readlink: Permission denied) 2024-03-30T00:37:28.867 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41620 ms_dispat ceph NOFD /proc/38355/task/41620/fd (opendir: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41621 ms_local ceph cwd unknown /proc/38355/task/41621/cwd (readlink: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41621 ms_local ceph rtd unknown /proc/38355/task/41621/root (readlink: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41621 ms_local ceph txt unknown /proc/38355/task/41621/exe (readlink: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41621 ms_local ceph NOFD /proc/38355/task/41621/fd (opendir: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41622 safe_time ceph cwd unknown /proc/38355/task/41622/cwd (readlink: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41622 safe_time ceph rtd unknown /proc/38355/task/41622/root (readlink: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41622 safe_time ceph txt unknown /proc/38355/task/41622/exe (readlink: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41622 safe_time ceph NOFD /proc/38355/task/41622/fd (opendir: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41623 safe_time ceph cwd unknown /proc/38355/task/41623/cwd (readlink: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41623 safe_time ceph rtd unknown /proc/38355/task/41623/root (readlink: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41623 safe_time ceph txt unknown /proc/38355/task/41623/exe (readlink: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41623 safe_time ceph NOFD /proc/38355/task/41623/fd (opendir: Permission denied) 2024-03-30T00:37:28.868 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41624 mgr-fin ceph cwd unknown /proc/38355/task/41624/cwd (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41624 mgr-fin ceph rtd unknown /proc/38355/task/41624/root (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41624 mgr-fin ceph txt unknown /proc/38355/task/41624/exe (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41624 mgr-fin ceph NOFD /proc/38355/task/41624/fd (opendir: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41626 m-fin-rbd ceph cwd unknown /proc/38355/task/41626/cwd (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41626 m-fin-rbd ceph rtd unknown /proc/38355/task/41626/root (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41626 m-fin-rbd ceph txt unknown /proc/38355/task/41626/exe (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41626 m-fin-rbd ceph NOFD /proc/38355/task/41626/fd (opendir: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41627 restful ceph cwd unknown /proc/38355/task/41627/cwd (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41627 restful ceph rtd unknown /proc/38355/task/41627/root (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41627 restful ceph txt unknown /proc/38355/task/41627/exe (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41627 restful ceph NOFD /proc/38355/task/41627/fd (opendir: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41628 m-fin-res ceph cwd unknown /proc/38355/task/41628/cwd (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41628 m-fin-res ceph rtd unknown /proc/38355/task/41628/root (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41628 m-fin-res ceph txt unknown /proc/38355/task/41628/exe (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41628 m-fin-res ceph NOFD /proc/38355/task/41628/fd (opendir: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41630 m-fin-sta ceph cwd unknown /proc/38355/task/41630/cwd (readlink: Permission denied) 2024-03-30T00:37:28.869 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41630 m-fin-sta ceph rtd unknown /proc/38355/task/41630/root (readlink: Permission denied) 2024-03-30T00:37:28.870 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41630 m-fin-sta ceph txt unknown /proc/38355/task/41630/exe (readlink: Permission denied) 2024-03-30T00:37:28.870 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41630 m-fin-sta ceph NOFD /proc/38355/task/41630/fd (opendir: Permission denied) 2024-03-30T00:37:28.870 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41631 mgr-fin ceph cwd unknown /proc/38355/task/41631/cwd (readlink: Permission denied) 2024-03-30T00:37:28.870 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41631 mgr-fin ceph rtd unknown /proc/38355/task/41631/root (readlink: Permission denied) 2024-03-30T00:37:28.870 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41631 mgr-fin ceph txt unknown /proc/38355/task/41631/exe (readlink: Permission denied) 2024-03-30T00:37:28.870 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41631 mgr-fin ceph NOFD /proc/38355/task/41631/fd (opendir: Permission denied) 2024-03-30T00:37:28.870 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41632 telemetry ceph cwd unknown /proc/38355/task/41632/cwd (readlink: Permission denied) 2024-03-30T00:37:28.870 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41632 telemetry ceph rtd unknown /proc/38355/task/41632/root (readlink: Permission denied) 2024-03-30T00:37:28.870 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41632 telemetry ceph txt unknown /proc/38355/task/41632/exe (readlink: Permission denied) 2024-03-30T00:37:28.871 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41632 telemetry ceph NOFD /proc/38355/task/41632/fd (opendir: Permission denied) 2024-03-30T00:37:28.871 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41633 m-fin-tel ceph cwd unknown /proc/38355/task/41633/cwd (readlink: Permission denied) 2024-03-30T00:37:28.871 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41633 m-fin-tel ceph rtd unknown /proc/38355/task/41633/root (readlink: Permission denied) 2024-03-30T00:37:28.871 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41633 m-fin-tel ceph txt unknown /proc/38355/task/41633/exe (readlink: Permission denied) 2024-03-30T00:37:28.871 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41633 m-fin-tel ceph NOFD /proc/38355/task/41633/fd (opendir: Permission denied) 2024-03-30T00:37:28.871 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41634 mgr-fin ceph cwd unknown /proc/38355/task/41634/cwd (readlink: Permission denied) 2024-03-30T00:37:28.871 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41634 mgr-fin ceph rtd unknown /proc/38355/task/41634/root (readlink: Permission denied) 2024-03-30T00:37:28.871 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41634 mgr-fin ceph txt unknown /proc/38355/task/41634/exe (readlink: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41634 mgr-fin ceph NOFD /proc/38355/task/41634/fd (opendir: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41635 mgr-fin ceph cwd unknown /proc/38355/task/41635/cwd (readlink: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41635 mgr-fin ceph rtd unknown /proc/38355/task/41635/root (readlink: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41635 mgr-fin ceph txt unknown /proc/38355/task/41635/exe (readlink: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41635 mgr-fin ceph NOFD /proc/38355/task/41635/fd (opendir: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41636 mgr-fin ceph cwd unknown /proc/38355/task/41636/cwd (readlink: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41636 mgr-fin ceph rtd unknown /proc/38355/task/41636/root (readlink: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41636 mgr-fin ceph txt unknown /proc/38355/task/41636/exe (readlink: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41636 mgr-fin ceph NOFD /proc/38355/task/41636/fd (opendir: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41640 mgr-fin ceph cwd unknown /proc/38355/task/41640/cwd (readlink: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41640 mgr-fin ceph rtd unknown /proc/38355/task/41640/root (readlink: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41640 mgr-fin ceph txt unknown /proc/38355/task/41640/exe (readlink: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41640 mgr-fin ceph NOFD /proc/38355/task/41640/fd (opendir: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41641 io_contex ceph cwd unknown /proc/38355/task/41641/cwd (readlink: Permission denied) 2024-03-30T00:37:28.872 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41641 io_contex ceph rtd unknown /proc/38355/task/41641/root (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41641 io_contex ceph txt unknown /proc/38355/task/41641/exe (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41641 io_contex ceph NOFD /proc/38355/task/41641/fd (opendir: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41642 io_contex ceph cwd unknown /proc/38355/task/41642/cwd (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41642 io_contex ceph rtd unknown /proc/38355/task/41642/root (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41642 io_contex ceph txt unknown /proc/38355/task/41642/exe (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41642 io_contex ceph NOFD /proc/38355/task/41642/fd (opendir: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41643 ceph_time ceph cwd unknown /proc/38355/task/41643/cwd (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41643 ceph_time ceph rtd unknown /proc/38355/task/41643/root (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41643 ceph_time ceph txt unknown /proc/38355/task/41643/exe (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41643 ceph_time ceph NOFD /proc/38355/task/41643/fd (opendir: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41644 ms_dispat ceph cwd unknown /proc/38355/task/41644/cwd (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41644 ms_dispat ceph rtd unknown /proc/38355/task/41644/root (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41644 ms_dispat ceph txt unknown /proc/38355/task/41644/exe (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41644 ms_dispat ceph NOFD /proc/38355/task/41644/fd (opendir: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41645 ms_local ceph cwd unknown /proc/38355/task/41645/cwd (readlink: Permission denied) 2024-03-30T00:37:28.873 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41645 ms_local ceph rtd unknown /proc/38355/task/41645/root (readlink: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41645 ms_local ceph txt unknown /proc/38355/task/41645/exe (readlink: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41645 ms_local ceph NOFD /proc/38355/task/41645/fd (opendir: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41646 safe_time ceph cwd unknown /proc/38355/task/41646/cwd (readlink: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41646 safe_time ceph rtd unknown /proc/38355/task/41646/root (readlink: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41646 safe_time ceph txt unknown /proc/38355/task/41646/exe (readlink: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41646 safe_time ceph NOFD /proc/38355/task/41646/fd (opendir: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41647 safe_time ceph cwd unknown /proc/38355/task/41647/cwd (readlink: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41647 safe_time ceph rtd unknown /proc/38355/task/41647/root (readlink: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41647 safe_time ceph txt unknown /proc/38355/task/41647/exe (readlink: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41647 safe_time ceph NOFD /proc/38355/task/41647/fd (opendir: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41648 mgr-fin ceph cwd unknown /proc/38355/task/41648/cwd (readlink: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41648 mgr-fin ceph rtd unknown /proc/38355/task/41648/root (readlink: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41648 mgr-fin ceph txt unknown /proc/38355/task/41648/exe (readlink: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41648 mgr-fin ceph NOFD /proc/38355/task/41648/fd (opendir: Permission denied) 2024-03-30T00:37:28.874 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41649 mgr-fin ceph cwd unknown /proc/38355/task/41649/cwd (readlink: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41649 mgr-fin ceph rtd unknown /proc/38355/task/41649/root (readlink: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41649 mgr-fin ceph txt unknown /proc/38355/task/41649/exe (readlink: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41649 mgr-fin ceph NOFD /proc/38355/task/41649/fd (opendir: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41650 mgr-fin ceph cwd unknown /proc/38355/task/41650/cwd (readlink: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41650 mgr-fin ceph rtd unknown /proc/38355/task/41650/root (readlink: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41650 mgr-fin ceph txt unknown /proc/38355/task/41650/exe (readlink: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41650 mgr-fin ceph NOFD /proc/38355/task/41650/fd (opendir: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41651 mgr-fin ceph cwd unknown /proc/38355/task/41651/cwd (readlink: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41651 mgr-fin ceph rtd unknown /proc/38355/task/41651/root (readlink: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41651 mgr-fin ceph txt unknown /proc/38355/task/41651/exe (readlink: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41651 mgr-fin ceph NOFD /proc/38355/task/41651/fd (opendir: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41652 mgr-fin ceph cwd unknown /proc/38355/task/41652/cwd (readlink: Permission denied) 2024-03-30T00:37:28.875 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41652 mgr-fin ceph rtd unknown /proc/38355/task/41652/root (readlink: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41652 mgr-fin ceph txt unknown /proc/38355/task/41652/exe (readlink: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41652 mgr-fin ceph NOFD /proc/38355/task/41652/fd (opendir: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41653 mgr-fin ceph cwd unknown /proc/38355/task/41653/cwd (readlink: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41653 mgr-fin ceph rtd unknown /proc/38355/task/41653/root (readlink: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41653 mgr-fin ceph txt unknown /proc/38355/task/41653/exe (readlink: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41653 mgr-fin ceph NOFD /proc/38355/task/41653/fd (opendir: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41654 mgr-fin ceph cwd unknown /proc/38355/task/41654/cwd (readlink: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41654 mgr-fin ceph rtd unknown /proc/38355/task/41654/root (readlink: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41654 mgr-fin ceph txt unknown /proc/38355/task/41654/exe (readlink: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41654 mgr-fin ceph NOFD /proc/38355/task/41654/fd (opendir: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41655 mgr-fin ceph cwd unknown /proc/38355/task/41655/cwd (readlink: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41655 mgr-fin ceph rtd unknown /proc/38355/task/41655/root (readlink: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41655 mgr-fin ceph txt unknown /proc/38355/task/41655/exe (readlink: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41655 mgr-fin ceph NOFD /proc/38355/task/41655/fd (opendir: Permission denied) 2024-03-30T00:37:28.876 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41656 mgr-fin ceph cwd unknown /proc/38355/task/41656/cwd (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41656 mgr-fin ceph rtd unknown /proc/38355/task/41656/root (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41656 mgr-fin ceph txt unknown /proc/38355/task/41656/exe (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41656 mgr-fin ceph NOFD /proc/38355/task/41656/fd (opendir: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41658 mgr-fin ceph cwd unknown /proc/38355/task/41658/cwd (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41658 mgr-fin ceph rtd unknown /proc/38355/task/41658/root (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41658 mgr-fin ceph txt unknown /proc/38355/task/41658/exe (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41658 mgr-fin ceph NOFD /proc/38355/task/41658/fd (opendir: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41659 mgr-fin ceph cwd unknown /proc/38355/task/41659/cwd (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41659 mgr-fin ceph rtd unknown /proc/38355/task/41659/root (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41659 mgr-fin ceph txt unknown /proc/38355/task/41659/exe (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41659 mgr-fin ceph NOFD /proc/38355/task/41659/fd (opendir: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41660 mgr-fin ceph cwd unknown /proc/38355/task/41660/cwd (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41660 mgr-fin ceph rtd unknown /proc/38355/task/41660/root (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41660 mgr-fin ceph txt unknown /proc/38355/task/41660/exe (readlink: Permission denied) 2024-03-30T00:37:28.877 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41660 mgr-fin ceph NOFD /proc/38355/task/41660/fd (opendir: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41662 m-fin-vol ceph cwd unknown /proc/38355/task/41662/cwd (readlink: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41662 m-fin-vol ceph rtd unknown /proc/38355/task/41662/root (readlink: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41662 m-fin-vol ceph txt unknown /proc/38355/task/41662/exe (readlink: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41662 m-fin-vol ceph NOFD /proc/38355/task/41662/fd (opendir: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41861 mgr-fin ceph cwd unknown /proc/38355/task/41861/cwd (readlink: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41861 mgr-fin ceph rtd unknown /proc/38355/task/41861/root (readlink: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41861 mgr-fin ceph txt unknown /proc/38355/task/41861/exe (readlink: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41861 mgr-fin ceph NOFD /proc/38355/task/41861/fd (opendir: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41862 mgr-fin ceph cwd unknown /proc/38355/task/41862/cwd (readlink: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41862 mgr-fin ceph rtd unknown /proc/38355/task/41862/root (readlink: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41862 mgr-fin ceph txt unknown /proc/38355/task/41862/exe (readlink: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41862 mgr-fin ceph NOFD /proc/38355/task/41862/fd (opendir: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41863 mgr-fin ceph cwd unknown /proc/38355/task/41863/cwd (readlink: Permission denied) 2024-03-30T00:37:28.878 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41863 mgr-fin ceph rtd unknown /proc/38355/task/41863/root (readlink: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41863 mgr-fin ceph txt unknown /proc/38355/task/41863/exe (readlink: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41863 mgr-fin ceph NOFD /proc/38355/task/41863/fd (opendir: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41864 mgr-fin ceph cwd unknown /proc/38355/task/41864/cwd (readlink: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41864 mgr-fin ceph rtd unknown /proc/38355/task/41864/root (readlink: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41864 mgr-fin ceph txt unknown /proc/38355/task/41864/exe (readlink: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41864 mgr-fin ceph NOFD /proc/38355/task/41864/fd (opendir: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41865 mgr-fin ceph cwd unknown /proc/38355/task/41865/cwd (readlink: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41865 mgr-fin ceph rtd unknown /proc/38355/task/41865/root (readlink: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41865 mgr-fin ceph txt unknown /proc/38355/task/41865/exe (readlink: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41865 mgr-fin ceph NOFD /proc/38355/task/41865/fd (opendir: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41866 mgr-fin ceph cwd unknown /proc/38355/task/41866/cwd (readlink: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41866 mgr-fin ceph rtd unknown /proc/38355/task/41866/root (readlink: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41866 mgr-fin ceph txt unknown /proc/38355/task/41866/exe (readlink: Permission denied) 2024-03-30T00:37:28.879 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41866 mgr-fin ceph NOFD /proc/38355/task/41866/fd (opendir: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41867 mgr-fin ceph cwd unknown /proc/38355/task/41867/cwd (readlink: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41867 mgr-fin ceph rtd unknown /proc/38355/task/41867/root (readlink: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41867 mgr-fin ceph txt unknown /proc/38355/task/41867/exe (readlink: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41867 mgr-fin ceph NOFD /proc/38355/task/41867/fd (opendir: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41868 mgr-fin ceph cwd unknown /proc/38355/task/41868/cwd (readlink: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41868 mgr-fin ceph rtd unknown /proc/38355/task/41868/root (readlink: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41868 mgr-fin ceph txt unknown /proc/38355/task/41868/exe (readlink: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41868 mgr-fin ceph NOFD /proc/38355/task/41868/fd (opendir: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41869 mgr-fin ceph cwd unknown /proc/38355/task/41869/cwd (readlink: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41869 mgr-fin ceph rtd unknown /proc/38355/task/41869/root (readlink: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41869 mgr-fin ceph txt unknown /proc/38355/task/41869/exe (readlink: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41869 mgr-fin ceph NOFD /proc/38355/task/41869/fd (opendir: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41870 mgr-fin ceph cwd unknown /proc/38355/task/41870/cwd (readlink: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41870 mgr-fin ceph rtd unknown /proc/38355/task/41870/root (readlink: Permission denied) 2024-03-30T00:37:28.880 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41870 mgr-fin ceph txt unknown /proc/38355/task/41870/exe (readlink: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41870 mgr-fin ceph NOFD /proc/38355/task/41870/fd (opendir: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41871 mgr-fin ceph cwd unknown /proc/38355/task/41871/cwd (readlink: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41871 mgr-fin ceph rtd unknown /proc/38355/task/41871/root (readlink: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41871 mgr-fin ceph txt unknown /proc/38355/task/41871/exe (readlink: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41871 mgr-fin ceph NOFD /proc/38355/task/41871/fd (opendir: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41901 mgr-fin ceph cwd unknown /proc/38355/task/41901/cwd (readlink: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41901 mgr-fin ceph rtd unknown /proc/38355/task/41901/root (readlink: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41901 mgr-fin ceph txt unknown /proc/38355/task/41901/exe (readlink: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41901 mgr-fin ceph NOFD /proc/38355/task/41901/fd (opendir: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41903 mgr-fin ceph cwd unknown /proc/38355/task/41903/cwd (readlink: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41903 mgr-fin ceph rtd unknown /proc/38355/task/41903/root (readlink: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41903 mgr-fin ceph txt unknown /proc/38355/task/41903/exe (readlink: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41903 mgr-fin ceph NOFD /proc/38355/task/41903/fd (opendir: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41904 mgr-fin ceph cwd unknown /proc/38355/task/41904/cwd (readlink: Permission denied) 2024-03-30T00:37:28.881 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41904 mgr-fin ceph rtd unknown /proc/38355/task/41904/root (readlink: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41904 mgr-fin ceph txt unknown /proc/38355/task/41904/exe (readlink: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41904 mgr-fin ceph NOFD /proc/38355/task/41904/fd (opendir: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41905 mgr-fin ceph cwd unknown /proc/38355/task/41905/cwd (readlink: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41905 mgr-fin ceph rtd unknown /proc/38355/task/41905/root (readlink: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41905 mgr-fin ceph txt unknown /proc/38355/task/41905/exe (readlink: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41905 mgr-fin ceph NOFD /proc/38355/task/41905/fd (opendir: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41906 mgr-fin ceph cwd unknown /proc/38355/task/41906/cwd (readlink: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41906 mgr-fin ceph rtd unknown /proc/38355/task/41906/root (readlink: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41906 mgr-fin ceph txt unknown /proc/38355/task/41906/exe (readlink: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41906 mgr-fin ceph NOFD /proc/38355/task/41906/fd (opendir: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41907 mgr-fin ceph cwd unknown /proc/38355/task/41907/cwd (readlink: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41907 mgr-fin ceph rtd unknown /proc/38355/task/41907/root (readlink: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41907 mgr-fin ceph txt unknown /proc/38355/task/41907/exe (readlink: Permission denied) 2024-03-30T00:37:28.882 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41907 mgr-fin ceph NOFD /proc/38355/task/41907/fd (opendir: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41908 mgr-fin ceph cwd unknown /proc/38355/task/41908/cwd (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41908 mgr-fin ceph rtd unknown /proc/38355/task/41908/root (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41908 mgr-fin ceph txt unknown /proc/38355/task/41908/exe (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41908 mgr-fin ceph NOFD /proc/38355/task/41908/fd (opendir: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41909 mgr-fin ceph cwd unknown /proc/38355/task/41909/cwd (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41909 mgr-fin ceph rtd unknown /proc/38355/task/41909/root (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41909 mgr-fin ceph txt unknown /proc/38355/task/41909/exe (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41909 mgr-fin ceph NOFD /proc/38355/task/41909/fd (opendir: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41910 mgr-fin ceph cwd unknown /proc/38355/task/41910/cwd (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41910 mgr-fin ceph rtd unknown /proc/38355/task/41910/root (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41910 mgr-fin ceph txt unknown /proc/38355/task/41910/exe (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41910 mgr-fin ceph NOFD /proc/38355/task/41910/fd (opendir: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41911 mgr-fin ceph cwd unknown /proc/38355/task/41911/cwd (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41911 mgr-fin ceph rtd unknown /proc/38355/task/41911/root (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41911 mgr-fin ceph txt unknown /proc/38355/task/41911/exe (readlink: Permission denied) 2024-03-30T00:37:28.883 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41911 mgr-fin ceph NOFD /proc/38355/task/41911/fd (opendir: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41912 mgr-fin ceph cwd unknown /proc/38355/task/41912/cwd (readlink: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41912 mgr-fin ceph rtd unknown /proc/38355/task/41912/root (readlink: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41912 mgr-fin ceph txt unknown /proc/38355/task/41912/exe (readlink: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 41912 mgr-fin ceph NOFD /proc/38355/task/41912/fd (opendir: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42567 dashboard ceph cwd unknown /proc/38355/task/42567/cwd (readlink: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42567 dashboard ceph rtd unknown /proc/38355/task/42567/root (readlink: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42567 dashboard ceph txt unknown /proc/38355/task/42567/exe (readlink: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42567 dashboard ceph NOFD /proc/38355/task/42567/fd (opendir: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42568 dashboard ceph cwd unknown /proc/38355/task/42568/cwd (readlink: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42568 dashboard ceph rtd unknown /proc/38355/task/42568/root (readlink: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42568 dashboard ceph txt unknown /proc/38355/task/42568/exe (readlink: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42568 dashboard ceph NOFD /proc/38355/task/42568/fd (opendir: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42569 dashboard ceph cwd unknown /proc/38355/task/42569/cwd (readlink: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42569 dashboard ceph rtd unknown /proc/38355/task/42569/root (readlink: Permission denied) 2024-03-30T00:37:28.884 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42569 dashboard ceph txt unknown /proc/38355/task/42569/exe (readlink: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42569 dashboard ceph NOFD /proc/38355/task/42569/fd (opendir: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42570 dashboard ceph cwd unknown /proc/38355/task/42570/cwd (readlink: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42570 dashboard ceph rtd unknown /proc/38355/task/42570/root (readlink: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42570 dashboard ceph txt unknown /proc/38355/task/42570/exe (readlink: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42570 dashboard ceph NOFD /proc/38355/task/42570/fd (opendir: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42571 dashboard ceph cwd unknown /proc/38355/task/42571/cwd (readlink: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42571 dashboard ceph rtd unknown /proc/38355/task/42571/root (readlink: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42571 dashboard ceph txt unknown /proc/38355/task/42571/exe (readlink: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42571 dashboard ceph NOFD /proc/38355/task/42571/fd (opendir: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42572 dashboard ceph cwd unknown /proc/38355/task/42572/cwd (readlink: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42572 dashboard ceph rtd unknown /proc/38355/task/42572/root (readlink: Permission denied) 2024-03-30T00:37:28.885 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42572 dashboard ceph txt unknown /proc/38355/task/42572/exe (readlink: Permission denied) 2024-03-30T00:37:28.886 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42572 dashboard ceph NOFD /proc/38355/task/42572/fd (opendir: Permission denied) 2024-03-30T00:37:28.886 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42573 dashboard ceph cwd unknown /proc/38355/task/42573/cwd (readlink: Permission denied) 2024-03-30T00:37:28.886 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42573 dashboard ceph rtd unknown /proc/38355/task/42573/root (readlink: Permission denied) 2024-03-30T00:37:28.886 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42573 dashboard ceph txt unknown /proc/38355/task/42573/exe (readlink: Permission denied) 2024-03-30T00:37:28.886 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42573 dashboard ceph NOFD /proc/38355/task/42573/fd (opendir: Permission denied) 2024-03-30T00:37:28.886 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42574 dashboard ceph cwd unknown /proc/38355/task/42574/cwd (readlink: Permission denied) 2024-03-30T00:37:28.886 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42574 dashboard ceph rtd unknown /proc/38355/task/42574/root (readlink: Permission denied) 2024-03-30T00:37:28.886 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42574 dashboard ceph txt unknown /proc/38355/task/42574/exe (readlink: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42574 dashboard ceph NOFD /proc/38355/task/42574/fd (opendir: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42575 dashboard ceph cwd unknown /proc/38355/task/42575/cwd (readlink: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42575 dashboard ceph rtd unknown /proc/38355/task/42575/root (readlink: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42575 dashboard ceph txt unknown /proc/38355/task/42575/exe (readlink: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42575 dashboard ceph NOFD /proc/38355/task/42575/fd (opendir: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42576 dashboard ceph cwd unknown /proc/38355/task/42576/cwd (readlink: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42576 dashboard ceph rtd unknown /proc/38355/task/42576/root (readlink: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42576 dashboard ceph txt unknown /proc/38355/task/42576/exe (readlink: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42576 dashboard ceph NOFD /proc/38355/task/42576/fd (opendir: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42577 dashboard ceph cwd unknown /proc/38355/task/42577/cwd (readlink: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42577 dashboard ceph rtd unknown /proc/38355/task/42577/root (readlink: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42577 dashboard ceph txt unknown /proc/38355/task/42577/exe (readlink: Permission denied) 2024-03-30T00:37:28.887 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42577 dashboard ceph NOFD /proc/38355/task/42577/fd (opendir: Permission denied) 2024-03-30T00:37:28.888 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42578 dashboard ceph cwd unknown /proc/38355/task/42578/cwd (readlink: Permission denied) 2024-03-30T00:37:28.888 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42578 dashboard ceph rtd unknown /proc/38355/task/42578/root (readlink: Permission denied) 2024-03-30T00:37:28.888 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42578 dashboard ceph txt unknown /proc/38355/task/42578/exe (readlink: Permission denied) 2024-03-30T00:37:28.888 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 42578 dashboard ceph NOFD /proc/38355/task/42578/fd (opendir: Permission denied) 2024-03-30T00:37:28.888 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50300 cephadm ceph cwd unknown /proc/38355/task/50300/cwd (readlink: Permission denied) 2024-03-30T00:37:28.888 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50300 cephadm ceph rtd unknown /proc/38355/task/50300/root (readlink: Permission denied) 2024-03-30T00:37:28.888 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50300 cephadm ceph txt unknown /proc/38355/task/50300/exe (readlink: Permission denied) 2024-03-30T00:37:28.888 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50300 cephadm ceph NOFD /proc/38355/task/50300/fd (opendir: Permission denied) 2024-03-30T00:37:28.888 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50301 cephadm ceph cwd unknown /proc/38355/task/50301/cwd (readlink: Permission denied) 2024-03-30T00:37:28.888 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50301 cephadm ceph rtd unknown /proc/38355/task/50301/root (readlink: Permission denied) 2024-03-30T00:37:28.889 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50301 cephadm ceph txt unknown /proc/38355/task/50301/exe (readlink: Permission denied) 2024-03-30T00:37:28.889 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50301 cephadm ceph NOFD /proc/38355/task/50301/fd (opendir: Permission denied) 2024-03-30T00:37:28.889 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50302 cephadm ceph cwd unknown /proc/38355/task/50302/cwd (readlink: Permission denied) 2024-03-30T00:37:28.889 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50302 cephadm ceph rtd unknown /proc/38355/task/50302/root (readlink: Permission denied) 2024-03-30T00:37:28.889 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50302 cephadm ceph txt unknown /proc/38355/task/50302/exe (readlink: Permission denied) 2024-03-30T00:37:28.889 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50302 cephadm ceph NOFD /proc/38355/task/50302/fd (opendir: Permission denied) 2024-03-30T00:37:28.889 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50303 cephadm ceph cwd unknown /proc/38355/task/50303/cwd (readlink: Permission denied) 2024-03-30T00:37:28.889 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50303 cephadm ceph rtd unknown /proc/38355/task/50303/root (readlink: Permission denied) 2024-03-30T00:37:28.890 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50303 cephadm ceph txt unknown /proc/38355/task/50303/exe (readlink: Permission denied) 2024-03-30T00:37:28.890 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50303 cephadm ceph NOFD /proc/38355/task/50303/fd (opendir: Permission denied) 2024-03-30T00:37:28.890 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50304 cephadm ceph cwd unknown /proc/38355/task/50304/cwd (readlink: Permission denied) 2024-03-30T00:37:28.890 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50304 cephadm ceph rtd unknown /proc/38355/task/50304/root (readlink: Permission denied) 2024-03-30T00:37:28.890 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50304 cephadm ceph txt unknown /proc/38355/task/50304/exe (readlink: Permission denied) 2024-03-30T00:37:28.890 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50304 cephadm ceph NOFD /proc/38355/task/50304/fd (opendir: Permission denied) 2024-03-30T00:37:28.890 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50305 cephadm ceph cwd unknown /proc/38355/task/50305/cwd (readlink: Permission denied) 2024-03-30T00:37:28.890 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50305 cephadm ceph rtd unknown /proc/38355/task/50305/root (readlink: Permission denied) 2024-03-30T00:37:28.890 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50305 cephadm ceph txt unknown /proc/38355/task/50305/exe (readlink: Permission denied) 2024-03-30T00:37:28.891 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50305 cephadm ceph NOFD /proc/38355/task/50305/fd (opendir: Permission denied) 2024-03-30T00:37:28.891 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50306 cephadm ceph cwd unknown /proc/38355/task/50306/cwd (readlink: Permission denied) 2024-03-30T00:37:28.891 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50306 cephadm ceph rtd unknown /proc/38355/task/50306/root (readlink: Permission denied) 2024-03-30T00:37:28.891 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50306 cephadm ceph txt unknown /proc/38355/task/50306/exe (readlink: Permission denied) 2024-03-30T00:37:28.891 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50306 cephadm ceph NOFD /proc/38355/task/50306/fd (opendir: Permission denied) 2024-03-30T00:37:28.891 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50307 cephadm ceph cwd unknown /proc/38355/task/50307/cwd (readlink: Permission denied) 2024-03-30T00:37:28.891 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50307 cephadm ceph rtd unknown /proc/38355/task/50307/root (readlink: Permission denied) 2024-03-30T00:37:28.891 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50307 cephadm ceph txt unknown /proc/38355/task/50307/exe (readlink: Permission denied) 2024-03-30T00:37:28.891 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50307 cephadm ceph NOFD /proc/38355/task/50307/fd (opendir: Permission denied) 2024-03-30T00:37:28.892 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50308 cephadm ceph cwd unknown /proc/38355/task/50308/cwd (readlink: Permission denied) 2024-03-30T00:37:28.892 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50308 cephadm ceph rtd unknown /proc/38355/task/50308/root (readlink: Permission denied) 2024-03-30T00:37:28.892 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50308 cephadm ceph txt unknown /proc/38355/task/50308/exe (readlink: Permission denied) 2024-03-30T00:37:28.892 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50308 cephadm ceph NOFD /proc/38355/task/50308/fd (opendir: Permission denied) 2024-03-30T00:37:28.892 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50309 cephadm ceph cwd unknown /proc/38355/task/50309/cwd (readlink: Permission denied) 2024-03-30T00:37:28.892 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50309 cephadm ceph rtd unknown /proc/38355/task/50309/root (readlink: Permission denied) 2024-03-30T00:37:28.892 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50309 cephadm ceph txt unknown /proc/38355/task/50309/exe (readlink: Permission denied) 2024-03-30T00:37:28.892 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50309 cephadm ceph NOFD /proc/38355/task/50309/fd (opendir: Permission denied) 2024-03-30T00:37:28.892 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50745 cephadm ceph cwd unknown /proc/38355/task/50745/cwd (readlink: Permission denied) 2024-03-30T00:37:28.893 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50745 cephadm ceph rtd unknown /proc/38355/task/50745/root (readlink: Permission denied) 2024-03-30T00:37:28.893 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50745 cephadm ceph txt unknown /proc/38355/task/50745/exe (readlink: Permission denied) 2024-03-30T00:37:28.893 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50745 cephadm ceph NOFD /proc/38355/task/50745/fd (opendir: Permission denied) 2024-03-30T00:37:28.893 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50746 cephadm ceph cwd unknown /proc/38355/task/50746/cwd (readlink: Permission denied) 2024-03-30T00:37:28.893 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50746 cephadm ceph rtd unknown /proc/38355/task/50746/root (readlink: Permission denied) 2024-03-30T00:37:28.893 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50746 cephadm ceph txt unknown /proc/38355/task/50746/exe (readlink: Permission denied) 2024-03-30T00:37:28.893 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50746 cephadm ceph NOFD /proc/38355/task/50746/fd (opendir: Permission denied) 2024-03-30T00:37:28.893 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50747 cephadm ceph cwd unknown /proc/38355/task/50747/cwd (readlink: Permission denied) 2024-03-30T00:37:28.893 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50747 cephadm ceph rtd unknown /proc/38355/task/50747/root (readlink: Permission denied) 2024-03-30T00:37:28.893 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50747 cephadm ceph txt unknown /proc/38355/task/50747/exe (readlink: Permission denied) 2024-03-30T00:37:28.894 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50747 cephadm ceph NOFD /proc/38355/task/50747/fd (opendir: Permission denied) 2024-03-30T00:37:28.894 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50748 cephadm ceph cwd unknown /proc/38355/task/50748/cwd (readlink: Permission denied) 2024-03-30T00:37:28.894 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50748 cephadm ceph rtd unknown /proc/38355/task/50748/root (readlink: Permission denied) 2024-03-30T00:37:28.894 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50748 cephadm ceph txt unknown /proc/38355/task/50748/exe (readlink: Permission denied) 2024-03-30T00:37:28.894 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50748 cephadm ceph NOFD /proc/38355/task/50748/fd (opendir: Permission denied) 2024-03-30T00:37:28.894 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50749 cephadm ceph cwd unknown /proc/38355/task/50749/cwd (readlink: Permission denied) 2024-03-30T00:37:28.894 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50749 cephadm ceph rtd unknown /proc/38355/task/50749/root (readlink: Permission denied) 2024-03-30T00:37:28.894 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50749 cephadm ceph txt unknown /proc/38355/task/50749/exe (readlink: Permission denied) 2024-03-30T00:37:28.895 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50749 cephadm ceph NOFD /proc/38355/task/50749/fd (opendir: Permission denied) 2024-03-30T00:37:28.895 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50750 cephadm ceph cwd unknown /proc/38355/task/50750/cwd (readlink: Permission denied) 2024-03-30T00:37:28.895 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50750 cephadm ceph rtd unknown /proc/38355/task/50750/root (readlink: Permission denied) 2024-03-30T00:37:28.895 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50750 cephadm ceph txt unknown /proc/38355/task/50750/exe (readlink: Permission denied) 2024-03-30T00:37:28.895 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50750 cephadm ceph NOFD /proc/38355/task/50750/fd (opendir: Permission denied) 2024-03-30T00:37:28.895 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50751 cephadm ceph cwd unknown /proc/38355/task/50751/cwd (readlink: Permission denied) 2024-03-30T00:37:28.895 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50751 cephadm ceph rtd unknown /proc/38355/task/50751/root (readlink: Permission denied) 2024-03-30T00:37:28.895 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50751 cephadm ceph txt unknown /proc/38355/task/50751/exe (readlink: Permission denied) 2024-03-30T00:37:28.896 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50751 cephadm ceph NOFD /proc/38355/task/50751/fd (opendir: Permission denied) 2024-03-30T00:37:28.896 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50752 cephadm ceph cwd unknown /proc/38355/task/50752/cwd (readlink: Permission denied) 2024-03-30T00:37:28.896 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50752 cephadm ceph rtd unknown /proc/38355/task/50752/root (readlink: Permission denied) 2024-03-30T00:37:28.896 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50752 cephadm ceph txt unknown /proc/38355/task/50752/exe (readlink: Permission denied) 2024-03-30T00:37:28.896 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50752 cephadm ceph NOFD /proc/38355/task/50752/fd (opendir: Permission denied) 2024-03-30T00:37:28.896 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50753 cephadm ceph cwd unknown /proc/38355/task/50753/cwd (readlink: Permission denied) 2024-03-30T00:37:28.896 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50753 cephadm ceph rtd unknown /proc/38355/task/50753/root (readlink: Permission denied) 2024-03-30T00:37:28.896 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50753 cephadm ceph txt unknown /proc/38355/task/50753/exe (readlink: Permission denied) 2024-03-30T00:37:28.896 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50753 cephadm ceph NOFD /proc/38355/task/50753/fd (opendir: Permission denied) 2024-03-30T00:37:28.897 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50754 cephadm ceph cwd unknown /proc/38355/task/50754/cwd (readlink: Permission denied) 2024-03-30T00:37:28.897 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50754 cephadm ceph rtd unknown /proc/38355/task/50754/root (readlink: Permission denied) 2024-03-30T00:37:28.897 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50754 cephadm ceph txt unknown /proc/38355/task/50754/exe (readlink: Permission denied) 2024-03-30T00:37:28.897 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 50754 cephadm ceph NOFD /proc/38355/task/50754/fd (opendir: Permission denied) 2024-03-30T00:37:28.897 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61872 devicehea ceph cwd unknown /proc/38355/task/61872/cwd (readlink: Permission denied) 2024-03-30T00:37:28.897 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61872 devicehea ceph rtd unknown /proc/38355/task/61872/root (readlink: Permission denied) 2024-03-30T00:37:28.897 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61872 devicehea ceph txt unknown /proc/38355/task/61872/exe (readlink: Permission denied) 2024-03-30T00:37:28.897 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61872 devicehea ceph NOFD /proc/38355/task/61872/fd (opendir: Permission denied) 2024-03-30T00:37:28.897 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61893 io_contex ceph cwd unknown /proc/38355/task/61893/cwd (readlink: Permission denied) 2024-03-30T00:37:28.897 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61893 io_contex ceph rtd unknown /proc/38355/task/61893/root (readlink: Permission denied) 2024-03-30T00:37:28.897 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61893 io_contex ceph txt unknown /proc/38355/task/61893/exe (readlink: Permission denied) 2024-03-30T00:37:28.898 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61893 io_contex ceph NOFD /proc/38355/task/61893/fd (opendir: Permission denied) 2024-03-30T00:37:28.898 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61894 io_contex ceph cwd unknown /proc/38355/task/61894/cwd (readlink: Permission denied) 2024-03-30T00:37:28.898 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61894 io_contex ceph rtd unknown /proc/38355/task/61894/root (readlink: Permission denied) 2024-03-30T00:37:28.898 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61894 io_contex ceph txt unknown /proc/38355/task/61894/exe (readlink: Permission denied) 2024-03-30T00:37:28.898 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61894 io_contex ceph NOFD /proc/38355/task/61894/fd (opendir: Permission denied) 2024-03-30T00:37:28.898 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61895 ceph_time ceph cwd unknown /proc/38355/task/61895/cwd (readlink: Permission denied) 2024-03-30T00:37:28.898 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61895 ceph_time ceph rtd unknown /proc/38355/task/61895/root (readlink: Permission denied) 2024-03-30T00:37:28.898 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61895 ceph_time ceph txt unknown /proc/38355/task/61895/exe (readlink: Permission denied) 2024-03-30T00:37:28.898 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61895 ceph_time ceph NOFD /proc/38355/task/61895/fd (opendir: Permission denied) 2024-03-30T00:37:28.898 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61896 ms_dispat ceph cwd unknown /proc/38355/task/61896/cwd (readlink: Permission denied) 2024-03-30T00:37:28.898 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61896 ms_dispat ceph rtd unknown /proc/38355/task/61896/root (readlink: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61896 ms_dispat ceph txt unknown /proc/38355/task/61896/exe (readlink: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61896 ms_dispat ceph NOFD /proc/38355/task/61896/fd (opendir: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61897 ms_local ceph cwd unknown /proc/38355/task/61897/cwd (readlink: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61897 ms_local ceph rtd unknown /proc/38355/task/61897/root (readlink: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61897 ms_local ceph txt unknown /proc/38355/task/61897/exe (readlink: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61897 ms_local ceph NOFD /proc/38355/task/61897/fd (opendir: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61898 safe_time ceph cwd unknown /proc/38355/task/61898/cwd (readlink: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61898 safe_time ceph rtd unknown /proc/38355/task/61898/root (readlink: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61898 safe_time ceph txt unknown /proc/38355/task/61898/exe (readlink: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61898 safe_time ceph NOFD /proc/38355/task/61898/fd (opendir: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61899 safe_time ceph cwd unknown /proc/38355/task/61899/cwd (readlink: Permission denied) 2024-03-30T00:37:28.899 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61899 safe_time ceph rtd unknown /proc/38355/task/61899/root (readlink: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61899 safe_time ceph txt unknown /proc/38355/task/61899/exe (readlink: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mgr 38355 61899 safe_time ceph NOFD /proc/38355/task/61899/fd (opendir: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 38478 root cwd unknown /proc/38478/cwd (readlink: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 38478 root rtd unknown /proc/38478/root (readlink: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 38478 root txt unknown /proc/38478/exe (readlink: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 38478 root NOFD /proc/38478/fd (opendir: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 39302 root cwd unknown /proc/39302/cwd (readlink: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 39302 root rtd unknown /proc/39302/root (readlink: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 39302 root txt unknown /proc/39302/exe (readlink: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 39302 root NOFD /proc/39302/fd (opendir: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 39758 root cwd unknown /proc/39758/cwd (readlink: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 39758 root rtd unknown /proc/39758/root (readlink: Permission denied) 2024-03-30T00:37:28.900 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 39758 root txt unknown /proc/39758/exe (readlink: Permission denied) 2024-03-30T00:37:28.901 INFO:tasks.workunit.client.0.smithi099.stdout:systemd 39758 root NOFD /proc/39758/fd (opendir: Permission denied) 2024-03-30T00:37:28.901 INFO:tasks.workunit.client.0.smithi099.stdout:(sd-pam) 39761 root cwd unknown /proc/39761/cwd (readlink: Permission denied) 2024-03-30T00:37:28.901 INFO:tasks.workunit.client.0.smithi099.stdout:(sd-pam) 39761 root rtd unknown /proc/39761/root (readlink: Permission denied) 2024-03-30T00:37:28.901 INFO:tasks.workunit.client.0.smithi099.stdout:(sd-pam) 39761 root txt unknown /proc/39761/exe (readlink: Permission denied) 2024-03-30T00:37:28.901 INFO:tasks.workunit.client.0.smithi099.stdout:(sd-pam) 39761 root NOFD /proc/39761/fd (opendir: Permission denied) 2024-03-30T00:37:28.901 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 40877 root cwd unknown /proc/40877/cwd (readlink: Permission denied) 2024-03-30T00:37:28.901 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 40877 root rtd unknown /proc/40877/root (readlink: Permission denied) 2024-03-30T00:37:28.901 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 40877 root txt unknown /proc/40877/exe (readlink: Permission denied) 2024-03-30T00:37:28.901 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 40877 root NOFD /proc/40877/fd (opendir: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 41298 root cwd unknown /proc/41298/cwd (readlink: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 41298 root rtd unknown /proc/41298/root (readlink: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 41298 root txt unknown /proc/41298/exe (readlink: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 41298 root NOFD /proc/41298/fd (opendir: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 41319 root cwd unknown /proc/41319/cwd (readlink: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 41319 root rtd unknown /proc/41319/root (readlink: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 41319 root txt unknown /proc/41319/exe (readlink: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 41319 root NOFD /proc/41319/fd (opendir: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 41657 root cwd unknown /proc/41657/cwd (readlink: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 41657 root rtd unknown /proc/41657/root (readlink: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 41657 root txt unknown /proc/41657/exe (readlink: Permission denied) 2024-03-30T00:37:28.902 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 41657 root NOFD /proc/41657/fd (opendir: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 41665 root cwd unknown /proc/41665/cwd (readlink: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 41665 root rtd unknown /proc/41665/root (readlink: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 41665 root txt unknown /proc/41665/exe (readlink: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:sshd 41665 root NOFD /proc/41665/fd (opendir: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 55748 root cwd unknown /proc/55748/cwd (readlink: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 55748 root rtd unknown /proc/55748/root (readlink: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 55748 root txt unknown /proc/55748/exe (readlink: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 55748 root NOFD /proc/55748/fd (opendir: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 55759 root cwd unknown /proc/55759/cwd (readlink: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 55759 root rtd unknown /proc/55759/root (readlink: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 55759 root txt unknown /proc/55759/exe (readlink: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 55759 root NOFD /proc/55759/fd (opendir: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 ceph cwd unknown /proc/55772/cwd (readlink: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 ceph rtd unknown /proc/55772/root (readlink: Permission denied) 2024-03-30T00:37:28.903 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 ceph txt unknown /proc/55772/exe (readlink: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 ceph NOFD /proc/55772/fd (opendir: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55773 log ceph cwd unknown /proc/55772/task/55773/cwd (readlink: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55773 log ceph rtd unknown /proc/55772/task/55773/root (readlink: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55773 log ceph txt unknown /proc/55772/task/55773/exe (readlink: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55773 log ceph NOFD /proc/55772/task/55773/fd (opendir: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55775 msgr-work ceph cwd unknown /proc/55772/task/55775/cwd (readlink: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55775 msgr-work ceph rtd unknown /proc/55772/task/55775/root (readlink: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55775 msgr-work ceph txt unknown /proc/55772/task/55775/exe (readlink: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55775 msgr-work ceph NOFD /proc/55772/task/55775/fd (opendir: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55776 msgr-work ceph cwd unknown /proc/55772/task/55776/cwd (readlink: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55776 msgr-work ceph rtd unknown /proc/55772/task/55776/root (readlink: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55776 msgr-work ceph txt unknown /proc/55772/task/55776/exe (readlink: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55776 msgr-work ceph NOFD /proc/55772/task/55776/fd (opendir: Permission denied) 2024-03-30T00:37:28.904 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55777 msgr-work ceph cwd unknown /proc/55772/task/55777/cwd (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55777 msgr-work ceph rtd unknown /proc/55772/task/55777/root (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55777 msgr-work ceph txt unknown /proc/55772/task/55777/exe (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55777 msgr-work ceph NOFD /proc/55772/task/55777/fd (opendir: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55783 service ceph cwd unknown /proc/55772/task/55783/cwd (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55783 service ceph rtd unknown /proc/55772/task/55783/root (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55783 service ceph txt unknown /proc/55772/task/55783/exe (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55783 service ceph NOFD /proc/55772/task/55783/fd (opendir: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55784 admin_soc ceph cwd unknown /proc/55772/task/55784/cwd (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55784 admin_soc ceph rtd unknown /proc/55772/task/55784/root (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55784 admin_soc ceph txt unknown /proc/55772/task/55784/exe (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55784 admin_soc ceph NOFD /proc/55772/task/55784/fd (opendir: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55999 signal_ha ceph cwd unknown /proc/55772/task/55999/cwd (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55999 signal_ha ceph rtd unknown /proc/55772/task/55999/root (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55999 signal_ha ceph txt unknown /proc/55772/task/55999/exe (readlink: Permission denied) 2024-03-30T00:37:28.905 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 55999 signal_ha ceph NOFD /proc/55772/task/55999/fd (opendir: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56000 io_contex ceph cwd unknown /proc/55772/task/56000/cwd (readlink: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56000 io_contex ceph rtd unknown /proc/55772/task/56000/root (readlink: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56000 io_contex ceph txt unknown /proc/55772/task/56000/exe (readlink: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56000 io_contex ceph NOFD /proc/55772/task/56000/fd (opendir: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56001 io_contex ceph cwd unknown /proc/55772/task/56001/cwd (readlink: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56001 io_contex ceph rtd unknown /proc/55772/task/56001/root (readlink: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56001 io_contex ceph txt unknown /proc/55772/task/56001/exe (readlink: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56001 io_contex ceph NOFD /proc/55772/task/56001/fd (opendir: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56040 OpHistory ceph cwd unknown /proc/55772/task/56040/cwd (readlink: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56040 OpHistory ceph rtd unknown /proc/55772/task/56040/root (readlink: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56040 OpHistory ceph txt unknown /proc/55772/task/56040/exe (readlink: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56040 OpHistory ceph NOFD /proc/55772/task/56040/fd (opendir: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56041 ceph_time ceph cwd unknown /proc/55772/task/56041/cwd (readlink: Permission denied) 2024-03-30T00:37:28.906 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56041 ceph_time ceph rtd unknown /proc/55772/task/56041/root (readlink: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56041 ceph_time ceph txt unknown /proc/55772/task/56041/exe (readlink: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56041 ceph_time ceph NOFD /proc/55772/task/56041/fd (opendir: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56113 ceph-osd ceph cwd unknown /proc/55772/task/56113/cwd (readlink: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56113 ceph-osd ceph rtd unknown /proc/55772/task/56113/root (readlink: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56113 ceph-osd ceph txt unknown /proc/55772/task/56113/exe (readlink: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56113 ceph-osd ceph NOFD /proc/55772/task/56113/fd (opendir: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56165 ceph-osd ceph cwd unknown /proc/55772/task/56165/cwd (readlink: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56165 ceph-osd ceph rtd unknown /proc/55772/task/56165/root (readlink: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56165 ceph-osd ceph txt unknown /proc/55772/task/56165/exe (readlink: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56165 ceph-osd ceph NOFD /proc/55772/task/56165/fd (opendir: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56230 ceph-osd ceph cwd unknown /proc/55772/task/56230/cwd (readlink: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56230 ceph-osd ceph rtd unknown /proc/55772/task/56230/root (readlink: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56230 ceph-osd ceph txt unknown /proc/55772/task/56230/exe (readlink: Permission denied) 2024-03-30T00:37:28.907 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56230 ceph-osd ceph NOFD /proc/55772/task/56230/fd (opendir: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56288 ceph-osd ceph cwd unknown /proc/55772/task/56288/cwd (readlink: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56288 ceph-osd ceph rtd unknown /proc/55772/task/56288/root (readlink: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56288 ceph-osd ceph txt unknown /proc/55772/task/56288/exe (readlink: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56288 ceph-osd ceph NOFD /proc/55772/task/56288/fd (opendir: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56311 ceph-osd ceph cwd unknown /proc/55772/task/56311/cwd (readlink: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56311 ceph-osd ceph rtd unknown /proc/55772/task/56311/root (readlink: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56311 ceph-osd ceph txt unknown /proc/55772/task/56311/exe (readlink: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56311 ceph-osd ceph NOFD /proc/55772/task/56311/fd (opendir: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56346 ceph-osd ceph cwd unknown /proc/55772/task/56346/cwd (readlink: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56346 ceph-osd ceph rtd unknown /proc/55772/task/56346/root (readlink: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56346 ceph-osd ceph txt unknown /proc/55772/task/56346/exe (readlink: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56346 ceph-osd ceph NOFD /proc/55772/task/56346/fd (opendir: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56390 ceph-osd ceph cwd unknown /proc/55772/task/56390/cwd (readlink: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56390 ceph-osd ceph rtd unknown /proc/55772/task/56390/root (readlink: Permission denied) 2024-03-30T00:37:28.908 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56390 ceph-osd ceph txt unknown /proc/55772/task/56390/exe (readlink: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56390 ceph-osd ceph NOFD /proc/55772/task/56390/fd (opendir: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56459 ceph-osd ceph cwd unknown /proc/55772/task/56459/cwd (readlink: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56459 ceph-osd ceph rtd unknown /proc/55772/task/56459/root (readlink: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56459 ceph-osd ceph txt unknown /proc/55772/task/56459/exe (readlink: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56459 ceph-osd ceph NOFD /proc/55772/task/56459/fd (opendir: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56461 ceph-osd ceph cwd unknown /proc/55772/task/56461/cwd (readlink: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56461 ceph-osd ceph rtd unknown /proc/55772/task/56461/root (readlink: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56461 ceph-osd ceph txt unknown /proc/55772/task/56461/exe (readlink: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56461 ceph-osd ceph NOFD /proc/55772/task/56461/fd (opendir: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56462 safe_time ceph cwd unknown /proc/55772/task/56462/cwd (readlink: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56462 safe_time ceph rtd unknown /proc/55772/task/56462/root (readlink: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56462 safe_time ceph txt unknown /proc/55772/task/56462/exe (readlink: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56462 safe_time ceph NOFD /proc/55772/task/56462/fd (opendir: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56463 safe_time ceph cwd unknown /proc/55772/task/56463/cwd (readlink: Permission denied) 2024-03-30T00:37:28.909 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56463 safe_time ceph rtd unknown /proc/55772/task/56463/root (readlink: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56463 safe_time ceph txt unknown /proc/55772/task/56463/exe (readlink: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56463 safe_time ceph NOFD /proc/55772/task/56463/fd (opendir: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56464 safe_time ceph cwd unknown /proc/55772/task/56464/cwd (readlink: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56464 safe_time ceph rtd unknown /proc/55772/task/56464/root (readlink: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56464 safe_time ceph txt unknown /proc/55772/task/56464/exe (readlink: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56464 safe_time ceph NOFD /proc/55772/task/56464/fd (opendir: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56465 safe_time ceph cwd unknown /proc/55772/task/56465/cwd (readlink: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56465 safe_time ceph rtd unknown /proc/55772/task/56465/root (readlink: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56465 safe_time ceph txt unknown /proc/55772/task/56465/exe (readlink: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56465 safe_time ceph NOFD /proc/55772/task/56465/fd (opendir: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56466 fn_anonym ceph cwd unknown /proc/55772/task/56466/cwd (readlink: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56466 fn_anonym ceph rtd unknown /proc/55772/task/56466/root (readlink: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56466 fn_anonym ceph txt unknown /proc/55772/task/56466/exe (readlink: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56466 fn_anonym ceph NOFD /proc/55772/task/56466/fd (opendir: Permission denied) 2024-03-30T00:37:28.910 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56472 rocksdb:l ceph cwd unknown /proc/55772/task/56472/cwd (readlink: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56472 rocksdb:l ceph rtd unknown /proc/55772/task/56472/root (readlink: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56472 rocksdb:l ceph txt unknown /proc/55772/task/56472/exe (readlink: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56472 rocksdb:l ceph NOFD /proc/55772/task/56472/fd (opendir: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56473 rocksdb:l ceph cwd unknown /proc/55772/task/56473/cwd (readlink: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56473 rocksdb:l ceph rtd unknown /proc/55772/task/56473/root (readlink: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56473 rocksdb:l ceph txt unknown /proc/55772/task/56473/exe (readlink: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56473 rocksdb:l ceph NOFD /proc/55772/task/56473/fd (opendir: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56474 rocksdb:l ceph cwd unknown /proc/55772/task/56474/cwd (readlink: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56474 rocksdb:l ceph rtd unknown /proc/55772/task/56474/root (readlink: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56474 rocksdb:l ceph txt unknown /proc/55772/task/56474/exe (readlink: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56474 rocksdb:l ceph NOFD /proc/55772/task/56474/fd (opendir: Permission denied) 2024-03-30T00:37:28.911 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56475 rocksdb:h ceph cwd unknown /proc/55772/task/56475/cwd (readlink: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56475 rocksdb:h ceph rtd unknown /proc/55772/task/56475/root (readlink: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56475 rocksdb:h ceph txt unknown /proc/55772/task/56475/exe (readlink: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56475 rocksdb:h ceph NOFD /proc/55772/task/56475/fd (opendir: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56958 bstore_ai ceph cwd unknown /proc/55772/task/56958/cwd (readlink: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56958 bstore_ai ceph rtd unknown /proc/55772/task/56958/root (readlink: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56958 bstore_ai ceph txt unknown /proc/55772/task/56958/exe (readlink: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56958 bstore_ai ceph NOFD /proc/55772/task/56958/fd (opendir: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56959 bstore_di ceph cwd unknown /proc/55772/task/56959/cwd (readlink: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56959 bstore_di ceph rtd unknown /proc/55772/task/56959/root (readlink: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56959 bstore_di ceph txt unknown /proc/55772/task/56959/exe (readlink: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 56959 bstore_di ceph NOFD /proc/55772/task/56959/fd (opendir: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57170 bstore_ai ceph cwd unknown /proc/55772/task/57170/cwd (readlink: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57170 bstore_ai ceph rtd unknown /proc/55772/task/57170/root (readlink: Permission denied) 2024-03-30T00:37:28.912 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57170 bstore_ai ceph txt unknown /proc/55772/task/57170/exe (readlink: Permission denied) 2024-03-30T00:37:28.913 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57170 bstore_ai ceph NOFD /proc/55772/task/57170/fd (opendir: Permission denied) 2024-03-30T00:37:28.913 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57171 bstore_di ceph cwd unknown /proc/55772/task/57171/cwd (readlink: Permission denied) 2024-03-30T00:37:28.913 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57171 bstore_di ceph rtd unknown /proc/55772/task/57171/root (readlink: Permission denied) 2024-03-30T00:37:28.913 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57171 bstore_di ceph txt unknown /proc/55772/task/57171/exe (readlink: Permission denied) 2024-03-30T00:37:28.913 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57171 bstore_di ceph NOFD /proc/55772/task/57171/fd (opendir: Permission denied) 2024-03-30T00:37:28.913 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57367 ceph-osd ceph cwd unknown /proc/55772/task/57367/cwd (readlink: Permission denied) 2024-03-30T00:37:28.913 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57367 ceph-osd ceph rtd unknown /proc/55772/task/57367/root (readlink: Permission denied) 2024-03-30T00:37:28.913 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57367 ceph-osd ceph txt unknown /proc/55772/task/57367/exe (readlink: Permission denied) 2024-03-30T00:37:28.913 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57367 ceph-osd ceph NOFD /proc/55772/task/57367/fd (opendir: Permission denied) 2024-03-30T00:37:28.913 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57368 cfin ceph cwd unknown /proc/55772/task/57368/cwd (readlink: Permission denied) 2024-03-30T00:37:28.913 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57368 cfin ceph rtd unknown /proc/55772/task/57368/root (readlink: Permission denied) 2024-03-30T00:37:28.914 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57368 cfin ceph txt unknown /proc/55772/task/57368/exe (readlink: Permission denied) 2024-03-30T00:37:28.914 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57368 cfin ceph NOFD /proc/55772/task/57368/fd (opendir: Permission denied) 2024-03-30T00:37:28.914 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57369 bstore_kv ceph cwd unknown /proc/55772/task/57369/cwd (readlink: Permission denied) 2024-03-30T00:37:28.914 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57369 bstore_kv ceph rtd unknown /proc/55772/task/57369/root (readlink: Permission denied) 2024-03-30T00:37:28.914 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57369 bstore_kv ceph txt unknown /proc/55772/task/57369/exe (readlink: Permission denied) 2024-03-30T00:37:28.914 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57369 bstore_kv ceph NOFD /proc/55772/task/57369/fd (opendir: Permission denied) 2024-03-30T00:37:28.914 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57370 bstore_kv ceph cwd unknown /proc/55772/task/57370/cwd (readlink: Permission denied) 2024-03-30T00:37:28.914 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57370 bstore_kv ceph rtd unknown /proc/55772/task/57370/root (readlink: Permission denied) 2024-03-30T00:37:28.914 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57370 bstore_kv ceph txt unknown /proc/55772/task/57370/exe (readlink: Permission denied) 2024-03-30T00:37:28.914 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57370 bstore_kv ceph NOFD /proc/55772/task/57370/fd (opendir: Permission denied) 2024-03-30T00:37:28.914 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57371 bstore_me ceph cwd unknown /proc/55772/task/57371/cwd (readlink: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57371 bstore_me ceph rtd unknown /proc/55772/task/57371/root (readlink: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57371 bstore_me ceph txt unknown /proc/55772/task/57371/exe (readlink: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57371 bstore_me ceph NOFD /proc/55772/task/57371/fd (opendir: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57373 ms_dispat ceph cwd unknown /proc/55772/task/57373/cwd (readlink: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57373 ms_dispat ceph rtd unknown /proc/55772/task/57373/root (readlink: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57373 ms_dispat ceph txt unknown /proc/55772/task/57373/exe (readlink: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57373 ms_dispat ceph NOFD /proc/55772/task/57373/fd (opendir: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57374 ms_local ceph cwd unknown /proc/55772/task/57374/cwd (readlink: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57374 ms_local ceph rtd unknown /proc/55772/task/57374/root (readlink: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57374 ms_local ceph txt unknown /proc/55772/task/57374/exe (readlink: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57374 ms_local ceph NOFD /proc/55772/task/57374/fd (opendir: Permission denied) 2024-03-30T00:37:28.915 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57375 safe_time ceph cwd unknown /proc/55772/task/57375/cwd (readlink: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57375 safe_time ceph rtd unknown /proc/55772/task/57375/root (readlink: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57375 safe_time ceph txt unknown /proc/55772/task/57375/exe (readlink: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57375 safe_time ceph NOFD /proc/55772/task/57375/fd (opendir: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57376 safe_time ceph cwd unknown /proc/55772/task/57376/cwd (readlink: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57376 safe_time ceph rtd unknown /proc/55772/task/57376/root (readlink: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57376 safe_time ceph txt unknown /proc/55772/task/57376/exe (readlink: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57376 safe_time ceph NOFD /proc/55772/task/57376/fd (opendir: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57378 ms_dispat ceph cwd unknown /proc/55772/task/57378/cwd (readlink: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57378 ms_dispat ceph rtd unknown /proc/55772/task/57378/root (readlink: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57378 ms_dispat ceph txt unknown /proc/55772/task/57378/exe (readlink: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57378 ms_dispat ceph NOFD /proc/55772/task/57378/fd (opendir: Permission denied) 2024-03-30T00:37:28.916 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57379 ms_local ceph cwd unknown /proc/55772/task/57379/cwd (readlink: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57379 ms_local ceph rtd unknown /proc/55772/task/57379/root (readlink: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57379 ms_local ceph txt unknown /proc/55772/task/57379/exe (readlink: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57379 ms_local ceph NOFD /proc/55772/task/57379/fd (opendir: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57380 ms_dispat ceph cwd unknown /proc/55772/task/57380/cwd (readlink: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57380 ms_dispat ceph rtd unknown /proc/55772/task/57380/root (readlink: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57380 ms_dispat ceph txt unknown /proc/55772/task/57380/exe (readlink: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57380 ms_dispat ceph NOFD /proc/55772/task/57380/fd (opendir: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57381 ms_local ceph cwd unknown /proc/55772/task/57381/cwd (readlink: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57381 ms_local ceph rtd unknown /proc/55772/task/57381/root (readlink: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57381 ms_local ceph txt unknown /proc/55772/task/57381/exe (readlink: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57381 ms_local ceph NOFD /proc/55772/task/57381/fd (opendir: Permission denied) 2024-03-30T00:37:28.917 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57382 ms_dispat ceph cwd unknown /proc/55772/task/57382/cwd (readlink: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57382 ms_dispat ceph rtd unknown /proc/55772/task/57382/root (readlink: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57382 ms_dispat ceph txt unknown /proc/55772/task/57382/exe (readlink: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57382 ms_dispat ceph NOFD /proc/55772/task/57382/fd (opendir: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57383 ms_local ceph cwd unknown /proc/55772/task/57383/cwd (readlink: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57383 ms_local ceph rtd unknown /proc/55772/task/57383/root (readlink: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57383 ms_local ceph txt unknown /proc/55772/task/57383/exe (readlink: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57383 ms_local ceph NOFD /proc/55772/task/57383/fd (opendir: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57384 ms_dispat ceph cwd unknown /proc/55772/task/57384/cwd (readlink: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57384 ms_dispat ceph rtd unknown /proc/55772/task/57384/root (readlink: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57384 ms_dispat ceph txt unknown /proc/55772/task/57384/exe (readlink: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57384 ms_dispat ceph NOFD /proc/55772/task/57384/fd (opendir: Permission denied) 2024-03-30T00:37:28.918 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57385 ms_local ceph cwd unknown /proc/55772/task/57385/cwd (readlink: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57385 ms_local ceph rtd unknown /proc/55772/task/57385/root (readlink: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57385 ms_local ceph txt unknown /proc/55772/task/57385/exe (readlink: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57385 ms_local ceph NOFD /proc/55772/task/57385/fd (opendir: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57386 ms_dispat ceph cwd unknown /proc/55772/task/57386/cwd (readlink: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57386 ms_dispat ceph rtd unknown /proc/55772/task/57386/root (readlink: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57386 ms_dispat ceph txt unknown /proc/55772/task/57386/exe (readlink: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57386 ms_dispat ceph NOFD /proc/55772/task/57386/fd (opendir: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57387 ms_local ceph cwd unknown /proc/55772/task/57387/cwd (readlink: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57387 ms_local ceph rtd unknown /proc/55772/task/57387/root (readlink: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57387 ms_local ceph txt unknown /proc/55772/task/57387/exe (readlink: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57387 ms_local ceph NOFD /proc/55772/task/57387/fd (opendir: Permission denied) 2024-03-30T00:37:28.919 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57388 ms_dispat ceph cwd unknown /proc/55772/task/57388/cwd (readlink: Permission denied) 2024-03-30T00:37:28.920 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57388 ms_dispat ceph rtd unknown /proc/55772/task/57388/root (readlink: Permission denied) 2024-03-30T00:37:28.920 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57388 ms_dispat ceph txt unknown /proc/55772/task/57388/exe (readlink: Permission denied) 2024-03-30T00:37:28.920 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57388 ms_dispat ceph NOFD /proc/55772/task/57388/fd (opendir: Permission denied) 2024-03-30T00:37:28.920 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57389 ms_local ceph cwd unknown /proc/55772/task/57389/cwd (readlink: Permission denied) 2024-03-30T00:37:28.920 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57389 ms_local ceph rtd unknown /proc/55772/task/57389/root (readlink: Permission denied) 2024-03-30T00:37:28.920 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57389 ms_local ceph txt unknown /proc/55772/task/57389/exe (readlink: Permission denied) 2024-03-30T00:37:28.920 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57389 ms_local ceph NOFD /proc/55772/task/57389/fd (opendir: Permission denied) 2024-03-30T00:37:28.920 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57390 fn_anonym ceph cwd unknown /proc/55772/task/57390/cwd (readlink: Permission denied) 2024-03-30T00:37:28.920 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57390 fn_anonym ceph rtd unknown /proc/55772/task/57390/root (readlink: Permission denied) 2024-03-30T00:37:28.920 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57390 fn_anonym ceph txt unknown /proc/55772/task/57390/exe (readlink: Permission denied) 2024-03-30T00:37:28.920 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57390 fn_anonym ceph NOFD /proc/55772/task/57390/fd (opendir: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57391 finisher ceph cwd unknown /proc/55772/task/57391/cwd (readlink: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57391 finisher ceph rtd unknown /proc/55772/task/57391/root (readlink: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57391 finisher ceph txt unknown /proc/55772/task/57391/exe (readlink: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57391 finisher ceph NOFD /proc/55772/task/57391/fd (opendir: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57392 safe_time ceph cwd unknown /proc/55772/task/57392/cwd (readlink: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57392 safe_time ceph rtd unknown /proc/55772/task/57392/root (readlink: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57392 safe_time ceph txt unknown /proc/55772/task/57392/exe (readlink: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57392 safe_time ceph NOFD /proc/55772/task/57392/fd (opendir: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57393 safe_time ceph cwd unknown /proc/55772/task/57393/cwd (readlink: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57393 safe_time ceph rtd unknown /proc/55772/task/57393/root (readlink: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57393 safe_time ceph txt unknown /proc/55772/task/57393/exe (readlink: Permission denied) 2024-03-30T00:37:28.921 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57393 safe_time ceph NOFD /proc/55772/task/57393/fd (opendir: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57394 ceph-osd ceph cwd unknown /proc/55772/task/57394/cwd (readlink: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57394 ceph-osd ceph rtd unknown /proc/55772/task/57394/root (readlink: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57394 ceph-osd ceph txt unknown /proc/55772/task/57394/exe (readlink: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57394 ceph-osd ceph NOFD /proc/55772/task/57394/fd (opendir: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57395 osd_srv_a ceph cwd unknown /proc/55772/task/57395/cwd (readlink: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57395 osd_srv_a ceph rtd unknown /proc/55772/task/57395/root (readlink: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57395 osd_srv_a ceph txt unknown /proc/55772/task/57395/exe (readlink: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57395 osd_srv_a ceph NOFD /proc/55772/task/57395/fd (opendir: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57396 tp_osd_tp ceph cwd unknown /proc/55772/task/57396/cwd (readlink: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57396 tp_osd_tp ceph rtd unknown /proc/55772/task/57396/root (readlink: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57396 tp_osd_tp ceph txt unknown /proc/55772/task/57396/exe (readlink: Permission denied) 2024-03-30T00:37:28.922 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57396 tp_osd_tp ceph NOFD /proc/55772/task/57396/fd (opendir: Permission denied) 2024-03-30T00:37:28.923 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57397 tp_osd_tp ceph cwd unknown /proc/55772/task/57397/cwd (readlink: Permission denied) 2024-03-30T00:37:28.923 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57397 tp_osd_tp ceph rtd unknown /proc/55772/task/57397/root (readlink: Permission denied) 2024-03-30T00:37:28.923 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57397 tp_osd_tp ceph txt unknown /proc/55772/task/57397/exe (readlink: Permission denied) 2024-03-30T00:37:28.923 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57397 tp_osd_tp ceph NOFD /proc/55772/task/57397/fd (opendir: Permission denied) 2024-03-30T00:37:28.923 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57398 tp_osd_tp ceph cwd unknown /proc/55772/task/57398/cwd (readlink: Permission denied) 2024-03-30T00:37:28.923 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57398 tp_osd_tp ceph rtd unknown /proc/55772/task/57398/root (readlink: Permission denied) 2024-03-30T00:37:28.923 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57398 tp_osd_tp ceph txt unknown /proc/55772/task/57398/exe (readlink: Permission denied) 2024-03-30T00:37:28.923 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57398 tp_osd_tp ceph NOFD /proc/55772/task/57398/fd (opendir: Permission denied) 2024-03-30T00:37:28.923 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57400 tp_osd_tp ceph cwd unknown /proc/55772/task/57400/cwd (readlink: Permission denied) 2024-03-30T00:37:28.923 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57400 tp_osd_tp ceph rtd unknown /proc/55772/task/57400/root (readlink: Permission denied) 2024-03-30T00:37:28.923 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57400 tp_osd_tp ceph txt unknown /proc/55772/task/57400/exe (readlink: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57400 tp_osd_tp ceph NOFD /proc/55772/task/57400/fd (opendir: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57401 tp_osd_tp ceph cwd unknown /proc/55772/task/57401/cwd (readlink: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57401 tp_osd_tp ceph rtd unknown /proc/55772/task/57401/root (readlink: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57401 tp_osd_tp ceph txt unknown /proc/55772/task/57401/exe (readlink: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57401 tp_osd_tp ceph NOFD /proc/55772/task/57401/fd (opendir: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57403 tp_osd_tp ceph cwd unknown /proc/55772/task/57403/cwd (readlink: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57403 tp_osd_tp ceph rtd unknown /proc/55772/task/57403/root (readlink: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57403 tp_osd_tp ceph txt unknown /proc/55772/task/57403/exe (readlink: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57403 tp_osd_tp ceph NOFD /proc/55772/task/57403/fd (opendir: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57404 tp_osd_tp ceph cwd unknown /proc/55772/task/57404/cwd (readlink: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57404 tp_osd_tp ceph rtd unknown /proc/55772/task/57404/root (readlink: Permission denied) 2024-03-30T00:37:28.924 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57404 tp_osd_tp ceph txt unknown /proc/55772/task/57404/exe (readlink: Permission denied) 2024-03-30T00:37:28.925 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57404 tp_osd_tp ceph NOFD /proc/55772/task/57404/fd (opendir: Permission denied) 2024-03-30T00:37:28.925 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57407 tp_osd_tp ceph cwd unknown /proc/55772/task/57407/cwd (readlink: Permission denied) 2024-03-30T00:37:28.925 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57407 tp_osd_tp ceph rtd unknown /proc/55772/task/57407/root (readlink: Permission denied) 2024-03-30T00:37:28.925 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57407 tp_osd_tp ceph txt unknown /proc/55772/task/57407/exe (readlink: Permission denied) 2024-03-30T00:37:28.925 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57407 tp_osd_tp ceph NOFD /proc/55772/task/57407/fd (opendir: Permission denied) 2024-03-30T00:37:28.925 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57408 tp_osd_tp ceph cwd unknown /proc/55772/task/57408/cwd (readlink: Permission denied) 2024-03-30T00:37:28.925 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57408 tp_osd_tp ceph rtd unknown /proc/55772/task/57408/root (readlink: Permission denied) 2024-03-30T00:37:28.925 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57408 tp_osd_tp ceph txt unknown /proc/55772/task/57408/exe (readlink: Permission denied) 2024-03-30T00:37:28.925 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57408 tp_osd_tp ceph NOFD /proc/55772/task/57408/fd (opendir: Permission denied) 2024-03-30T00:37:28.925 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57410 tp_osd_tp ceph cwd unknown /proc/55772/task/57410/cwd (readlink: Permission denied) 2024-03-30T00:37:28.925 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57410 tp_osd_tp ceph rtd unknown /proc/55772/task/57410/root (readlink: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57410 tp_osd_tp ceph txt unknown /proc/55772/task/57410/exe (readlink: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57410 tp_osd_tp ceph NOFD /proc/55772/task/57410/fd (opendir: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57411 tp_osd_tp ceph cwd unknown /proc/55772/task/57411/cwd (readlink: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57411 tp_osd_tp ceph rtd unknown /proc/55772/task/57411/root (readlink: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57411 tp_osd_tp ceph txt unknown /proc/55772/task/57411/exe (readlink: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57411 tp_osd_tp ceph NOFD /proc/55772/task/57411/fd (opendir: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57413 tp_osd_tp ceph cwd unknown /proc/55772/task/57413/cwd (readlink: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57413 tp_osd_tp ceph rtd unknown /proc/55772/task/57413/root (readlink: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57413 tp_osd_tp ceph txt unknown /proc/55772/task/57413/exe (readlink: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57413 tp_osd_tp ceph NOFD /proc/55772/task/57413/fd (opendir: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57414 tp_osd_tp ceph cwd unknown /proc/55772/task/57414/cwd (readlink: Permission denied) 2024-03-30T00:37:28.926 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57414 tp_osd_tp ceph rtd unknown /proc/55772/task/57414/root (readlink: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57414 tp_osd_tp ceph txt unknown /proc/55772/task/57414/exe (readlink: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57414 tp_osd_tp ceph NOFD /proc/55772/task/57414/fd (opendir: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57415 tp_osd_tp ceph cwd unknown /proc/55772/task/57415/cwd (readlink: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57415 tp_osd_tp ceph rtd unknown /proc/55772/task/57415/root (readlink: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57415 tp_osd_tp ceph txt unknown /proc/55772/task/57415/exe (readlink: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57415 tp_osd_tp ceph NOFD /proc/55772/task/57415/fd (opendir: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57416 tp_osd_tp ceph cwd unknown /proc/55772/task/57416/cwd (readlink: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57416 tp_osd_tp ceph rtd unknown /proc/55772/task/57416/root (readlink: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57416 tp_osd_tp ceph txt unknown /proc/55772/task/57416/exe (readlink: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57416 tp_osd_tp ceph NOFD /proc/55772/task/57416/fd (opendir: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57417 tp_osd_tp ceph cwd unknown /proc/55772/task/57417/cwd (readlink: Permission denied) 2024-03-30T00:37:28.927 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57417 tp_osd_tp ceph rtd unknown /proc/55772/task/57417/root (readlink: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57417 tp_osd_tp ceph txt unknown /proc/55772/task/57417/exe (readlink: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57417 tp_osd_tp ceph NOFD /proc/55772/task/57417/fd (opendir: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57418 osd_srv_h ceph cwd unknown /proc/55772/task/57418/cwd (readlink: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57418 osd_srv_h ceph rtd unknown /proc/55772/task/57418/root (readlink: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57418 osd_srv_h ceph txt unknown /proc/55772/task/57418/exe (readlink: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 55772 57418 osd_srv_h ceph NOFD /proc/55772/task/57418/fd (opendir: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 56383 root cwd unknown /proc/56383/cwd (readlink: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 56383 root rtd unknown /proc/56383/root (readlink: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 56383 root txt unknown /proc/56383/exe (readlink: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 56383 root NOFD /proc/56383/fd (opendir: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 56661 root cwd unknown /proc/56661/cwd (readlink: Permission denied) 2024-03-30T00:37:28.928 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 56661 root rtd unknown /proc/56661/root (readlink: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 56661 root txt unknown /proc/56661/exe (readlink: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 56661 root NOFD /proc/56661/fd (opendir: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 56700 root cwd unknown /proc/56700/cwd (readlink: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 56700 root rtd unknown /proc/56700/root (readlink: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 56700 root txt unknown /proc/56700/exe (readlink: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 56700 root NOFD /proc/56700/fd (opendir: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 64120 root cwd unknown /proc/64120/cwd (readlink: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 64120 root rtd unknown /proc/64120/root (readlink: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 64120 root txt unknown /proc/64120/exe (readlink: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 64120 root NOFD /proc/64120/fd (opendir: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 64131 root cwd unknown /proc/64131/cwd (readlink: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 64131 root rtd unknown /proc/64131/root (readlink: Permission denied) 2024-03-30T00:37:28.929 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 64131 root txt unknown /proc/64131/exe (readlink: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 64131 root NOFD /proc/64131/fd (opendir: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 ceph cwd unknown /proc/64143/cwd (readlink: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 ceph rtd unknown /proc/64143/root (readlink: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 ceph txt unknown /proc/64143/exe (readlink: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 ceph NOFD /proc/64143/fd (opendir: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64144 log ceph cwd unknown /proc/64143/task/64144/cwd (readlink: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64144 log ceph rtd unknown /proc/64143/task/64144/root (readlink: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64144 log ceph txt unknown /proc/64143/task/64144/exe (readlink: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64144 log ceph NOFD /proc/64143/task/64144/fd (opendir: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64146 msgr-work ceph cwd unknown /proc/64143/task/64146/cwd (readlink: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64146 msgr-work ceph rtd unknown /proc/64143/task/64146/root (readlink: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64146 msgr-work ceph txt unknown /proc/64143/task/64146/exe (readlink: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64146 msgr-work ceph NOFD /proc/64143/task/64146/fd (opendir: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64147 msgr-work ceph cwd unknown /proc/64143/task/64147/cwd (readlink: Permission denied) 2024-03-30T00:37:28.930 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64147 msgr-work ceph rtd unknown /proc/64143/task/64147/root (readlink: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64147 msgr-work ceph txt unknown /proc/64143/task/64147/exe (readlink: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64147 msgr-work ceph NOFD /proc/64143/task/64147/fd (opendir: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64148 msgr-work ceph cwd unknown /proc/64143/task/64148/cwd (readlink: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64148 msgr-work ceph rtd unknown /proc/64143/task/64148/root (readlink: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64148 msgr-work ceph txt unknown /proc/64143/task/64148/exe (readlink: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64148 msgr-work ceph NOFD /proc/64143/task/64148/fd (opendir: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64152 service ceph cwd unknown /proc/64143/task/64152/cwd (readlink: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64152 service ceph rtd unknown /proc/64143/task/64152/root (readlink: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64152 service ceph txt unknown /proc/64143/task/64152/exe (readlink: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64152 service ceph NOFD /proc/64143/task/64152/fd (opendir: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64153 admin_soc ceph cwd unknown /proc/64143/task/64153/cwd (readlink: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64153 admin_soc ceph rtd unknown /proc/64143/task/64153/root (readlink: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64153 admin_soc ceph txt unknown /proc/64143/task/64153/exe (readlink: Permission denied) 2024-03-30T00:37:28.931 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64153 admin_soc ceph NOFD /proc/64143/task/64153/fd (opendir: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64365 signal_ha ceph cwd unknown /proc/64143/task/64365/cwd (readlink: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64365 signal_ha ceph rtd unknown /proc/64143/task/64365/root (readlink: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64365 signal_ha ceph txt unknown /proc/64143/task/64365/exe (readlink: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64365 signal_ha ceph NOFD /proc/64143/task/64365/fd (opendir: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64366 io_contex ceph cwd unknown /proc/64143/task/64366/cwd (readlink: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64366 io_contex ceph rtd unknown /proc/64143/task/64366/root (readlink: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64366 io_contex ceph txt unknown /proc/64143/task/64366/exe (readlink: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64366 io_contex ceph NOFD /proc/64143/task/64366/fd (opendir: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64367 io_contex ceph cwd unknown /proc/64143/task/64367/cwd (readlink: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64367 io_contex ceph rtd unknown /proc/64143/task/64367/root (readlink: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64367 io_contex ceph txt unknown /proc/64143/task/64367/exe (readlink: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64367 io_contex ceph NOFD /proc/64143/task/64367/fd (opendir: Permission denied) 2024-03-30T00:37:28.932 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64444 OpHistory ceph cwd unknown /proc/64143/task/64444/cwd (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64444 OpHistory ceph rtd unknown /proc/64143/task/64444/root (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64444 OpHistory ceph txt unknown /proc/64143/task/64444/exe (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64444 OpHistory ceph NOFD /proc/64143/task/64444/fd (opendir: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64445 ceph_time ceph cwd unknown /proc/64143/task/64445/cwd (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64445 ceph_time ceph rtd unknown /proc/64143/task/64445/root (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64445 ceph_time ceph txt unknown /proc/64143/task/64445/exe (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64445 ceph_time ceph NOFD /proc/64143/task/64445/fd (opendir: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64861 ceph-osd ceph cwd unknown /proc/64143/task/64861/cwd (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64861 ceph-osd ceph rtd unknown /proc/64143/task/64861/root (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64861 ceph-osd ceph txt unknown /proc/64143/task/64861/exe (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64861 ceph-osd ceph NOFD /proc/64143/task/64861/fd (opendir: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64862 safe_time ceph cwd unknown /proc/64143/task/64862/cwd (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64862 safe_time ceph rtd unknown /proc/64143/task/64862/root (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64862 safe_time ceph txt unknown /proc/64143/task/64862/exe (readlink: Permission denied) 2024-03-30T00:37:28.933 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64862 safe_time ceph NOFD /proc/64143/task/64862/fd (opendir: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64863 safe_time ceph cwd unknown /proc/64143/task/64863/cwd (readlink: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64863 safe_time ceph rtd unknown /proc/64143/task/64863/root (readlink: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64863 safe_time ceph txt unknown /proc/64143/task/64863/exe (readlink: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64863 safe_time ceph NOFD /proc/64143/task/64863/fd (opendir: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64864 safe_time ceph cwd unknown /proc/64143/task/64864/cwd (readlink: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64864 safe_time ceph rtd unknown /proc/64143/task/64864/root (readlink: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64864 safe_time ceph txt unknown /proc/64143/task/64864/exe (readlink: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64864 safe_time ceph NOFD /proc/64143/task/64864/fd (opendir: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64865 safe_time ceph cwd unknown /proc/64143/task/64865/cwd (readlink: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64865 safe_time ceph rtd unknown /proc/64143/task/64865/root (readlink: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64865 safe_time ceph txt unknown /proc/64143/task/64865/exe (readlink: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64865 safe_time ceph NOFD /proc/64143/task/64865/fd (opendir: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64866 fn_anonym ceph cwd unknown /proc/64143/task/64866/cwd (readlink: Permission denied) 2024-03-30T00:37:28.934 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64866 fn_anonym ceph rtd unknown /proc/64143/task/64866/root (readlink: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64866 fn_anonym ceph txt unknown /proc/64143/task/64866/exe (readlink: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64866 fn_anonym ceph NOFD /proc/64143/task/64866/fd (opendir: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64873 rocksdb:l ceph cwd unknown /proc/64143/task/64873/cwd (readlink: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64873 rocksdb:l ceph rtd unknown /proc/64143/task/64873/root (readlink: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64873 rocksdb:l ceph txt unknown /proc/64143/task/64873/exe (readlink: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64873 rocksdb:l ceph NOFD /proc/64143/task/64873/fd (opendir: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64874 rocksdb:l ceph cwd unknown /proc/64143/task/64874/cwd (readlink: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64874 rocksdb:l ceph rtd unknown /proc/64143/task/64874/root (readlink: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64874 rocksdb:l ceph txt unknown /proc/64143/task/64874/exe (readlink: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64874 rocksdb:l ceph NOFD /proc/64143/task/64874/fd (opendir: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64875 rocksdb:l ceph cwd unknown /proc/64143/task/64875/cwd (readlink: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64875 rocksdb:l ceph rtd unknown /proc/64143/task/64875/root (readlink: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64875 rocksdb:l ceph txt unknown /proc/64143/task/64875/exe (readlink: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64875 rocksdb:l ceph NOFD /proc/64143/task/64875/fd (opendir: Permission denied) 2024-03-30T00:37:28.935 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64876 rocksdb:h ceph cwd unknown /proc/64143/task/64876/cwd (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64876 rocksdb:h ceph rtd unknown /proc/64143/task/64876/root (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64876 rocksdb:h ceph txt unknown /proc/64143/task/64876/exe (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 64876 rocksdb:h ceph NOFD /proc/64143/task/64876/fd (opendir: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65416 bstore_ai ceph cwd unknown /proc/64143/task/65416/cwd (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65416 bstore_ai ceph rtd unknown /proc/64143/task/65416/root (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65416 bstore_ai ceph txt unknown /proc/64143/task/65416/exe (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65416 bstore_ai ceph NOFD /proc/64143/task/65416/fd (opendir: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65417 bstore_di ceph cwd unknown /proc/64143/task/65417/cwd (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65417 bstore_di ceph rtd unknown /proc/64143/task/65417/root (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65417 bstore_di ceph txt unknown /proc/64143/task/65417/exe (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65417 bstore_di ceph NOFD /proc/64143/task/65417/fd (opendir: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65621 bstore_ai ceph cwd unknown /proc/64143/task/65621/cwd (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65621 bstore_ai ceph rtd unknown /proc/64143/task/65621/root (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65621 bstore_ai ceph txt unknown /proc/64143/task/65621/exe (readlink: Permission denied) 2024-03-30T00:37:28.936 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65621 bstore_ai ceph NOFD /proc/64143/task/65621/fd (opendir: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65622 bstore_di ceph cwd unknown /proc/64143/task/65622/cwd (readlink: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65622 bstore_di ceph rtd unknown /proc/64143/task/65622/root (readlink: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65622 bstore_di ceph txt unknown /proc/64143/task/65622/exe (readlink: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65622 bstore_di ceph NOFD /proc/64143/task/65622/fd (opendir: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65810 ceph-osd ceph cwd unknown /proc/64143/task/65810/cwd (readlink: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65810 ceph-osd ceph rtd unknown /proc/64143/task/65810/root (readlink: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65810 ceph-osd ceph txt unknown /proc/64143/task/65810/exe (readlink: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65810 ceph-osd ceph NOFD /proc/64143/task/65810/fd (opendir: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65811 cfin ceph cwd unknown /proc/64143/task/65811/cwd (readlink: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65811 cfin ceph rtd unknown /proc/64143/task/65811/root (readlink: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65811 cfin ceph txt unknown /proc/64143/task/65811/exe (readlink: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65811 cfin ceph NOFD /proc/64143/task/65811/fd (opendir: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65812 bstore_kv ceph cwd unknown /proc/64143/task/65812/cwd (readlink: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65812 bstore_kv ceph rtd unknown /proc/64143/task/65812/root (readlink: Permission denied) 2024-03-30T00:37:28.937 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65812 bstore_kv ceph txt unknown /proc/64143/task/65812/exe (readlink: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65812 bstore_kv ceph NOFD /proc/64143/task/65812/fd (opendir: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65813 bstore_kv ceph cwd unknown /proc/64143/task/65813/cwd (readlink: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65813 bstore_kv ceph rtd unknown /proc/64143/task/65813/root (readlink: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65813 bstore_kv ceph txt unknown /proc/64143/task/65813/exe (readlink: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65813 bstore_kv ceph NOFD /proc/64143/task/65813/fd (opendir: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65814 bstore_me ceph cwd unknown /proc/64143/task/65814/cwd (readlink: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65814 bstore_me ceph rtd unknown /proc/64143/task/65814/root (readlink: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65814 bstore_me ceph txt unknown /proc/64143/task/65814/exe (readlink: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65814 bstore_me ceph NOFD /proc/64143/task/65814/fd (opendir: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65815 ms_dispat ceph cwd unknown /proc/64143/task/65815/cwd (readlink: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65815 ms_dispat ceph rtd unknown /proc/64143/task/65815/root (readlink: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65815 ms_dispat ceph txt unknown /proc/64143/task/65815/exe (readlink: Permission denied) 2024-03-30T00:37:28.938 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65815 ms_dispat ceph NOFD /proc/64143/task/65815/fd (opendir: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65816 ms_local ceph cwd unknown /proc/64143/task/65816/cwd (readlink: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65816 ms_local ceph rtd unknown /proc/64143/task/65816/root (readlink: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65816 ms_local ceph txt unknown /proc/64143/task/65816/exe (readlink: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65816 ms_local ceph NOFD /proc/64143/task/65816/fd (opendir: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65817 safe_time ceph cwd unknown /proc/64143/task/65817/cwd (readlink: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65817 safe_time ceph rtd unknown /proc/64143/task/65817/root (readlink: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65817 safe_time ceph txt unknown /proc/64143/task/65817/exe (readlink: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65817 safe_time ceph NOFD /proc/64143/task/65817/fd (opendir: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65818 safe_time ceph cwd unknown /proc/64143/task/65818/cwd (readlink: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65818 safe_time ceph rtd unknown /proc/64143/task/65818/root (readlink: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65818 safe_time ceph txt unknown /proc/64143/task/65818/exe (readlink: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65818 safe_time ceph NOFD /proc/64143/task/65818/fd (opendir: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65819 ms_dispat ceph cwd unknown /proc/64143/task/65819/cwd (readlink: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65819 ms_dispat ceph rtd unknown /proc/64143/task/65819/root (readlink: Permission denied) 2024-03-30T00:37:28.939 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65819 ms_dispat ceph txt unknown /proc/64143/task/65819/exe (readlink: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65819 ms_dispat ceph NOFD /proc/64143/task/65819/fd (opendir: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65820 ms_local ceph cwd unknown /proc/64143/task/65820/cwd (readlink: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65820 ms_local ceph rtd unknown /proc/64143/task/65820/root (readlink: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65820 ms_local ceph txt unknown /proc/64143/task/65820/exe (readlink: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65820 ms_local ceph NOFD /proc/64143/task/65820/fd (opendir: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65821 ms_dispat ceph cwd unknown /proc/64143/task/65821/cwd (readlink: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65821 ms_dispat ceph rtd unknown /proc/64143/task/65821/root (readlink: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65821 ms_dispat ceph txt unknown /proc/64143/task/65821/exe (readlink: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65821 ms_dispat ceph NOFD /proc/64143/task/65821/fd (opendir: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65822 ms_local ceph cwd unknown /proc/64143/task/65822/cwd (readlink: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65822 ms_local ceph rtd unknown /proc/64143/task/65822/root (readlink: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65822 ms_local ceph txt unknown /proc/64143/task/65822/exe (readlink: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65822 ms_local ceph NOFD /proc/64143/task/65822/fd (opendir: Permission denied) 2024-03-30T00:37:28.940 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65823 ms_dispat ceph cwd unknown /proc/64143/task/65823/cwd (readlink: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65823 ms_dispat ceph rtd unknown /proc/64143/task/65823/root (readlink: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65823 ms_dispat ceph txt unknown /proc/64143/task/65823/exe (readlink: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65823 ms_dispat ceph NOFD /proc/64143/task/65823/fd (opendir: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65824 ms_local ceph cwd unknown /proc/64143/task/65824/cwd (readlink: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65824 ms_local ceph rtd unknown /proc/64143/task/65824/root (readlink: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65824 ms_local ceph txt unknown /proc/64143/task/65824/exe (readlink: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65824 ms_local ceph NOFD /proc/64143/task/65824/fd (opendir: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65825 ms_dispat ceph cwd unknown /proc/64143/task/65825/cwd (readlink: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65825 ms_dispat ceph rtd unknown /proc/64143/task/65825/root (readlink: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65825 ms_dispat ceph txt unknown /proc/64143/task/65825/exe (readlink: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65825 ms_dispat ceph NOFD /proc/64143/task/65825/fd (opendir: Permission denied) 2024-03-30T00:37:28.941 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65826 ms_local ceph cwd unknown /proc/64143/task/65826/cwd (readlink: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65826 ms_local ceph rtd unknown /proc/64143/task/65826/root (readlink: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65826 ms_local ceph txt unknown /proc/64143/task/65826/exe (readlink: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65826 ms_local ceph NOFD /proc/64143/task/65826/fd (opendir: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65827 ms_dispat ceph cwd unknown /proc/64143/task/65827/cwd (readlink: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65827 ms_dispat ceph rtd unknown /proc/64143/task/65827/root (readlink: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65827 ms_dispat ceph txt unknown /proc/64143/task/65827/exe (readlink: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65827 ms_dispat ceph NOFD /proc/64143/task/65827/fd (opendir: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65828 ms_local ceph cwd unknown /proc/64143/task/65828/cwd (readlink: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65828 ms_local ceph rtd unknown /proc/64143/task/65828/root (readlink: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65828 ms_local ceph txt unknown /proc/64143/task/65828/exe (readlink: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65828 ms_local ceph NOFD /proc/64143/task/65828/fd (opendir: Permission denied) 2024-03-30T00:37:28.942 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65829 ms_dispat ceph cwd unknown /proc/64143/task/65829/cwd (readlink: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65829 ms_dispat ceph rtd unknown /proc/64143/task/65829/root (readlink: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65829 ms_dispat ceph txt unknown /proc/64143/task/65829/exe (readlink: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65829 ms_dispat ceph NOFD /proc/64143/task/65829/fd (opendir: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65830 ms_local ceph cwd unknown /proc/64143/task/65830/cwd (readlink: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65830 ms_local ceph rtd unknown /proc/64143/task/65830/root (readlink: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65830 ms_local ceph txt unknown /proc/64143/task/65830/exe (readlink: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65830 ms_local ceph NOFD /proc/64143/task/65830/fd (opendir: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65831 fn_anonym ceph cwd unknown /proc/64143/task/65831/cwd (readlink: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65831 fn_anonym ceph rtd unknown /proc/64143/task/65831/root (readlink: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65831 fn_anonym ceph txt unknown /proc/64143/task/65831/exe (readlink: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65831 fn_anonym ceph NOFD /proc/64143/task/65831/fd (opendir: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65832 finisher ceph cwd unknown /proc/64143/task/65832/cwd (readlink: Permission denied) 2024-03-30T00:37:28.943 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65832 finisher ceph rtd unknown /proc/64143/task/65832/root (readlink: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65832 finisher ceph txt unknown /proc/64143/task/65832/exe (readlink: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65832 finisher ceph NOFD /proc/64143/task/65832/fd (opendir: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65833 safe_time ceph cwd unknown /proc/64143/task/65833/cwd (readlink: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65833 safe_time ceph rtd unknown /proc/64143/task/65833/root (readlink: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65833 safe_time ceph txt unknown /proc/64143/task/65833/exe (readlink: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65833 safe_time ceph NOFD /proc/64143/task/65833/fd (opendir: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65834 safe_time ceph cwd unknown /proc/64143/task/65834/cwd (readlink: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65834 safe_time ceph rtd unknown /proc/64143/task/65834/root (readlink: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65834 safe_time ceph txt unknown /proc/64143/task/65834/exe (readlink: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65834 safe_time ceph NOFD /proc/64143/task/65834/fd (opendir: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65835 ceph-osd ceph cwd unknown /proc/64143/task/65835/cwd (readlink: Permission denied) 2024-03-30T00:37:28.944 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65835 ceph-osd ceph rtd unknown /proc/64143/task/65835/root (readlink: Permission denied) 2024-03-30T00:37:28.945 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65835 ceph-osd ceph txt unknown /proc/64143/task/65835/exe (readlink: Permission denied) 2024-03-30T00:37:28.945 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65835 ceph-osd ceph NOFD /proc/64143/task/65835/fd (opendir: Permission denied) 2024-03-30T00:37:28.945 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65836 osd_srv_a ceph cwd unknown /proc/64143/task/65836/cwd (readlink: Permission denied) 2024-03-30T00:37:28.945 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65836 osd_srv_a ceph rtd unknown /proc/64143/task/65836/root (readlink: Permission denied) 2024-03-30T00:37:28.945 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65836 osd_srv_a ceph txt unknown /proc/64143/task/65836/exe (readlink: Permission denied) 2024-03-30T00:37:28.945 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65836 osd_srv_a ceph NOFD /proc/64143/task/65836/fd (opendir: Permission denied) 2024-03-30T00:37:28.945 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65837 tp_osd_tp ceph cwd unknown /proc/64143/task/65837/cwd (readlink: Permission denied) 2024-03-30T00:37:28.945 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65837 tp_osd_tp ceph rtd unknown /proc/64143/task/65837/root (readlink: Permission denied) 2024-03-30T00:37:28.945 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65837 tp_osd_tp ceph txt unknown /proc/64143/task/65837/exe (readlink: Permission denied) 2024-03-30T00:37:28.945 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65837 tp_osd_tp ceph NOFD /proc/64143/task/65837/fd (opendir: Permission denied) 2024-03-30T00:37:28.945 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65838 tp_osd_tp ceph cwd unknown /proc/64143/task/65838/cwd (readlink: Permission denied) 2024-03-30T00:37:28.946 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65838 tp_osd_tp ceph rtd unknown /proc/64143/task/65838/root (readlink: Permission denied) 2024-03-30T00:37:28.946 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65838 tp_osd_tp ceph txt unknown /proc/64143/task/65838/exe (readlink: Permission denied) 2024-03-30T00:37:28.946 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65838 tp_osd_tp ceph NOFD /proc/64143/task/65838/fd (opendir: Permission denied) 2024-03-30T00:37:28.946 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65839 tp_osd_tp ceph cwd unknown /proc/64143/task/65839/cwd (readlink: Permission denied) 2024-03-30T00:37:28.946 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65839 tp_osd_tp ceph rtd unknown /proc/64143/task/65839/root (readlink: Permission denied) 2024-03-30T00:37:28.946 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65839 tp_osd_tp ceph txt unknown /proc/64143/task/65839/exe (readlink: Permission denied) 2024-03-30T00:37:28.946 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65839 tp_osd_tp ceph NOFD /proc/64143/task/65839/fd (opendir: Permission denied) 2024-03-30T00:37:28.946 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65840 tp_osd_tp ceph cwd unknown /proc/64143/task/65840/cwd (readlink: Permission denied) 2024-03-30T00:37:28.946 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65840 tp_osd_tp ceph rtd unknown /proc/64143/task/65840/root (readlink: Permission denied) 2024-03-30T00:37:28.946 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65840 tp_osd_tp ceph txt unknown /proc/64143/task/65840/exe (readlink: Permission denied) 2024-03-30T00:37:28.947 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65840 tp_osd_tp ceph NOFD /proc/64143/task/65840/fd (opendir: Permission denied) 2024-03-30T00:37:28.947 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65841 tp_osd_tp ceph cwd unknown /proc/64143/task/65841/cwd (readlink: Permission denied) 2024-03-30T00:37:28.947 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65841 tp_osd_tp ceph rtd unknown /proc/64143/task/65841/root (readlink: Permission denied) 2024-03-30T00:37:28.947 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65841 tp_osd_tp ceph txt unknown /proc/64143/task/65841/exe (readlink: Permission denied) 2024-03-30T00:37:28.947 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65841 tp_osd_tp ceph NOFD /proc/64143/task/65841/fd (opendir: Permission denied) 2024-03-30T00:37:28.947 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65842 tp_osd_tp ceph cwd unknown /proc/64143/task/65842/cwd (readlink: Permission denied) 2024-03-30T00:37:28.947 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65842 tp_osd_tp ceph rtd unknown /proc/64143/task/65842/root (readlink: Permission denied) 2024-03-30T00:37:28.947 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65842 tp_osd_tp ceph txt unknown /proc/64143/task/65842/exe (readlink: Permission denied) 2024-03-30T00:37:28.947 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65842 tp_osd_tp ceph NOFD /proc/64143/task/65842/fd (opendir: Permission denied) 2024-03-30T00:37:28.947 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65843 tp_osd_tp ceph cwd unknown /proc/64143/task/65843/cwd (readlink: Permission denied) 2024-03-30T00:37:28.947 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65843 tp_osd_tp ceph rtd unknown /proc/64143/task/65843/root (readlink: Permission denied) 2024-03-30T00:37:28.948 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65843 tp_osd_tp ceph txt unknown /proc/64143/task/65843/exe (readlink: Permission denied) 2024-03-30T00:37:28.948 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65843 tp_osd_tp ceph NOFD /proc/64143/task/65843/fd (opendir: Permission denied) 2024-03-30T00:37:28.948 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65844 tp_osd_tp ceph cwd unknown /proc/64143/task/65844/cwd (readlink: Permission denied) 2024-03-30T00:37:28.948 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65844 tp_osd_tp ceph rtd unknown /proc/64143/task/65844/root (readlink: Permission denied) 2024-03-30T00:37:28.948 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65844 tp_osd_tp ceph txt unknown /proc/64143/task/65844/exe (readlink: Permission denied) 2024-03-30T00:37:28.948 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65844 tp_osd_tp ceph NOFD /proc/64143/task/65844/fd (opendir: Permission denied) 2024-03-30T00:37:28.948 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65845 tp_osd_tp ceph cwd unknown /proc/64143/task/65845/cwd (readlink: Permission denied) 2024-03-30T00:37:28.948 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65845 tp_osd_tp ceph rtd unknown /proc/64143/task/65845/root (readlink: Permission denied) 2024-03-30T00:37:28.948 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65845 tp_osd_tp ceph txt unknown /proc/64143/task/65845/exe (readlink: Permission denied) 2024-03-30T00:37:28.948 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65845 tp_osd_tp ceph NOFD /proc/64143/task/65845/fd (opendir: Permission denied) 2024-03-30T00:37:28.949 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65846 tp_osd_tp ceph cwd unknown /proc/64143/task/65846/cwd (readlink: Permission denied) 2024-03-30T00:37:28.949 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65846 tp_osd_tp ceph rtd unknown /proc/64143/task/65846/root (readlink: Permission denied) 2024-03-30T00:37:28.949 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65846 tp_osd_tp ceph txt unknown /proc/64143/task/65846/exe (readlink: Permission denied) 2024-03-30T00:37:28.949 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65846 tp_osd_tp ceph NOFD /proc/64143/task/65846/fd (opendir: Permission denied) 2024-03-30T00:37:28.949 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65847 tp_osd_tp ceph cwd unknown /proc/64143/task/65847/cwd (readlink: Permission denied) 2024-03-30T00:37:28.949 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65847 tp_osd_tp ceph rtd unknown /proc/64143/task/65847/root (readlink: Permission denied) 2024-03-30T00:37:28.949 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65847 tp_osd_tp ceph txt unknown /proc/64143/task/65847/exe (readlink: Permission denied) 2024-03-30T00:37:28.949 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65847 tp_osd_tp ceph NOFD /proc/64143/task/65847/fd (opendir: Permission denied) 2024-03-30T00:37:28.949 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65848 tp_osd_tp ceph cwd unknown /proc/64143/task/65848/cwd (readlink: Permission denied) 2024-03-30T00:37:28.950 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65848 tp_osd_tp ceph rtd unknown /proc/64143/task/65848/root (readlink: Permission denied) 2024-03-30T00:37:28.950 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65848 tp_osd_tp ceph txt unknown /proc/64143/task/65848/exe (readlink: Permission denied) 2024-03-30T00:37:28.950 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65848 tp_osd_tp ceph NOFD /proc/64143/task/65848/fd (opendir: Permission denied) 2024-03-30T00:37:28.950 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65849 tp_osd_tp ceph cwd unknown /proc/64143/task/65849/cwd (readlink: Permission denied) 2024-03-30T00:37:28.950 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65849 tp_osd_tp ceph rtd unknown /proc/64143/task/65849/root (readlink: Permission denied) 2024-03-30T00:37:28.950 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65849 tp_osd_tp ceph txt unknown /proc/64143/task/65849/exe (readlink: Permission denied) 2024-03-30T00:37:28.950 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65849 tp_osd_tp ceph NOFD /proc/64143/task/65849/fd (opendir: Permission denied) 2024-03-30T00:37:28.950 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65850 tp_osd_tp ceph cwd unknown /proc/64143/task/65850/cwd (readlink: Permission denied) 2024-03-30T00:37:28.950 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65850 tp_osd_tp ceph rtd unknown /proc/64143/task/65850/root (readlink: Permission denied) 2024-03-30T00:37:28.950 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65850 tp_osd_tp ceph txt unknown /proc/64143/task/65850/exe (readlink: Permission denied) 2024-03-30T00:37:28.951 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65850 tp_osd_tp ceph NOFD /proc/64143/task/65850/fd (opendir: Permission denied) 2024-03-30T00:37:28.951 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65851 tp_osd_tp ceph cwd unknown /proc/64143/task/65851/cwd (readlink: Permission denied) 2024-03-30T00:37:28.951 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65851 tp_osd_tp ceph rtd unknown /proc/64143/task/65851/root (readlink: Permission denied) 2024-03-30T00:37:28.951 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65851 tp_osd_tp ceph txt unknown /proc/64143/task/65851/exe (readlink: Permission denied) 2024-03-30T00:37:28.951 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65851 tp_osd_tp ceph NOFD /proc/64143/task/65851/fd (opendir: Permission denied) 2024-03-30T00:37:28.951 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65852 tp_osd_tp ceph cwd unknown /proc/64143/task/65852/cwd (readlink: Permission denied) 2024-03-30T00:37:28.951 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65852 tp_osd_tp ceph rtd unknown /proc/64143/task/65852/root (readlink: Permission denied) 2024-03-30T00:37:28.951 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65852 tp_osd_tp ceph txt unknown /proc/64143/task/65852/exe (readlink: Permission denied) 2024-03-30T00:37:28.951 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65852 tp_osd_tp ceph NOFD /proc/64143/task/65852/fd (opendir: Permission denied) 2024-03-30T00:37:28.951 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65853 osd_srv_h ceph cwd unknown /proc/64143/task/65853/cwd (readlink: Permission denied) 2024-03-30T00:37:28.952 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65853 osd_srv_h ceph rtd unknown /proc/64143/task/65853/root (readlink: Permission denied) 2024-03-30T00:37:28.952 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65853 osd_srv_h ceph txt unknown /proc/64143/task/65853/exe (readlink: Permission denied) 2024-03-30T00:37:28.952 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 64143 65853 osd_srv_h ceph NOFD /proc/64143/task/65853/fd (opendir: Permission denied) 2024-03-30T00:37:28.952 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 65108 root cwd unknown /proc/65108/cwd (readlink: Permission denied) 2024-03-30T00:37:28.952 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 65108 root rtd unknown /proc/65108/root (readlink: Permission denied) 2024-03-30T00:37:28.952 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 65108 root txt unknown /proc/65108/exe (readlink: Permission denied) 2024-03-30T00:37:28.952 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 65108 root NOFD /proc/65108/fd (opendir: Permission denied) 2024-03-30T00:37:28.952 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 65142 root cwd unknown /proc/65142/cwd (readlink: Permission denied) 2024-03-30T00:37:28.952 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 65142 root rtd unknown /proc/65142/root (readlink: Permission denied) 2024-03-30T00:37:28.952 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 65142 root txt unknown /proc/65142/exe (readlink: Permission denied) 2024-03-30T00:37:28.952 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 65142 root NOFD /proc/65142/fd (opendir: Permission denied) 2024-03-30T00:37:28.953 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 72166 root cwd unknown /proc/72166/cwd (readlink: Permission denied) 2024-03-30T00:37:28.953 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 72166 root rtd unknown /proc/72166/root (readlink: Permission denied) 2024-03-30T00:37:28.953 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 72166 root txt unknown /proc/72166/exe (readlink: Permission denied) 2024-03-30T00:37:28.953 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 72166 root NOFD /proc/72166/fd (opendir: Permission denied) 2024-03-30T00:37:28.953 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 72177 root cwd unknown /proc/72177/cwd (readlink: Permission denied) 2024-03-30T00:37:28.953 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 72177 root rtd unknown /proc/72177/root (readlink: Permission denied) 2024-03-30T00:37:28.953 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 72177 root txt unknown /proc/72177/exe (readlink: Permission denied) 2024-03-30T00:37:28.953 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 72177 root NOFD /proc/72177/fd (opendir: Permission denied) 2024-03-30T00:37:28.953 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 ceph cwd unknown /proc/72189/cwd (readlink: Permission denied) 2024-03-30T00:37:28.953 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 ceph rtd unknown /proc/72189/root (readlink: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 ceph txt unknown /proc/72189/exe (readlink: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 ceph NOFD /proc/72189/fd (opendir: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72190 log ceph cwd unknown /proc/72189/task/72190/cwd (readlink: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72190 log ceph rtd unknown /proc/72189/task/72190/root (readlink: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72190 log ceph txt unknown /proc/72189/task/72190/exe (readlink: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72190 log ceph NOFD /proc/72189/task/72190/fd (opendir: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72192 msgr-work ceph cwd unknown /proc/72189/task/72192/cwd (readlink: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72192 msgr-work ceph rtd unknown /proc/72189/task/72192/root (readlink: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72192 msgr-work ceph txt unknown /proc/72189/task/72192/exe (readlink: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72192 msgr-work ceph NOFD /proc/72189/task/72192/fd (opendir: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72193 msgr-work ceph cwd unknown /proc/72189/task/72193/cwd (readlink: Permission denied) 2024-03-30T00:37:28.954 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72193 msgr-work ceph rtd unknown /proc/72189/task/72193/root (readlink: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72193 msgr-work ceph txt unknown /proc/72189/task/72193/exe (readlink: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72193 msgr-work ceph NOFD /proc/72189/task/72193/fd (opendir: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72194 msgr-work ceph cwd unknown /proc/72189/task/72194/cwd (readlink: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72194 msgr-work ceph rtd unknown /proc/72189/task/72194/root (readlink: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72194 msgr-work ceph txt unknown /proc/72189/task/72194/exe (readlink: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72194 msgr-work ceph NOFD /proc/72189/task/72194/fd (opendir: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72198 service ceph cwd unknown /proc/72189/task/72198/cwd (readlink: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72198 service ceph rtd unknown /proc/72189/task/72198/root (readlink: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72198 service ceph txt unknown /proc/72189/task/72198/exe (readlink: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72198 service ceph NOFD /proc/72189/task/72198/fd (opendir: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72199 admin_soc ceph cwd unknown /proc/72189/task/72199/cwd (readlink: Permission denied) 2024-03-30T00:37:28.955 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72199 admin_soc ceph rtd unknown /proc/72189/task/72199/root (readlink: Permission denied) 2024-03-30T00:37:28.956 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72199 admin_soc ceph txt unknown /proc/72189/task/72199/exe (readlink: Permission denied) 2024-03-30T00:37:28.956 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72199 admin_soc ceph NOFD /proc/72189/task/72199/fd (opendir: Permission denied) 2024-03-30T00:37:28.956 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72411 signal_ha ceph cwd unknown /proc/72189/task/72411/cwd (readlink: Permission denied) 2024-03-30T00:37:28.956 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72411 signal_ha ceph rtd unknown /proc/72189/task/72411/root (readlink: Permission denied) 2024-03-30T00:37:28.956 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72411 signal_ha ceph txt unknown /proc/72189/task/72411/exe (readlink: Permission denied) 2024-03-30T00:37:28.956 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72411 signal_ha ceph NOFD /proc/72189/task/72411/fd (opendir: Permission denied) 2024-03-30T00:37:28.956 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72412 io_contex ceph cwd unknown /proc/72189/task/72412/cwd (readlink: Permission denied) 2024-03-30T00:37:28.956 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72412 io_contex ceph rtd unknown /proc/72189/task/72412/root (readlink: Permission denied) 2024-03-30T00:37:28.956 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72412 io_contex ceph txt unknown /proc/72189/task/72412/exe (readlink: Permission denied) 2024-03-30T00:37:28.956 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72412 io_contex ceph NOFD /proc/72189/task/72412/fd (opendir: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72413 io_contex ceph cwd unknown /proc/72189/task/72413/cwd (readlink: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72413 io_contex ceph rtd unknown /proc/72189/task/72413/root (readlink: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72413 io_contex ceph txt unknown /proc/72189/task/72413/exe (readlink: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72413 io_contex ceph NOFD /proc/72189/task/72413/fd (opendir: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72485 OpHistory ceph cwd unknown /proc/72189/task/72485/cwd (readlink: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72485 OpHistory ceph rtd unknown /proc/72189/task/72485/root (readlink: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72485 OpHistory ceph txt unknown /proc/72189/task/72485/exe (readlink: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72485 OpHistory ceph NOFD /proc/72189/task/72485/fd (opendir: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72486 ceph_time ceph cwd unknown /proc/72189/task/72486/cwd (readlink: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72486 ceph_time ceph rtd unknown /proc/72189/task/72486/root (readlink: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72486 ceph_time ceph txt unknown /proc/72189/task/72486/exe (readlink: Permission denied) 2024-03-30T00:37:28.957 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72486 ceph_time ceph NOFD /proc/72189/task/72486/fd (opendir: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72532 ceph-osd ceph cwd unknown /proc/72189/task/72532/cwd (readlink: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72532 ceph-osd ceph rtd unknown /proc/72189/task/72532/root (readlink: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72532 ceph-osd ceph txt unknown /proc/72189/task/72532/exe (readlink: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72532 ceph-osd ceph NOFD /proc/72189/task/72532/fd (opendir: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72576 ceph-osd ceph cwd unknown /proc/72189/task/72576/cwd (readlink: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72576 ceph-osd ceph rtd unknown /proc/72189/task/72576/root (readlink: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72576 ceph-osd ceph txt unknown /proc/72189/task/72576/exe (readlink: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72576 ceph-osd ceph NOFD /proc/72189/task/72576/fd (opendir: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72614 ceph-osd ceph cwd unknown /proc/72189/task/72614/cwd (readlink: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72614 ceph-osd ceph rtd unknown /proc/72189/task/72614/root (readlink: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72614 ceph-osd ceph txt unknown /proc/72189/task/72614/exe (readlink: Permission denied) 2024-03-30T00:37:28.958 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72614 ceph-osd ceph NOFD /proc/72189/task/72614/fd (opendir: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72703 ceph-osd ceph cwd unknown /proc/72189/task/72703/cwd (readlink: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72703 ceph-osd ceph rtd unknown /proc/72189/task/72703/root (readlink: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72703 ceph-osd ceph txt unknown /proc/72189/task/72703/exe (readlink: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72703 ceph-osd ceph NOFD /proc/72189/task/72703/fd (opendir: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72782 ceph-osd ceph cwd unknown /proc/72189/task/72782/cwd (readlink: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72782 ceph-osd ceph rtd unknown /proc/72189/task/72782/root (readlink: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72782 ceph-osd ceph txt unknown /proc/72189/task/72782/exe (readlink: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72782 ceph-osd ceph NOFD /proc/72189/task/72782/fd (opendir: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72826 ceph-osd ceph cwd unknown /proc/72189/task/72826/cwd (readlink: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72826 ceph-osd ceph rtd unknown /proc/72189/task/72826/root (readlink: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72826 ceph-osd ceph txt unknown /proc/72189/task/72826/exe (readlink: Permission denied) 2024-03-30T00:37:28.959 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72826 ceph-osd ceph NOFD /proc/72189/task/72826/fd (opendir: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72897 ceph-osd ceph cwd unknown /proc/72189/task/72897/cwd (readlink: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72897 ceph-osd ceph rtd unknown /proc/72189/task/72897/root (readlink: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72897 ceph-osd ceph txt unknown /proc/72189/task/72897/exe (readlink: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72897 ceph-osd ceph NOFD /proc/72189/task/72897/fd (opendir: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72913 ceph-osd ceph cwd unknown /proc/72189/task/72913/cwd (readlink: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72913 ceph-osd ceph rtd unknown /proc/72189/task/72913/root (readlink: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72913 ceph-osd ceph txt unknown /proc/72189/task/72913/exe (readlink: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72913 ceph-osd ceph NOFD /proc/72189/task/72913/fd (opendir: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72915 ceph-osd ceph cwd unknown /proc/72189/task/72915/cwd (readlink: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72915 ceph-osd ceph rtd unknown /proc/72189/task/72915/root (readlink: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72915 ceph-osd ceph txt unknown /proc/72189/task/72915/exe (readlink: Permission denied) 2024-03-30T00:37:28.960 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72915 ceph-osd ceph NOFD /proc/72189/task/72915/fd (opendir: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72916 safe_time ceph cwd unknown /proc/72189/task/72916/cwd (readlink: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72916 safe_time ceph rtd unknown /proc/72189/task/72916/root (readlink: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72916 safe_time ceph txt unknown /proc/72189/task/72916/exe (readlink: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72916 safe_time ceph NOFD /proc/72189/task/72916/fd (opendir: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72917 safe_time ceph cwd unknown /proc/72189/task/72917/cwd (readlink: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72917 safe_time ceph rtd unknown /proc/72189/task/72917/root (readlink: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72917 safe_time ceph txt unknown /proc/72189/task/72917/exe (readlink: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72917 safe_time ceph NOFD /proc/72189/task/72917/fd (opendir: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72918 safe_time ceph cwd unknown /proc/72189/task/72918/cwd (readlink: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72918 safe_time ceph rtd unknown /proc/72189/task/72918/root (readlink: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72918 safe_time ceph txt unknown /proc/72189/task/72918/exe (readlink: Permission denied) 2024-03-30T00:37:28.961 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72918 safe_time ceph NOFD /proc/72189/task/72918/fd (opendir: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72919 safe_time ceph cwd unknown /proc/72189/task/72919/cwd (readlink: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72919 safe_time ceph rtd unknown /proc/72189/task/72919/root (readlink: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72919 safe_time ceph txt unknown /proc/72189/task/72919/exe (readlink: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72919 safe_time ceph NOFD /proc/72189/task/72919/fd (opendir: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72920 fn_anonym ceph cwd unknown /proc/72189/task/72920/cwd (readlink: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72920 fn_anonym ceph rtd unknown /proc/72189/task/72920/root (readlink: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72920 fn_anonym ceph txt unknown /proc/72189/task/72920/exe (readlink: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72920 fn_anonym ceph NOFD /proc/72189/task/72920/fd (opendir: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72932 rocksdb:l ceph cwd unknown /proc/72189/task/72932/cwd (readlink: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72932 rocksdb:l ceph rtd unknown /proc/72189/task/72932/root (readlink: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72932 rocksdb:l ceph txt unknown /proc/72189/task/72932/exe (readlink: Permission denied) 2024-03-30T00:37:28.962 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72932 rocksdb:l ceph NOFD /proc/72189/task/72932/fd (opendir: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72933 rocksdb:l ceph cwd unknown /proc/72189/task/72933/cwd (readlink: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72933 rocksdb:l ceph rtd unknown /proc/72189/task/72933/root (readlink: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72933 rocksdb:l ceph txt unknown /proc/72189/task/72933/exe (readlink: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72933 rocksdb:l ceph NOFD /proc/72189/task/72933/fd (opendir: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72934 rocksdb:l ceph cwd unknown /proc/72189/task/72934/cwd (readlink: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72934 rocksdb:l ceph rtd unknown /proc/72189/task/72934/root (readlink: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72934 rocksdb:l ceph txt unknown /proc/72189/task/72934/exe (readlink: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72934 rocksdb:l ceph NOFD /proc/72189/task/72934/fd (opendir: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72935 rocksdb:h ceph cwd unknown /proc/72189/task/72935/cwd (readlink: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72935 rocksdb:h ceph rtd unknown /proc/72189/task/72935/root (readlink: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72935 rocksdb:h ceph txt unknown /proc/72189/task/72935/exe (readlink: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 72935 rocksdb:h ceph NOFD /proc/72189/task/72935/fd (opendir: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73486 bstore_ai ceph cwd unknown /proc/72189/task/73486/cwd (readlink: Permission denied) 2024-03-30T00:37:28.963 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73486 bstore_ai ceph rtd unknown /proc/72189/task/73486/root (readlink: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73486 bstore_ai ceph txt unknown /proc/72189/task/73486/exe (readlink: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73486 bstore_ai ceph NOFD /proc/72189/task/73486/fd (opendir: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73487 bstore_di ceph cwd unknown /proc/72189/task/73487/cwd (readlink: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73487 bstore_di ceph rtd unknown /proc/72189/task/73487/root (readlink: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73487 bstore_di ceph txt unknown /proc/72189/task/73487/exe (readlink: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73487 bstore_di ceph NOFD /proc/72189/task/73487/fd (opendir: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73693 bstore_ai ceph cwd unknown /proc/72189/task/73693/cwd (readlink: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73693 bstore_ai ceph rtd unknown /proc/72189/task/73693/root (readlink: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73693 bstore_ai ceph txt unknown /proc/72189/task/73693/exe (readlink: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73693 bstore_ai ceph NOFD /proc/72189/task/73693/fd (opendir: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73694 bstore_di ceph cwd unknown /proc/72189/task/73694/cwd (readlink: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73694 bstore_di ceph rtd unknown /proc/72189/task/73694/root (readlink: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73694 bstore_di ceph txt unknown /proc/72189/task/73694/exe (readlink: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73694 bstore_di ceph NOFD /proc/72189/task/73694/fd (opendir: Permission denied) 2024-03-30T00:37:28.964 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73878 ceph-osd ceph cwd unknown /proc/72189/task/73878/cwd (readlink: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73878 ceph-osd ceph rtd unknown /proc/72189/task/73878/root (readlink: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73878 ceph-osd ceph txt unknown /proc/72189/task/73878/exe (readlink: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73878 ceph-osd ceph NOFD /proc/72189/task/73878/fd (opendir: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73879 cfin ceph cwd unknown /proc/72189/task/73879/cwd (readlink: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73879 cfin ceph rtd unknown /proc/72189/task/73879/root (readlink: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73879 cfin ceph txt unknown /proc/72189/task/73879/exe (readlink: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73879 cfin ceph NOFD /proc/72189/task/73879/fd (opendir: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73880 bstore_kv ceph cwd unknown /proc/72189/task/73880/cwd (readlink: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73880 bstore_kv ceph rtd unknown /proc/72189/task/73880/root (readlink: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73880 bstore_kv ceph txt unknown /proc/72189/task/73880/exe (readlink: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73880 bstore_kv ceph NOFD /proc/72189/task/73880/fd (opendir: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73881 bstore_kv ceph cwd unknown /proc/72189/task/73881/cwd (readlink: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73881 bstore_kv ceph rtd unknown /proc/72189/task/73881/root (readlink: Permission denied) 2024-03-30T00:37:28.965 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73881 bstore_kv ceph txt unknown /proc/72189/task/73881/exe (readlink: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73881 bstore_kv ceph NOFD /proc/72189/task/73881/fd (opendir: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73882 bstore_me ceph cwd unknown /proc/72189/task/73882/cwd (readlink: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73882 bstore_me ceph rtd unknown /proc/72189/task/73882/root (readlink: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73882 bstore_me ceph txt unknown /proc/72189/task/73882/exe (readlink: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73882 bstore_me ceph NOFD /proc/72189/task/73882/fd (opendir: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73885 ms_dispat ceph cwd unknown /proc/72189/task/73885/cwd (readlink: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73885 ms_dispat ceph rtd unknown /proc/72189/task/73885/root (readlink: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73885 ms_dispat ceph txt unknown /proc/72189/task/73885/exe (readlink: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73885 ms_dispat ceph NOFD /proc/72189/task/73885/fd (opendir: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73886 ms_local ceph cwd unknown /proc/72189/task/73886/cwd (readlink: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73886 ms_local ceph rtd unknown /proc/72189/task/73886/root (readlink: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73886 ms_local ceph txt unknown /proc/72189/task/73886/exe (readlink: Permission denied) 2024-03-30T00:37:28.966 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73886 ms_local ceph NOFD /proc/72189/task/73886/fd (opendir: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73887 safe_time ceph cwd unknown /proc/72189/task/73887/cwd (readlink: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73887 safe_time ceph rtd unknown /proc/72189/task/73887/root (readlink: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73887 safe_time ceph txt unknown /proc/72189/task/73887/exe (readlink: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73887 safe_time ceph NOFD /proc/72189/task/73887/fd (opendir: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73888 safe_time ceph cwd unknown /proc/72189/task/73888/cwd (readlink: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73888 safe_time ceph rtd unknown /proc/72189/task/73888/root (readlink: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73888 safe_time ceph txt unknown /proc/72189/task/73888/exe (readlink: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73888 safe_time ceph NOFD /proc/72189/task/73888/fd (opendir: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73889 ms_dispat ceph cwd unknown /proc/72189/task/73889/cwd (readlink: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73889 ms_dispat ceph rtd unknown /proc/72189/task/73889/root (readlink: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73889 ms_dispat ceph txt unknown /proc/72189/task/73889/exe (readlink: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73889 ms_dispat ceph NOFD /proc/72189/task/73889/fd (opendir: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73890 ms_local ceph cwd unknown /proc/72189/task/73890/cwd (readlink: Permission denied) 2024-03-30T00:37:28.967 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73890 ms_local ceph rtd unknown /proc/72189/task/73890/root (readlink: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73890 ms_local ceph txt unknown /proc/72189/task/73890/exe (readlink: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73890 ms_local ceph NOFD /proc/72189/task/73890/fd (opendir: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73891 ms_dispat ceph cwd unknown /proc/72189/task/73891/cwd (readlink: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73891 ms_dispat ceph rtd unknown /proc/72189/task/73891/root (readlink: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73891 ms_dispat ceph txt unknown /proc/72189/task/73891/exe (readlink: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73891 ms_dispat ceph NOFD /proc/72189/task/73891/fd (opendir: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73892 ms_local ceph cwd unknown /proc/72189/task/73892/cwd (readlink: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73892 ms_local ceph rtd unknown /proc/72189/task/73892/root (readlink: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73892 ms_local ceph txt unknown /proc/72189/task/73892/exe (readlink: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73892 ms_local ceph NOFD /proc/72189/task/73892/fd (opendir: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73893 ms_dispat ceph cwd unknown /proc/72189/task/73893/cwd (readlink: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73893 ms_dispat ceph rtd unknown /proc/72189/task/73893/root (readlink: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73893 ms_dispat ceph txt unknown /proc/72189/task/73893/exe (readlink: Permission denied) 2024-03-30T00:37:28.968 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73893 ms_dispat ceph NOFD /proc/72189/task/73893/fd (opendir: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73894 ms_local ceph cwd unknown /proc/72189/task/73894/cwd (readlink: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73894 ms_local ceph rtd unknown /proc/72189/task/73894/root (readlink: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73894 ms_local ceph txt unknown /proc/72189/task/73894/exe (readlink: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73894 ms_local ceph NOFD /proc/72189/task/73894/fd (opendir: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73895 ms_dispat ceph cwd unknown /proc/72189/task/73895/cwd (readlink: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73895 ms_dispat ceph rtd unknown /proc/72189/task/73895/root (readlink: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73895 ms_dispat ceph txt unknown /proc/72189/task/73895/exe (readlink: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73895 ms_dispat ceph NOFD /proc/72189/task/73895/fd (opendir: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73896 ms_local ceph cwd unknown /proc/72189/task/73896/cwd (readlink: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73896 ms_local ceph rtd unknown /proc/72189/task/73896/root (readlink: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73896 ms_local ceph txt unknown /proc/72189/task/73896/exe (readlink: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73896 ms_local ceph NOFD /proc/72189/task/73896/fd (opendir: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73897 ms_dispat ceph cwd unknown /proc/72189/task/73897/cwd (readlink: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73897 ms_dispat ceph rtd unknown /proc/72189/task/73897/root (readlink: Permission denied) 2024-03-30T00:37:28.969 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73897 ms_dispat ceph txt unknown /proc/72189/task/73897/exe (readlink: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73897 ms_dispat ceph NOFD /proc/72189/task/73897/fd (opendir: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73898 ms_local ceph cwd unknown /proc/72189/task/73898/cwd (readlink: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73898 ms_local ceph rtd unknown /proc/72189/task/73898/root (readlink: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73898 ms_local ceph txt unknown /proc/72189/task/73898/exe (readlink: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73898 ms_local ceph NOFD /proc/72189/task/73898/fd (opendir: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73899 ms_dispat ceph cwd unknown /proc/72189/task/73899/cwd (readlink: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73899 ms_dispat ceph rtd unknown /proc/72189/task/73899/root (readlink: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73899 ms_dispat ceph txt unknown /proc/72189/task/73899/exe (readlink: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73899 ms_dispat ceph NOFD /proc/72189/task/73899/fd (opendir: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73900 ms_local ceph cwd unknown /proc/72189/task/73900/cwd (readlink: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73900 ms_local ceph rtd unknown /proc/72189/task/73900/root (readlink: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73900 ms_local ceph txt unknown /proc/72189/task/73900/exe (readlink: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73900 ms_local ceph NOFD /proc/72189/task/73900/fd (opendir: Permission denied) 2024-03-30T00:37:28.970 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73901 fn_anonym ceph cwd unknown /proc/72189/task/73901/cwd (readlink: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73901 fn_anonym ceph rtd unknown /proc/72189/task/73901/root (readlink: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73901 fn_anonym ceph txt unknown /proc/72189/task/73901/exe (readlink: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73901 fn_anonym ceph NOFD /proc/72189/task/73901/fd (opendir: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73902 finisher ceph cwd unknown /proc/72189/task/73902/cwd (readlink: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73902 finisher ceph rtd unknown /proc/72189/task/73902/root (readlink: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73902 finisher ceph txt unknown /proc/72189/task/73902/exe (readlink: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73902 finisher ceph NOFD /proc/72189/task/73902/fd (opendir: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73903 safe_time ceph cwd unknown /proc/72189/task/73903/cwd (readlink: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73903 safe_time ceph rtd unknown /proc/72189/task/73903/root (readlink: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73903 safe_time ceph txt unknown /proc/72189/task/73903/exe (readlink: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73903 safe_time ceph NOFD /proc/72189/task/73903/fd (opendir: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73904 safe_time ceph cwd unknown /proc/72189/task/73904/cwd (readlink: Permission denied) 2024-03-30T00:37:28.971 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73904 safe_time ceph rtd unknown /proc/72189/task/73904/root (readlink: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73904 safe_time ceph txt unknown /proc/72189/task/73904/exe (readlink: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73904 safe_time ceph NOFD /proc/72189/task/73904/fd (opendir: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73905 ceph-osd ceph cwd unknown /proc/72189/task/73905/cwd (readlink: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73905 ceph-osd ceph rtd unknown /proc/72189/task/73905/root (readlink: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73905 ceph-osd ceph txt unknown /proc/72189/task/73905/exe (readlink: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73905 ceph-osd ceph NOFD /proc/72189/task/73905/fd (opendir: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73906 osd_srv_a ceph cwd unknown /proc/72189/task/73906/cwd (readlink: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73906 osd_srv_a ceph rtd unknown /proc/72189/task/73906/root (readlink: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73906 osd_srv_a ceph txt unknown /proc/72189/task/73906/exe (readlink: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73906 osd_srv_a ceph NOFD /proc/72189/task/73906/fd (opendir: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73907 tp_osd_tp ceph cwd unknown /proc/72189/task/73907/cwd (readlink: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73907 tp_osd_tp ceph rtd unknown /proc/72189/task/73907/root (readlink: Permission denied) 2024-03-30T00:37:28.972 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73907 tp_osd_tp ceph txt unknown /proc/72189/task/73907/exe (readlink: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73907 tp_osd_tp ceph NOFD /proc/72189/task/73907/fd (opendir: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73908 tp_osd_tp ceph cwd unknown /proc/72189/task/73908/cwd (readlink: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73908 tp_osd_tp ceph rtd unknown /proc/72189/task/73908/root (readlink: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73908 tp_osd_tp ceph txt unknown /proc/72189/task/73908/exe (readlink: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73908 tp_osd_tp ceph NOFD /proc/72189/task/73908/fd (opendir: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73910 tp_osd_tp ceph cwd unknown /proc/72189/task/73910/cwd (readlink: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73910 tp_osd_tp ceph rtd unknown /proc/72189/task/73910/root (readlink: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73910 tp_osd_tp ceph txt unknown /proc/72189/task/73910/exe (readlink: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73910 tp_osd_tp ceph NOFD /proc/72189/task/73910/fd (opendir: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73911 tp_osd_tp ceph cwd unknown /proc/72189/task/73911/cwd (readlink: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73911 tp_osd_tp ceph rtd unknown /proc/72189/task/73911/root (readlink: Permission denied) 2024-03-30T00:37:28.973 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73911 tp_osd_tp ceph txt unknown /proc/72189/task/73911/exe (readlink: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73911 tp_osd_tp ceph NOFD /proc/72189/task/73911/fd (opendir: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73912 tp_osd_tp ceph cwd unknown /proc/72189/task/73912/cwd (readlink: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73912 tp_osd_tp ceph rtd unknown /proc/72189/task/73912/root (readlink: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73912 tp_osd_tp ceph txt unknown /proc/72189/task/73912/exe (readlink: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73912 tp_osd_tp ceph NOFD /proc/72189/task/73912/fd (opendir: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73913 tp_osd_tp ceph cwd unknown /proc/72189/task/73913/cwd (readlink: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73913 tp_osd_tp ceph rtd unknown /proc/72189/task/73913/root (readlink: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73913 tp_osd_tp ceph txt unknown /proc/72189/task/73913/exe (readlink: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73913 tp_osd_tp ceph NOFD /proc/72189/task/73913/fd (opendir: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73914 tp_osd_tp ceph cwd unknown /proc/72189/task/73914/cwd (readlink: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73914 tp_osd_tp ceph rtd unknown /proc/72189/task/73914/root (readlink: Permission denied) 2024-03-30T00:37:28.974 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73914 tp_osd_tp ceph txt unknown /proc/72189/task/73914/exe (readlink: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73914 tp_osd_tp ceph NOFD /proc/72189/task/73914/fd (opendir: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73915 tp_osd_tp ceph cwd unknown /proc/72189/task/73915/cwd (readlink: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73915 tp_osd_tp ceph rtd unknown /proc/72189/task/73915/root (readlink: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73915 tp_osd_tp ceph txt unknown /proc/72189/task/73915/exe (readlink: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73915 tp_osd_tp ceph NOFD /proc/72189/task/73915/fd (opendir: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73916 tp_osd_tp ceph cwd unknown /proc/72189/task/73916/cwd (readlink: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73916 tp_osd_tp ceph rtd unknown /proc/72189/task/73916/root (readlink: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73916 tp_osd_tp ceph txt unknown /proc/72189/task/73916/exe (readlink: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73916 tp_osd_tp ceph NOFD /proc/72189/task/73916/fd (opendir: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73917 tp_osd_tp ceph cwd unknown /proc/72189/task/73917/cwd (readlink: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73917 tp_osd_tp ceph rtd unknown /proc/72189/task/73917/root (readlink: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73917 tp_osd_tp ceph txt unknown /proc/72189/task/73917/exe (readlink: Permission denied) 2024-03-30T00:37:28.975 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73917 tp_osd_tp ceph NOFD /proc/72189/task/73917/fd (opendir: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73918 tp_osd_tp ceph cwd unknown /proc/72189/task/73918/cwd (readlink: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73918 tp_osd_tp ceph rtd unknown /proc/72189/task/73918/root (readlink: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73918 tp_osd_tp ceph txt unknown /proc/72189/task/73918/exe (readlink: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73918 tp_osd_tp ceph NOFD /proc/72189/task/73918/fd (opendir: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73919 tp_osd_tp ceph cwd unknown /proc/72189/task/73919/cwd (readlink: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73919 tp_osd_tp ceph rtd unknown /proc/72189/task/73919/root (readlink: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73919 tp_osd_tp ceph txt unknown /proc/72189/task/73919/exe (readlink: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73919 tp_osd_tp ceph NOFD /proc/72189/task/73919/fd (opendir: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73920 tp_osd_tp ceph cwd unknown /proc/72189/task/73920/cwd (readlink: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73920 tp_osd_tp ceph rtd unknown /proc/72189/task/73920/root (readlink: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73920 tp_osd_tp ceph txt unknown /proc/72189/task/73920/exe (readlink: Permission denied) 2024-03-30T00:37:28.976 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73920 tp_osd_tp ceph NOFD /proc/72189/task/73920/fd (opendir: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73921 tp_osd_tp ceph cwd unknown /proc/72189/task/73921/cwd (readlink: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73921 tp_osd_tp ceph rtd unknown /proc/72189/task/73921/root (readlink: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73921 tp_osd_tp ceph txt unknown /proc/72189/task/73921/exe (readlink: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73921 tp_osd_tp ceph NOFD /proc/72189/task/73921/fd (opendir: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73922 tp_osd_tp ceph cwd unknown /proc/72189/task/73922/cwd (readlink: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73922 tp_osd_tp ceph rtd unknown /proc/72189/task/73922/root (readlink: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73922 tp_osd_tp ceph txt unknown /proc/72189/task/73922/exe (readlink: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73922 tp_osd_tp ceph NOFD /proc/72189/task/73922/fd (opendir: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73923 tp_osd_tp ceph cwd unknown /proc/72189/task/73923/cwd (readlink: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73923 tp_osd_tp ceph rtd unknown /proc/72189/task/73923/root (readlink: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73923 tp_osd_tp ceph txt unknown /proc/72189/task/73923/exe (readlink: Permission denied) 2024-03-30T00:37:28.977 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73923 tp_osd_tp ceph NOFD /proc/72189/task/73923/fd (opendir: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73924 osd_srv_h ceph cwd unknown /proc/72189/task/73924/cwd (readlink: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73924 osd_srv_h ceph rtd unknown /proc/72189/task/73924/root (readlink: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73924 osd_srv_h ceph txt unknown /proc/72189/task/73924/exe (readlink: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 72189 73924 osd_srv_h ceph NOFD /proc/72189/task/73924/fd (opendir: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 73195 root cwd unknown /proc/73195/cwd (readlink: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 73195 root rtd unknown /proc/73195/root (readlink: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 73195 root txt unknown /proc/73195/exe (readlink: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 73195 root NOFD /proc/73195/fd (opendir: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 73393 root cwd unknown /proc/73393/cwd (readlink: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 73393 root rtd unknown /proc/73393/root (readlink: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 73393 root txt unknown /proc/73393/exe (readlink: Permission denied) 2024-03-30T00:37:28.978 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 73393 root NOFD /proc/73393/fd (opendir: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 73428 root cwd unknown /proc/73428/cwd (readlink: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 73428 root rtd unknown /proc/73428/root (readlink: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 73428 root txt unknown /proc/73428/exe (readlink: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 73428 root NOFD /proc/73428/fd (opendir: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 79545 root cwd unknown /proc/79545/cwd (readlink: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 79545 root rtd unknown /proc/79545/root (readlink: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 79545 root txt unknown /proc/79545/exe (readlink: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/7 79545 root NOFD /proc/79545/fd (opendir: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 80301 root cwd unknown /proc/80301/cwd (readlink: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 80301 root rtd unknown /proc/80301/root (readlink: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 80301 root txt unknown /proc/80301/exe (readlink: Permission denied) 2024-03-30T00:37:28.979 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 80301 root NOFD /proc/80301/fd (opendir: Permission denied) 2024-03-30T00:37:28.980 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 80312 root cwd unknown /proc/80312/cwd (readlink: Permission denied) 2024-03-30T00:37:28.980 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 80312 root rtd unknown /proc/80312/root (readlink: Permission denied) 2024-03-30T00:37:28.980 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 80312 root txt unknown /proc/80312/exe (readlink: Permission denied) 2024-03-30T00:37:28.980 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 80312 root NOFD /proc/80312/fd (opendir: Permission denied) 2024-03-30T00:37:28.980 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 ceph cwd unknown /proc/80324/cwd (readlink: Permission denied) 2024-03-30T00:37:28.980 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 ceph rtd unknown /proc/80324/root (readlink: Permission denied) 2024-03-30T00:37:28.980 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 ceph txt unknown /proc/80324/exe (readlink: Permission denied) 2024-03-30T00:37:28.980 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 ceph NOFD /proc/80324/fd (opendir: Permission denied) 2024-03-30T00:37:28.980 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80325 log ceph cwd unknown /proc/80324/task/80325/cwd (readlink: Permission denied) 2024-03-30T00:37:28.980 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80325 log ceph rtd unknown /proc/80324/task/80325/root (readlink: Permission denied) 2024-03-30T00:37:28.980 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80325 log ceph txt unknown /proc/80324/task/80325/exe (readlink: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80325 log ceph NOFD /proc/80324/task/80325/fd (opendir: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80327 msgr-work ceph cwd unknown /proc/80324/task/80327/cwd (readlink: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80327 msgr-work ceph rtd unknown /proc/80324/task/80327/root (readlink: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80327 msgr-work ceph txt unknown /proc/80324/task/80327/exe (readlink: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80327 msgr-work ceph NOFD /proc/80324/task/80327/fd (opendir: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80328 msgr-work ceph cwd unknown /proc/80324/task/80328/cwd (readlink: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80328 msgr-work ceph rtd unknown /proc/80324/task/80328/root (readlink: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80328 msgr-work ceph txt unknown /proc/80324/task/80328/exe (readlink: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80328 msgr-work ceph NOFD /proc/80324/task/80328/fd (opendir: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80329 msgr-work ceph cwd unknown /proc/80324/task/80329/cwd (readlink: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80329 msgr-work ceph rtd unknown /proc/80324/task/80329/root (readlink: Permission denied) 2024-03-30T00:37:28.981 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80329 msgr-work ceph txt unknown /proc/80324/task/80329/exe (readlink: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80329 msgr-work ceph NOFD /proc/80324/task/80329/fd (opendir: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80335 service ceph cwd unknown /proc/80324/task/80335/cwd (readlink: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80335 service ceph rtd unknown /proc/80324/task/80335/root (readlink: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80335 service ceph txt unknown /proc/80324/task/80335/exe (readlink: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80335 service ceph NOFD /proc/80324/task/80335/fd (opendir: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80336 admin_soc ceph cwd unknown /proc/80324/task/80336/cwd (readlink: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80336 admin_soc ceph rtd unknown /proc/80324/task/80336/root (readlink: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80336 admin_soc ceph txt unknown /proc/80324/task/80336/exe (readlink: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80336 admin_soc ceph NOFD /proc/80324/task/80336/fd (opendir: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80561 signal_ha ceph cwd unknown /proc/80324/task/80561/cwd (readlink: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80561 signal_ha ceph rtd unknown /proc/80324/task/80561/root (readlink: Permission denied) 2024-03-30T00:37:28.982 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80561 signal_ha ceph txt unknown /proc/80324/task/80561/exe (readlink: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80561 signal_ha ceph NOFD /proc/80324/task/80561/fd (opendir: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80562 io_contex ceph cwd unknown /proc/80324/task/80562/cwd (readlink: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80562 io_contex ceph rtd unknown /proc/80324/task/80562/root (readlink: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80562 io_contex ceph txt unknown /proc/80324/task/80562/exe (readlink: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80562 io_contex ceph NOFD /proc/80324/task/80562/fd (opendir: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80563 io_contex ceph cwd unknown /proc/80324/task/80563/cwd (readlink: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80563 io_contex ceph rtd unknown /proc/80324/task/80563/root (readlink: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80563 io_contex ceph txt unknown /proc/80324/task/80563/exe (readlink: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80563 io_contex ceph NOFD /proc/80324/task/80563/fd (opendir: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80640 OpHistory ceph cwd unknown /proc/80324/task/80640/cwd (readlink: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80640 OpHistory ceph rtd unknown /proc/80324/task/80640/root (readlink: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80640 OpHistory ceph txt unknown /proc/80324/task/80640/exe (readlink: Permission denied) 2024-03-30T00:37:28.983 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80640 OpHistory ceph NOFD /proc/80324/task/80640/fd (opendir: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80641 ceph_time ceph cwd unknown /proc/80324/task/80641/cwd (readlink: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80641 ceph_time ceph rtd unknown /proc/80324/task/80641/root (readlink: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80641 ceph_time ceph txt unknown /proc/80324/task/80641/exe (readlink: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 80641 ceph_time ceph NOFD /proc/80324/task/80641/fd (opendir: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81055 ceph-osd ceph cwd unknown /proc/80324/task/81055/cwd (readlink: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81055 ceph-osd ceph rtd unknown /proc/80324/task/81055/root (readlink: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81055 ceph-osd ceph txt unknown /proc/80324/task/81055/exe (readlink: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81055 ceph-osd ceph NOFD /proc/80324/task/81055/fd (opendir: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81056 safe_time ceph cwd unknown /proc/80324/task/81056/cwd (readlink: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81056 safe_time ceph rtd unknown /proc/80324/task/81056/root (readlink: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81056 safe_time ceph txt unknown /proc/80324/task/81056/exe (readlink: Permission denied) 2024-03-30T00:37:28.984 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81056 safe_time ceph NOFD /proc/80324/task/81056/fd (opendir: Permission denied) 2024-03-30T00:37:28.985 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81057 safe_time ceph cwd unknown /proc/80324/task/81057/cwd (readlink: Permission denied) 2024-03-30T00:37:28.985 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81057 safe_time ceph rtd unknown /proc/80324/task/81057/root (readlink: Permission denied) 2024-03-30T00:37:28.985 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81057 safe_time ceph txt unknown /proc/80324/task/81057/exe (readlink: Permission denied) 2024-03-30T00:37:28.985 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81057 safe_time ceph NOFD /proc/80324/task/81057/fd (opendir: Permission denied) 2024-03-30T00:37:28.985 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81058 safe_time ceph cwd unknown /proc/80324/task/81058/cwd (readlink: Permission denied) 2024-03-30T00:37:28.985 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81058 safe_time ceph rtd unknown /proc/80324/task/81058/root (readlink: Permission denied) 2024-03-30T00:37:28.985 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81058 safe_time ceph txt unknown /proc/80324/task/81058/exe (readlink: Permission denied) 2024-03-30T00:37:28.985 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81058 safe_time ceph NOFD /proc/80324/task/81058/fd (opendir: Permission denied) 2024-03-30T00:37:28.985 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81059 safe_time ceph cwd unknown /proc/80324/task/81059/cwd (readlink: Permission denied) 2024-03-30T00:37:28.985 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81059 safe_time ceph rtd unknown /proc/80324/task/81059/root (readlink: Permission denied) 2024-03-30T00:37:28.985 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81059 safe_time ceph txt unknown /proc/80324/task/81059/exe (readlink: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81059 safe_time ceph NOFD /proc/80324/task/81059/fd (opendir: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81060 fn_anonym ceph cwd unknown /proc/80324/task/81060/cwd (readlink: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81060 fn_anonym ceph rtd unknown /proc/80324/task/81060/root (readlink: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81060 fn_anonym ceph txt unknown /proc/80324/task/81060/exe (readlink: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81060 fn_anonym ceph NOFD /proc/80324/task/81060/fd (opendir: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81067 rocksdb:l ceph cwd unknown /proc/80324/task/81067/cwd (readlink: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81067 rocksdb:l ceph rtd unknown /proc/80324/task/81067/root (readlink: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81067 rocksdb:l ceph txt unknown /proc/80324/task/81067/exe (readlink: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81067 rocksdb:l ceph NOFD /proc/80324/task/81067/fd (opendir: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81068 rocksdb:l ceph cwd unknown /proc/80324/task/81068/cwd (readlink: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81068 rocksdb:l ceph rtd unknown /proc/80324/task/81068/root (readlink: Permission denied) 2024-03-30T00:37:28.986 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81068 rocksdb:l ceph txt unknown /proc/80324/task/81068/exe (readlink: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81068 rocksdb:l ceph NOFD /proc/80324/task/81068/fd (opendir: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81069 rocksdb:l ceph cwd unknown /proc/80324/task/81069/cwd (readlink: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81069 rocksdb:l ceph rtd unknown /proc/80324/task/81069/root (readlink: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81069 rocksdb:l ceph txt unknown /proc/80324/task/81069/exe (readlink: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81069 rocksdb:l ceph NOFD /proc/80324/task/81069/fd (opendir: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81070 rocksdb:h ceph cwd unknown /proc/80324/task/81070/cwd (readlink: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81070 rocksdb:h ceph rtd unknown /proc/80324/task/81070/root (readlink: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81070 rocksdb:h ceph txt unknown /proc/80324/task/81070/exe (readlink: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81070 rocksdb:h ceph NOFD /proc/80324/task/81070/fd (opendir: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81605 bstore_ai ceph cwd unknown /proc/80324/task/81605/cwd (readlink: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81605 bstore_ai ceph rtd unknown /proc/80324/task/81605/root (readlink: Permission denied) 2024-03-30T00:37:28.987 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81605 bstore_ai ceph txt unknown /proc/80324/task/81605/exe (readlink: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81605 bstore_ai ceph NOFD /proc/80324/task/81605/fd (opendir: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81607 bstore_di ceph cwd unknown /proc/80324/task/81607/cwd (readlink: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81607 bstore_di ceph rtd unknown /proc/80324/task/81607/root (readlink: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81607 bstore_di ceph txt unknown /proc/80324/task/81607/exe (readlink: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81607 bstore_di ceph NOFD /proc/80324/task/81607/fd (opendir: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81830 bstore_ai ceph cwd unknown /proc/80324/task/81830/cwd (readlink: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81830 bstore_ai ceph rtd unknown /proc/80324/task/81830/root (readlink: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81830 bstore_ai ceph txt unknown /proc/80324/task/81830/exe (readlink: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81830 bstore_ai ceph NOFD /proc/80324/task/81830/fd (opendir: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81831 bstore_di ceph cwd unknown /proc/80324/task/81831/cwd (readlink: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81831 bstore_di ceph rtd unknown /proc/80324/task/81831/root (readlink: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81831 bstore_di ceph txt unknown /proc/80324/task/81831/exe (readlink: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 81831 bstore_di ceph NOFD /proc/80324/task/81831/fd (opendir: Permission denied) 2024-03-30T00:37:28.988 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82015 ceph-osd ceph cwd unknown /proc/80324/task/82015/cwd (readlink: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82015 ceph-osd ceph rtd unknown /proc/80324/task/82015/root (readlink: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82015 ceph-osd ceph txt unknown /proc/80324/task/82015/exe (readlink: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82015 ceph-osd ceph NOFD /proc/80324/task/82015/fd (opendir: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82017 cfin ceph cwd unknown /proc/80324/task/82017/cwd (readlink: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82017 cfin ceph rtd unknown /proc/80324/task/82017/root (readlink: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82017 cfin ceph txt unknown /proc/80324/task/82017/exe (readlink: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82017 cfin ceph NOFD /proc/80324/task/82017/fd (opendir: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82018 bstore_kv ceph cwd unknown /proc/80324/task/82018/cwd (readlink: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82018 bstore_kv ceph rtd unknown /proc/80324/task/82018/root (readlink: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82018 bstore_kv ceph txt unknown /proc/80324/task/82018/exe (readlink: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82018 bstore_kv ceph NOFD /proc/80324/task/82018/fd (opendir: Permission denied) 2024-03-30T00:37:28.989 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82019 bstore_kv ceph cwd unknown /proc/80324/task/82019/cwd (readlink: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82019 bstore_kv ceph rtd unknown /proc/80324/task/82019/root (readlink: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82019 bstore_kv ceph txt unknown /proc/80324/task/82019/exe (readlink: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82019 bstore_kv ceph NOFD /proc/80324/task/82019/fd (opendir: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82021 bstore_me ceph cwd unknown /proc/80324/task/82021/cwd (readlink: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82021 bstore_me ceph rtd unknown /proc/80324/task/82021/root (readlink: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82021 bstore_me ceph txt unknown /proc/80324/task/82021/exe (readlink: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82021 bstore_me ceph NOFD /proc/80324/task/82021/fd (opendir: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82028 ms_dispat ceph cwd unknown /proc/80324/task/82028/cwd (readlink: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82028 ms_dispat ceph rtd unknown /proc/80324/task/82028/root (readlink: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82028 ms_dispat ceph txt unknown /proc/80324/task/82028/exe (readlink: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82028 ms_dispat ceph NOFD /proc/80324/task/82028/fd (opendir: Permission denied) 2024-03-30T00:37:28.990 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82029 ms_local ceph cwd unknown /proc/80324/task/82029/cwd (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82029 ms_local ceph rtd unknown /proc/80324/task/82029/root (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82029 ms_local ceph txt unknown /proc/80324/task/82029/exe (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82029 ms_local ceph NOFD /proc/80324/task/82029/fd (opendir: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82030 safe_time ceph cwd unknown /proc/80324/task/82030/cwd (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82030 safe_time ceph rtd unknown /proc/80324/task/82030/root (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82030 safe_time ceph txt unknown /proc/80324/task/82030/exe (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82030 safe_time ceph NOFD /proc/80324/task/82030/fd (opendir: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82031 safe_time ceph cwd unknown /proc/80324/task/82031/cwd (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82031 safe_time ceph rtd unknown /proc/80324/task/82031/root (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82031 safe_time ceph txt unknown /proc/80324/task/82031/exe (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82031 safe_time ceph NOFD /proc/80324/task/82031/fd (opendir: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82032 ms_dispat ceph cwd unknown /proc/80324/task/82032/cwd (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82032 ms_dispat ceph rtd unknown /proc/80324/task/82032/root (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82032 ms_dispat ceph txt unknown /proc/80324/task/82032/exe (readlink: Permission denied) 2024-03-30T00:37:28.991 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82032 ms_dispat ceph NOFD /proc/80324/task/82032/fd (opendir: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82033 ms_local ceph cwd unknown /proc/80324/task/82033/cwd (readlink: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82033 ms_local ceph rtd unknown /proc/80324/task/82033/root (readlink: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82033 ms_local ceph txt unknown /proc/80324/task/82033/exe (readlink: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82033 ms_local ceph NOFD /proc/80324/task/82033/fd (opendir: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82034 ms_dispat ceph cwd unknown /proc/80324/task/82034/cwd (readlink: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82034 ms_dispat ceph rtd unknown /proc/80324/task/82034/root (readlink: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82034 ms_dispat ceph txt unknown /proc/80324/task/82034/exe (readlink: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82034 ms_dispat ceph NOFD /proc/80324/task/82034/fd (opendir: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82035 ms_local ceph cwd unknown /proc/80324/task/82035/cwd (readlink: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82035 ms_local ceph rtd unknown /proc/80324/task/82035/root (readlink: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82035 ms_local ceph txt unknown /proc/80324/task/82035/exe (readlink: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82035 ms_local ceph NOFD /proc/80324/task/82035/fd (opendir: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82036 ms_dispat ceph cwd unknown /proc/80324/task/82036/cwd (readlink: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82036 ms_dispat ceph rtd unknown /proc/80324/task/82036/root (readlink: Permission denied) 2024-03-30T00:37:28.992 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82036 ms_dispat ceph txt unknown /proc/80324/task/82036/exe (readlink: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82036 ms_dispat ceph NOFD /proc/80324/task/82036/fd (opendir: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82037 ms_local ceph cwd unknown /proc/80324/task/82037/cwd (readlink: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82037 ms_local ceph rtd unknown /proc/80324/task/82037/root (readlink: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82037 ms_local ceph txt unknown /proc/80324/task/82037/exe (readlink: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82037 ms_local ceph NOFD /proc/80324/task/82037/fd (opendir: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82038 ms_dispat ceph cwd unknown /proc/80324/task/82038/cwd (readlink: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82038 ms_dispat ceph rtd unknown /proc/80324/task/82038/root (readlink: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82038 ms_dispat ceph txt unknown /proc/80324/task/82038/exe (readlink: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82038 ms_dispat ceph NOFD /proc/80324/task/82038/fd (opendir: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82039 ms_local ceph cwd unknown /proc/80324/task/82039/cwd (readlink: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82039 ms_local ceph rtd unknown /proc/80324/task/82039/root (readlink: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82039 ms_local ceph txt unknown /proc/80324/task/82039/exe (readlink: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82039 ms_local ceph NOFD /proc/80324/task/82039/fd (opendir: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82040 ms_dispat ceph cwd unknown /proc/80324/task/82040/cwd (readlink: Permission denied) 2024-03-30T00:37:28.993 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82040 ms_dispat ceph rtd unknown /proc/80324/task/82040/root (readlink: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82040 ms_dispat ceph txt unknown /proc/80324/task/82040/exe (readlink: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82040 ms_dispat ceph NOFD /proc/80324/task/82040/fd (opendir: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82041 ms_local ceph cwd unknown /proc/80324/task/82041/cwd (readlink: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82041 ms_local ceph rtd unknown /proc/80324/task/82041/root (readlink: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82041 ms_local ceph txt unknown /proc/80324/task/82041/exe (readlink: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82041 ms_local ceph NOFD /proc/80324/task/82041/fd (opendir: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82042 ms_dispat ceph cwd unknown /proc/80324/task/82042/cwd (readlink: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82042 ms_dispat ceph rtd unknown /proc/80324/task/82042/root (readlink: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82042 ms_dispat ceph txt unknown /proc/80324/task/82042/exe (readlink: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82042 ms_dispat ceph NOFD /proc/80324/task/82042/fd (opendir: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82043 ms_local ceph cwd unknown /proc/80324/task/82043/cwd (readlink: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82043 ms_local ceph rtd unknown /proc/80324/task/82043/root (readlink: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82043 ms_local ceph txt unknown /proc/80324/task/82043/exe (readlink: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82043 ms_local ceph NOFD /proc/80324/task/82043/fd (opendir: Permission denied) 2024-03-30T00:37:28.994 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82044 fn_anonym ceph cwd unknown /proc/80324/task/82044/cwd (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82044 fn_anonym ceph rtd unknown /proc/80324/task/82044/root (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82044 fn_anonym ceph txt unknown /proc/80324/task/82044/exe (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82044 fn_anonym ceph NOFD /proc/80324/task/82044/fd (opendir: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82045 finisher ceph cwd unknown /proc/80324/task/82045/cwd (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82045 finisher ceph rtd unknown /proc/80324/task/82045/root (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82045 finisher ceph txt unknown /proc/80324/task/82045/exe (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82045 finisher ceph NOFD /proc/80324/task/82045/fd (opendir: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82046 safe_time ceph cwd unknown /proc/80324/task/82046/cwd (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82046 safe_time ceph rtd unknown /proc/80324/task/82046/root (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82046 safe_time ceph txt unknown /proc/80324/task/82046/exe (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82046 safe_time ceph NOFD /proc/80324/task/82046/fd (opendir: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82047 safe_time ceph cwd unknown /proc/80324/task/82047/cwd (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82047 safe_time ceph rtd unknown /proc/80324/task/82047/root (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82047 safe_time ceph txt unknown /proc/80324/task/82047/exe (readlink: Permission denied) 2024-03-30T00:37:28.995 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82047 safe_time ceph NOFD /proc/80324/task/82047/fd (opendir: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82048 ceph-osd ceph cwd unknown /proc/80324/task/82048/cwd (readlink: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82048 ceph-osd ceph rtd unknown /proc/80324/task/82048/root (readlink: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82048 ceph-osd ceph txt unknown /proc/80324/task/82048/exe (readlink: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82048 ceph-osd ceph NOFD /proc/80324/task/82048/fd (opendir: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82049 osd_srv_a ceph cwd unknown /proc/80324/task/82049/cwd (readlink: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82049 osd_srv_a ceph rtd unknown /proc/80324/task/82049/root (readlink: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82049 osd_srv_a ceph txt unknown /proc/80324/task/82049/exe (readlink: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82049 osd_srv_a ceph NOFD /proc/80324/task/82049/fd (opendir: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82050 tp_osd_tp ceph cwd unknown /proc/80324/task/82050/cwd (readlink: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82050 tp_osd_tp ceph rtd unknown /proc/80324/task/82050/root (readlink: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82050 tp_osd_tp ceph txt unknown /proc/80324/task/82050/exe (readlink: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82050 tp_osd_tp ceph NOFD /proc/80324/task/82050/fd (opendir: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82051 tp_osd_tp ceph cwd unknown /proc/80324/task/82051/cwd (readlink: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82051 tp_osd_tp ceph rtd unknown /proc/80324/task/82051/root (readlink: Permission denied) 2024-03-30T00:37:28.996 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82051 tp_osd_tp ceph txt unknown /proc/80324/task/82051/exe (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82051 tp_osd_tp ceph NOFD /proc/80324/task/82051/fd (opendir: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82052 tp_osd_tp ceph cwd unknown /proc/80324/task/82052/cwd (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82052 tp_osd_tp ceph rtd unknown /proc/80324/task/82052/root (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82052 tp_osd_tp ceph txt unknown /proc/80324/task/82052/exe (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82052 tp_osd_tp ceph NOFD /proc/80324/task/82052/fd (opendir: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82053 tp_osd_tp ceph cwd unknown /proc/80324/task/82053/cwd (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82053 tp_osd_tp ceph rtd unknown /proc/80324/task/82053/root (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82053 tp_osd_tp ceph txt unknown /proc/80324/task/82053/exe (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82053 tp_osd_tp ceph NOFD /proc/80324/task/82053/fd (opendir: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82054 tp_osd_tp ceph cwd unknown /proc/80324/task/82054/cwd (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82054 tp_osd_tp ceph rtd unknown /proc/80324/task/82054/root (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82054 tp_osd_tp ceph txt unknown /proc/80324/task/82054/exe (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82054 tp_osd_tp ceph NOFD /proc/80324/task/82054/fd (opendir: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82055 tp_osd_tp ceph cwd unknown /proc/80324/task/82055/cwd (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82055 tp_osd_tp ceph rtd unknown /proc/80324/task/82055/root (readlink: Permission denied) 2024-03-30T00:37:28.997 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82055 tp_osd_tp ceph txt unknown /proc/80324/task/82055/exe (readlink: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82055 tp_osd_tp ceph NOFD /proc/80324/task/82055/fd (opendir: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82056 tp_osd_tp ceph cwd unknown /proc/80324/task/82056/cwd (readlink: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82056 tp_osd_tp ceph rtd unknown /proc/80324/task/82056/root (readlink: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82056 tp_osd_tp ceph txt unknown /proc/80324/task/82056/exe (readlink: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82056 tp_osd_tp ceph NOFD /proc/80324/task/82056/fd (opendir: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82057 tp_osd_tp ceph cwd unknown /proc/80324/task/82057/cwd (readlink: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82057 tp_osd_tp ceph rtd unknown /proc/80324/task/82057/root (readlink: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82057 tp_osd_tp ceph txt unknown /proc/80324/task/82057/exe (readlink: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82057 tp_osd_tp ceph NOFD /proc/80324/task/82057/fd (opendir: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82058 tp_osd_tp ceph cwd unknown /proc/80324/task/82058/cwd (readlink: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82058 tp_osd_tp ceph rtd unknown /proc/80324/task/82058/root (readlink: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82058 tp_osd_tp ceph txt unknown /proc/80324/task/82058/exe (readlink: Permission denied) 2024-03-30T00:37:28.998 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82058 tp_osd_tp ceph NOFD /proc/80324/task/82058/fd (opendir: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82059 tp_osd_tp ceph cwd unknown /proc/80324/task/82059/cwd (readlink: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82059 tp_osd_tp ceph rtd unknown /proc/80324/task/82059/root (readlink: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82059 tp_osd_tp ceph txt unknown /proc/80324/task/82059/exe (readlink: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82059 tp_osd_tp ceph NOFD /proc/80324/task/82059/fd (opendir: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82060 tp_osd_tp ceph cwd unknown /proc/80324/task/82060/cwd (readlink: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82060 tp_osd_tp ceph rtd unknown /proc/80324/task/82060/root (readlink: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82060 tp_osd_tp ceph txt unknown /proc/80324/task/82060/exe (readlink: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82060 tp_osd_tp ceph NOFD /proc/80324/task/82060/fd (opendir: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82061 tp_osd_tp ceph cwd unknown /proc/80324/task/82061/cwd (readlink: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82061 tp_osd_tp ceph rtd unknown /proc/80324/task/82061/root (readlink: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82061 tp_osd_tp ceph txt unknown /proc/80324/task/82061/exe (readlink: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82061 tp_osd_tp ceph NOFD /proc/80324/task/82061/fd (opendir: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82062 tp_osd_tp ceph cwd unknown /proc/80324/task/82062/cwd (readlink: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82062 tp_osd_tp ceph rtd unknown /proc/80324/task/82062/root (readlink: Permission denied) 2024-03-30T00:37:28.999 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82062 tp_osd_tp ceph txt unknown /proc/80324/task/82062/exe (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82062 tp_osd_tp ceph NOFD /proc/80324/task/82062/fd (opendir: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82063 tp_osd_tp ceph cwd unknown /proc/80324/task/82063/cwd (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82063 tp_osd_tp ceph rtd unknown /proc/80324/task/82063/root (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82063 tp_osd_tp ceph txt unknown /proc/80324/task/82063/exe (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82063 tp_osd_tp ceph NOFD /proc/80324/task/82063/fd (opendir: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82064 tp_osd_tp ceph cwd unknown /proc/80324/task/82064/cwd (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82064 tp_osd_tp ceph rtd unknown /proc/80324/task/82064/root (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82064 tp_osd_tp ceph txt unknown /proc/80324/task/82064/exe (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82064 tp_osd_tp ceph NOFD /proc/80324/task/82064/fd (opendir: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82065 tp_osd_tp ceph cwd unknown /proc/80324/task/82065/cwd (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82065 tp_osd_tp ceph rtd unknown /proc/80324/task/82065/root (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82065 tp_osd_tp ceph txt unknown /proc/80324/task/82065/exe (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82065 tp_osd_tp ceph NOFD /proc/80324/task/82065/fd (opendir: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82066 osd_srv_h ceph cwd unknown /proc/80324/task/82066/cwd (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82066 osd_srv_h ceph rtd unknown /proc/80324/task/82066/root (readlink: Permission denied) 2024-03-30T00:37:29.000 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82066 osd_srv_h ceph txt unknown /proc/80324/task/82066/exe (readlink: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-osd 80324 82066 osd_srv_h ceph NOFD /proc/80324/task/82066/fd (opendir: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 81523 root cwd unknown /proc/81523/cwd (readlink: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 81523 root rtd unknown /proc/81523/root (readlink: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 81523 root txt unknown /proc/81523/exe (readlink: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 81523 root NOFD /proc/81523/fd (opendir: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 81544 root cwd unknown /proc/81544/cwd (readlink: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 81544 root rtd unknown /proc/81544/root (readlink: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 81544 root txt unknown /proc/81544/exe (readlink: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 81544 root NOFD /proc/81544/fd (opendir: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 85427 root cwd unknown /proc/85427/cwd (readlink: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 85427 root rtd unknown /proc/85427/root (readlink: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 85427 root txt unknown /proc/85427/exe (readlink: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 85427 root NOFD /proc/85427/fd (opendir: Permission denied) 2024-03-30T00:37:29.001 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 85677 root cwd unknown /proc/85677/cwd (readlink: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 85677 root rtd unknown /proc/85677/root (readlink: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 85677 root txt unknown /proc/85677/exe (readlink: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 85677 root NOFD /proc/85677/fd (opendir: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 86114 root cwd unknown /proc/86114/cwd (readlink: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 86114 root rtd unknown /proc/86114/root (readlink: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 86114 root txt unknown /proc/86114/exe (readlink: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/3 86114 root NOFD /proc/86114/fd (opendir: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86136 root cwd unknown /proc/86136/cwd (readlink: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86136 root rtd unknown /proc/86136/root (readlink: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86136 root txt unknown /proc/86136/exe (readlink: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86136 root NOFD /proc/86136/fd (opendir: Permission denied) 2024-03-30T00:37:29.002 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86155 root cwd unknown /proc/86155/cwd (readlink: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86155 root rtd unknown /proc/86155/root (readlink: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86155 root txt unknown /proc/86155/exe (readlink: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86155 root NOFD /proc/86155/fd (opendir: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86167 root cwd unknown /proc/86167/cwd (readlink: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86167 root rtd unknown /proc/86167/root (readlink: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86167 root txt unknown /proc/86167/exe (readlink: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86167 root NOFD /proc/86167/fd (opendir: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86175 root cwd unknown /proc/86175/cwd (readlink: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86175 root rtd unknown /proc/86175/root (readlink: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86175 root txt unknown /proc/86175/exe (readlink: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86175 root NOFD /proc/86175/fd (opendir: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86190 root cwd unknown /proc/86190/cwd (readlink: Permission denied) 2024-03-30T00:37:29.003 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86190 root rtd unknown /proc/86190/root (readlink: Permission denied) 2024-03-30T00:37:29.004 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86190 root txt unknown /proc/86190/exe (readlink: Permission denied) 2024-03-30T00:37:29.004 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86190 root NOFD /proc/86190/fd (opendir: Permission denied) 2024-03-30T00:37:29.004 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86197 root cwd unknown /proc/86197/cwd (readlink: Permission denied) 2024-03-30T00:37:29.004 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86197 root rtd unknown /proc/86197/root (readlink: Permission denied) 2024-03-30T00:37:29.004 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86197 root txt unknown /proc/86197/exe (readlink: Permission denied) 2024-03-30T00:37:29.004 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 86197 root NOFD /proc/86197/fd (opendir: Permission denied) 2024-03-30T00:37:29.004 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86209 root cwd unknown /proc/86209/cwd (readlink: Permission denied) 2024-03-30T00:37:29.004 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86209 root rtd unknown /proc/86209/root (readlink: Permission denied) 2024-03-30T00:37:29.004 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86209 root txt unknown /proc/86209/exe (readlink: Permission denied) 2024-03-30T00:37:29.004 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86209 root NOFD /proc/86209/fd (opendir: Permission denied) 2024-03-30T00:37:29.004 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86219 root cwd unknown /proc/86219/cwd (readlink: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86219 root rtd unknown /proc/86219/root (readlink: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86219 root txt unknown /proc/86219/exe (readlink: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:journalct 86219 root NOFD /proc/86219/fd (opendir: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 86546 root cwd unknown /proc/86546/cwd (readlink: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 86546 root rtd unknown /proc/86546/root (readlink: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 86546 root txt unknown /proc/86546/exe (readlink: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 86546 root NOFD /proc/86546/fd (opendir: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 86547 root cwd unknown /proc/86547/cwd (readlink: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 86547 root rtd unknown /proc/86547/root (readlink: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 86547 root txt unknown /proc/86547/exe (readlink: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 86547 root NOFD /proc/86547/fd (opendir: Permission denied) 2024-03-30T00:37:29.005 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 86741 root cwd unknown /proc/86741/cwd (readlink: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 86741 root rtd unknown /proc/86741/root (readlink: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 86741 root txt unknown /proc/86741/exe (readlink: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 86741 root NOFD /proc/86741/fd (opendir: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 86752 root cwd unknown /proc/86752/cwd (readlink: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 86752 root rtd unknown /proc/86752/root (readlink: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 86752 root txt unknown /proc/86752/exe (readlink: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 86752 root NOFD /proc/86752/fd (opendir: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 ceph cwd unknown /proc/86765/cwd (readlink: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 ceph rtd unknown /proc/86765/root (readlink: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 ceph txt unknown /proc/86765/exe (readlink: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 ceph NOFD /proc/86765/fd (opendir: Permission denied) 2024-03-30T00:37:29.006 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86768 log ceph cwd unknown /proc/86765/task/86768/cwd (readlink: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86768 log ceph rtd unknown /proc/86765/task/86768/root (readlink: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86768 log ceph txt unknown /proc/86765/task/86768/exe (readlink: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86768 log ceph NOFD /proc/86765/task/86768/fd (opendir: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86770 msgr-work ceph cwd unknown /proc/86765/task/86770/cwd (readlink: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86770 msgr-work ceph rtd unknown /proc/86765/task/86770/root (readlink: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86770 msgr-work ceph txt unknown /proc/86765/task/86770/exe (readlink: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86770 msgr-work ceph NOFD /proc/86765/task/86770/fd (opendir: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86771 msgr-work ceph cwd unknown /proc/86765/task/86771/cwd (readlink: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86771 msgr-work ceph rtd unknown /proc/86765/task/86771/root (readlink: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86771 msgr-work ceph txt unknown /proc/86765/task/86771/exe (readlink: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86771 msgr-work ceph NOFD /proc/86765/task/86771/fd (opendir: Permission denied) 2024-03-30T00:37:29.007 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86772 msgr-work ceph cwd unknown /proc/86765/task/86772/cwd (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86772 msgr-work ceph rtd unknown /proc/86765/task/86772/root (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86772 msgr-work ceph txt unknown /proc/86765/task/86772/exe (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86772 msgr-work ceph NOFD /proc/86765/task/86772/fd (opendir: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86776 service ceph cwd unknown /proc/86765/task/86776/cwd (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86776 service ceph rtd unknown /proc/86765/task/86776/root (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86776 service ceph txt unknown /proc/86765/task/86776/exe (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86776 service ceph NOFD /proc/86765/task/86776/fd (opendir: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86777 admin_soc ceph cwd unknown /proc/86765/task/86777/cwd (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86777 admin_soc ceph rtd unknown /proc/86765/task/86777/root (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86777 admin_soc ceph txt unknown /proc/86765/task/86777/exe (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86777 admin_soc ceph NOFD /proc/86765/task/86777/fd (opendir: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86780 signal_ha ceph cwd unknown /proc/86765/task/86780/cwd (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86780 signal_ha ceph rtd unknown /proc/86765/task/86780/root (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86780 signal_ha ceph txt unknown /proc/86765/task/86780/exe (readlink: Permission denied) 2024-03-30T00:37:29.008 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86780 signal_ha ceph NOFD /proc/86765/task/86780/fd (opendir: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86781 io_contex ceph cwd unknown /proc/86765/task/86781/cwd (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86781 io_contex ceph rtd unknown /proc/86765/task/86781/root (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86781 io_contex ceph txt unknown /proc/86765/task/86781/exe (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86781 io_contex ceph NOFD /proc/86765/task/86781/fd (opendir: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86782 io_contex ceph cwd unknown /proc/86765/task/86782/cwd (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86782 io_contex ceph rtd unknown /proc/86765/task/86782/root (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86782 io_contex ceph txt unknown /proc/86765/task/86782/exe (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86782 io_contex ceph NOFD /proc/86765/task/86782/fd (opendir: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86783 ms_dispat ceph cwd unknown /proc/86765/task/86783/cwd (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86783 ms_dispat ceph rtd unknown /proc/86765/task/86783/root (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86783 ms_dispat ceph txt unknown /proc/86765/task/86783/exe (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86783 ms_dispat ceph NOFD /proc/86765/task/86783/fd (opendir: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86784 ms_local ceph cwd unknown /proc/86765/task/86784/cwd (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86784 ms_local ceph rtd unknown /proc/86765/task/86784/root (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86784 ms_local ceph txt unknown /proc/86765/task/86784/exe (readlink: Permission denied) 2024-03-30T00:37:29.009 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86784 ms_local ceph NOFD /proc/86765/task/86784/fd (opendir: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86785 safe_time ceph cwd unknown /proc/86765/task/86785/cwd (readlink: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86785 safe_time ceph rtd unknown /proc/86765/task/86785/root (readlink: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86785 safe_time ceph txt unknown /proc/86765/task/86785/exe (readlink: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86785 safe_time ceph NOFD /proc/86765/task/86785/fd (opendir: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86786 safe_time ceph cwd unknown /proc/86765/task/86786/cwd (readlink: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86786 safe_time ceph rtd unknown /proc/86765/task/86786/root (readlink: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86786 safe_time ceph txt unknown /proc/86765/task/86786/exe (readlink: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86786 safe_time ceph NOFD /proc/86765/task/86786/fd (opendir: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86787 ceph-mds ceph cwd unknown /proc/86765/task/86787/cwd (readlink: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86787 ceph-mds ceph rtd unknown /proc/86765/task/86787/root (readlink: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86787 ceph-mds ceph txt unknown /proc/86765/task/86787/exe (readlink: Permission denied) 2024-03-30T00:37:29.010 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86787 ceph-mds ceph NOFD /proc/86765/task/86787/fd (opendir: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86927 safe_time ceph cwd unknown /proc/86765/task/86927/cwd (readlink: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86927 safe_time ceph rtd unknown /proc/86765/task/86927/root (readlink: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86927 safe_time ceph txt unknown /proc/86765/task/86927/exe (readlink: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 86765 86927 safe_time ceph NOFD /proc/86765/task/86927/fd (opendir: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 87108 root cwd unknown /proc/87108/cwd (readlink: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 87108 root rtd unknown /proc/87108/root (readlink: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 87108 root txt unknown /proc/87108/exe (readlink: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 87108 root NOFD /proc/87108/fd (opendir: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 87119 root cwd unknown /proc/87119/cwd (readlink: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 87119 root rtd unknown /proc/87119/root (readlink: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 87119 root txt unknown /proc/87119/exe (readlink: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 87119 root NOFD /proc/87119/fd (opendir: Permission denied) 2024-03-30T00:37:29.011 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 ceph cwd unknown /proc/87151/cwd (readlink: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 ceph rtd unknown /proc/87151/root (readlink: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 ceph txt unknown /proc/87151/exe (readlink: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 ceph NOFD /proc/87151/fd (opendir: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87152 log ceph cwd unknown /proc/87151/task/87152/cwd (readlink: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87152 log ceph rtd unknown /proc/87151/task/87152/root (readlink: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87152 log ceph txt unknown /proc/87151/task/87152/exe (readlink: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87152 log ceph NOFD /proc/87151/task/87152/fd (opendir: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87154 msgr-work ceph cwd unknown /proc/87151/task/87154/cwd (readlink: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87154 msgr-work ceph rtd unknown /proc/87151/task/87154/root (readlink: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87154 msgr-work ceph txt unknown /proc/87151/task/87154/exe (readlink: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87154 msgr-work ceph NOFD /proc/87151/task/87154/fd (opendir: Permission denied) 2024-03-30T00:37:29.012 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87155 msgr-work ceph cwd unknown /proc/87151/task/87155/cwd (readlink: Permission denied) 2024-03-30T00:37:29.013 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87155 msgr-work ceph rtd unknown /proc/87151/task/87155/root (readlink: Permission denied) 2024-03-30T00:37:29.013 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87155 msgr-work ceph txt unknown /proc/87151/task/87155/exe (readlink: Permission denied) 2024-03-30T00:37:29.013 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87155 msgr-work ceph NOFD /proc/87151/task/87155/fd (opendir: Permission denied) 2024-03-30T00:37:29.013 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87156 msgr-work ceph cwd unknown /proc/87151/task/87156/cwd (readlink: Permission denied) 2024-03-30T00:37:29.013 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87156 msgr-work ceph rtd unknown /proc/87151/task/87156/root (readlink: Permission denied) 2024-03-30T00:37:29.013 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87156 msgr-work ceph txt unknown /proc/87151/task/87156/exe (readlink: Permission denied) 2024-03-30T00:37:29.013 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87156 msgr-work ceph NOFD /proc/87151/task/87156/fd (opendir: Permission denied) 2024-03-30T00:37:29.013 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87160 service ceph cwd unknown /proc/87151/task/87160/cwd (readlink: Permission denied) 2024-03-30T00:37:29.013 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87160 service ceph rtd unknown /proc/87151/task/87160/root (readlink: Permission denied) 2024-03-30T00:37:29.013 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87160 service ceph txt unknown /proc/87151/task/87160/exe (readlink: Permission denied) 2024-03-30T00:37:29.013 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87160 service ceph NOFD /proc/87151/task/87160/fd (opendir: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87161 admin_soc ceph cwd unknown /proc/87151/task/87161/cwd (readlink: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87161 admin_soc ceph rtd unknown /proc/87151/task/87161/root (readlink: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87161 admin_soc ceph txt unknown /proc/87151/task/87161/exe (readlink: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87161 admin_soc ceph NOFD /proc/87151/task/87161/fd (opendir: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87162 signal_ha ceph cwd unknown /proc/87151/task/87162/cwd (readlink: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87162 signal_ha ceph rtd unknown /proc/87151/task/87162/root (readlink: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87162 signal_ha ceph txt unknown /proc/87151/task/87162/exe (readlink: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87162 signal_ha ceph NOFD /proc/87151/task/87162/fd (opendir: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87163 io_contex ceph cwd unknown /proc/87151/task/87163/cwd (readlink: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87163 io_contex ceph rtd unknown /proc/87151/task/87163/root (readlink: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87163 io_contex ceph txt unknown /proc/87151/task/87163/exe (readlink: Permission denied) 2024-03-30T00:37:29.014 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87163 io_contex ceph NOFD /proc/87151/task/87163/fd (opendir: Permission denied) 2024-03-30T00:37:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87164 io_contex ceph cwd unknown /proc/87151/task/87164/cwd (readlink: Permission denied) 2024-03-30T00:37:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87164 io_contex ceph rtd unknown /proc/87151/task/87164/root (readlink: Permission denied) 2024-03-30T00:37:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87164 io_contex ceph txt unknown /proc/87151/task/87164/exe (readlink: Permission denied) 2024-03-30T00:37:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87164 io_contex ceph NOFD /proc/87151/task/87164/fd (opendir: Permission denied) 2024-03-30T00:37:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87165 ms_dispat ceph cwd unknown /proc/87151/task/87165/cwd (readlink: Permission denied) 2024-03-30T00:37:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87165 ms_dispat ceph rtd unknown /proc/87151/task/87165/root (readlink: Permission denied) 2024-03-30T00:37:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87165 ms_dispat ceph txt unknown /proc/87151/task/87165/exe (readlink: Permission denied) 2024-03-30T00:37:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87165 ms_dispat ceph NOFD /proc/87151/task/87165/fd (opendir: Permission denied) 2024-03-30T00:37:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87166 ms_local ceph cwd unknown /proc/87151/task/87166/cwd (readlink: Permission denied) 2024-03-30T00:37:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87166 ms_local ceph rtd unknown /proc/87151/task/87166/root (readlink: Permission denied) 2024-03-30T00:37:29.015 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87166 ms_local ceph txt unknown /proc/87151/task/87166/exe (readlink: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87166 ms_local ceph NOFD /proc/87151/task/87166/fd (opendir: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87167 safe_time ceph cwd unknown /proc/87151/task/87167/cwd (readlink: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87167 safe_time ceph rtd unknown /proc/87151/task/87167/root (readlink: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87167 safe_time ceph txt unknown /proc/87151/task/87167/exe (readlink: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87167 safe_time ceph NOFD /proc/87151/task/87167/fd (opendir: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87168 safe_time ceph cwd unknown /proc/87151/task/87168/cwd (readlink: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87168 safe_time ceph rtd unknown /proc/87151/task/87168/root (readlink: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87168 safe_time ceph txt unknown /proc/87151/task/87168/exe (readlink: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87168 safe_time ceph NOFD /proc/87151/task/87168/fd (opendir: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87169 ceph-mds ceph cwd unknown /proc/87151/task/87169/cwd (readlink: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87169 ceph-mds ceph rtd unknown /proc/87151/task/87169/root (readlink: Permission denied) 2024-03-30T00:37:29.016 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87169 ceph-mds ceph txt unknown /proc/87151/task/87169/exe (readlink: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87169 ceph-mds ceph NOFD /proc/87151/task/87169/fd (opendir: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87176 safe_time ceph cwd unknown /proc/87151/task/87176/cwd (readlink: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87176 safe_time ceph rtd unknown /proc/87151/task/87176/root (readlink: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87176 safe_time ceph txt unknown /proc/87151/task/87176/exe (readlink: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87151 87176 safe_time ceph NOFD /proc/87151/task/87176/fd (opendir: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 87715 root cwd unknown /proc/87715/cwd (readlink: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 87715 root rtd unknown /proc/87715/root (readlink: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 87715 root txt unknown /proc/87715/exe (readlink: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 87715 root NOFD /proc/87715/fd (opendir: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 87727 root cwd unknown /proc/87727/cwd (readlink: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 87727 root rtd unknown /proc/87727/root (readlink: Permission denied) 2024-03-30T00:37:29.017 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 87727 root txt unknown /proc/87727/exe (readlink: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 87727 root NOFD /proc/87727/fd (opendir: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 ceph cwd unknown /proc/87742/cwd (readlink: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 ceph rtd unknown /proc/87742/root (readlink: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 ceph txt unknown /proc/87742/exe (readlink: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 ceph NOFD /proc/87742/fd (opendir: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87744 log ceph cwd unknown /proc/87742/task/87744/cwd (readlink: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87744 log ceph rtd unknown /proc/87742/task/87744/root (readlink: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87744 log ceph txt unknown /proc/87742/task/87744/exe (readlink: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87744 log ceph NOFD /proc/87742/task/87744/fd (opendir: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87746 msgr-work ceph cwd unknown /proc/87742/task/87746/cwd (readlink: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87746 msgr-work ceph rtd unknown /proc/87742/task/87746/root (readlink: Permission denied) 2024-03-30T00:37:29.018 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87746 msgr-work ceph txt unknown /proc/87742/task/87746/exe (readlink: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87746 msgr-work ceph NOFD /proc/87742/task/87746/fd (opendir: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87747 msgr-work ceph cwd unknown /proc/87742/task/87747/cwd (readlink: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87747 msgr-work ceph rtd unknown /proc/87742/task/87747/root (readlink: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87747 msgr-work ceph txt unknown /proc/87742/task/87747/exe (readlink: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87747 msgr-work ceph NOFD /proc/87742/task/87747/fd (opendir: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87748 msgr-work ceph cwd unknown /proc/87742/task/87748/cwd (readlink: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87748 msgr-work ceph rtd unknown /proc/87742/task/87748/root (readlink: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87748 msgr-work ceph txt unknown /proc/87742/task/87748/exe (readlink: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87748 msgr-work ceph NOFD /proc/87742/task/87748/fd (opendir: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87752 service ceph cwd unknown /proc/87742/task/87752/cwd (readlink: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87752 service ceph rtd unknown /proc/87742/task/87752/root (readlink: Permission denied) 2024-03-30T00:37:29.019 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87752 service ceph txt unknown /proc/87742/task/87752/exe (readlink: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87752 service ceph NOFD /proc/87742/task/87752/fd (opendir: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87753 admin_soc ceph cwd unknown /proc/87742/task/87753/cwd (readlink: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87753 admin_soc ceph rtd unknown /proc/87742/task/87753/root (readlink: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87753 admin_soc ceph txt unknown /proc/87742/task/87753/exe (readlink: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87753 admin_soc ceph NOFD /proc/87742/task/87753/fd (opendir: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87755 signal_ha ceph cwd unknown /proc/87742/task/87755/cwd (readlink: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87755 signal_ha ceph rtd unknown /proc/87742/task/87755/root (readlink: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87755 signal_ha ceph txt unknown /proc/87742/task/87755/exe (readlink: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87755 signal_ha ceph NOFD /proc/87742/task/87755/fd (opendir: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87756 io_contex ceph cwd unknown /proc/87742/task/87756/cwd (readlink: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87756 io_contex ceph rtd unknown /proc/87742/task/87756/root (readlink: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87756 io_contex ceph txt unknown /proc/87742/task/87756/exe (readlink: Permission denied) 2024-03-30T00:37:29.020 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87756 io_contex ceph NOFD /proc/87742/task/87756/fd (opendir: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87757 io_contex ceph cwd unknown /proc/87742/task/87757/cwd (readlink: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87757 io_contex ceph rtd unknown /proc/87742/task/87757/root (readlink: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87757 io_contex ceph txt unknown /proc/87742/task/87757/exe (readlink: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87757 io_contex ceph NOFD /proc/87742/task/87757/fd (opendir: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87758 ms_dispat ceph cwd unknown /proc/87742/task/87758/cwd (readlink: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87758 ms_dispat ceph rtd unknown /proc/87742/task/87758/root (readlink: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87758 ms_dispat ceph txt unknown /proc/87742/task/87758/exe (readlink: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87758 ms_dispat ceph NOFD /proc/87742/task/87758/fd (opendir: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87759 ms_local ceph cwd unknown /proc/87742/task/87759/cwd (readlink: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87759 ms_local ceph rtd unknown /proc/87742/task/87759/root (readlink: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87759 ms_local ceph txt unknown /proc/87742/task/87759/exe (readlink: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87759 ms_local ceph NOFD /proc/87742/task/87759/fd (opendir: Permission denied) 2024-03-30T00:37:29.021 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87760 safe_time ceph cwd unknown /proc/87742/task/87760/cwd (readlink: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87760 safe_time ceph rtd unknown /proc/87742/task/87760/root (readlink: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87760 safe_time ceph txt unknown /proc/87742/task/87760/exe (readlink: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87760 safe_time ceph NOFD /proc/87742/task/87760/fd (opendir: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87761 safe_time ceph cwd unknown /proc/87742/task/87761/cwd (readlink: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87761 safe_time ceph rtd unknown /proc/87742/task/87761/root (readlink: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87761 safe_time ceph txt unknown /proc/87742/task/87761/exe (readlink: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87761 safe_time ceph NOFD /proc/87742/task/87761/fd (opendir: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87762 ceph-mds ceph cwd unknown /proc/87742/task/87762/cwd (readlink: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87762 ceph-mds ceph rtd unknown /proc/87742/task/87762/root (readlink: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87762 ceph-mds ceph txt unknown /proc/87742/task/87762/exe (readlink: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87762 ceph-mds ceph NOFD /proc/87742/task/87762/fd (opendir: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87905 safe_time ceph cwd unknown /proc/87742/task/87905/cwd (readlink: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87905 safe_time ceph rtd unknown /proc/87742/task/87905/root (readlink: Permission denied) 2024-03-30T00:37:29.022 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87905 safe_time ceph txt unknown /proc/87742/task/87905/exe (readlink: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 87742 87905 safe_time ceph NOFD /proc/87742/task/87905/fd (opendir: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 88085 root cwd unknown /proc/88085/cwd (readlink: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 88085 root rtd unknown /proc/88085/root (readlink: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 88085 root txt unknown /proc/88085/exe (readlink: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:conmon 88085 root NOFD /proc/88085/fd (opendir: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 88096 root cwd unknown /proc/88096/cwd (readlink: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 88096 root rtd unknown /proc/88096/root (readlink: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 88096 root txt unknown /proc/88096/exe (readlink: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:podman-in 88096 root NOFD /proc/88096/fd (opendir: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 ceph cwd unknown /proc/88109/cwd (readlink: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 ceph rtd unknown /proc/88109/root (readlink: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 ceph txt unknown /proc/88109/exe (readlink: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 ceph NOFD /proc/88109/fd (opendir: Permission denied) 2024-03-30T00:37:29.023 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88128 log ceph cwd unknown /proc/88109/task/88128/cwd (readlink: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88128 log ceph rtd unknown /proc/88109/task/88128/root (readlink: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88128 log ceph txt unknown /proc/88109/task/88128/exe (readlink: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88128 log ceph NOFD /proc/88109/task/88128/fd (opendir: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88130 msgr-work ceph cwd unknown /proc/88109/task/88130/cwd (readlink: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88130 msgr-work ceph rtd unknown /proc/88109/task/88130/root (readlink: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88130 msgr-work ceph txt unknown /proc/88109/task/88130/exe (readlink: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88130 msgr-work ceph NOFD /proc/88109/task/88130/fd (opendir: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88131 msgr-work ceph cwd unknown /proc/88109/task/88131/cwd (readlink: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88131 msgr-work ceph rtd unknown /proc/88109/task/88131/root (readlink: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88131 msgr-work ceph txt unknown /proc/88109/task/88131/exe (readlink: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88131 msgr-work ceph NOFD /proc/88109/task/88131/fd (opendir: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88132 msgr-work ceph cwd unknown /proc/88109/task/88132/cwd (readlink: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88132 msgr-work ceph rtd unknown /proc/88109/task/88132/root (readlink: Permission denied) 2024-03-30T00:37:29.024 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88132 msgr-work ceph txt unknown /proc/88109/task/88132/exe (readlink: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88132 msgr-work ceph NOFD /proc/88109/task/88132/fd (opendir: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88136 service ceph cwd unknown /proc/88109/task/88136/cwd (readlink: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88136 service ceph rtd unknown /proc/88109/task/88136/root (readlink: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88136 service ceph txt unknown /proc/88109/task/88136/exe (readlink: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88136 service ceph NOFD /proc/88109/task/88136/fd (opendir: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88137 admin_soc ceph cwd unknown /proc/88109/task/88137/cwd (readlink: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88137 admin_soc ceph rtd unknown /proc/88109/task/88137/root (readlink: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88137 admin_soc ceph txt unknown /proc/88109/task/88137/exe (readlink: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88137 admin_soc ceph NOFD /proc/88109/task/88137/fd (opendir: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88138 signal_ha ceph cwd unknown /proc/88109/task/88138/cwd (readlink: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88138 signal_ha ceph rtd unknown /proc/88109/task/88138/root (readlink: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88138 signal_ha ceph txt unknown /proc/88109/task/88138/exe (readlink: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88138 signal_ha ceph NOFD /proc/88109/task/88138/fd (opendir: Permission denied) 2024-03-30T00:37:29.025 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88139 io_contex ceph cwd unknown /proc/88109/task/88139/cwd (readlink: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88139 io_contex ceph rtd unknown /proc/88109/task/88139/root (readlink: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88139 io_contex ceph txt unknown /proc/88109/task/88139/exe (readlink: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88139 io_contex ceph NOFD /proc/88109/task/88139/fd (opendir: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88140 io_contex ceph cwd unknown /proc/88109/task/88140/cwd (readlink: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88140 io_contex ceph rtd unknown /proc/88109/task/88140/root (readlink: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88140 io_contex ceph txt unknown /proc/88109/task/88140/exe (readlink: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88140 io_contex ceph NOFD /proc/88109/task/88140/fd (opendir: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88141 ms_dispat ceph cwd unknown /proc/88109/task/88141/cwd (readlink: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88141 ms_dispat ceph rtd unknown /proc/88109/task/88141/root (readlink: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88141 ms_dispat ceph txt unknown /proc/88109/task/88141/exe (readlink: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88141 ms_dispat ceph NOFD /proc/88109/task/88141/fd (opendir: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88142 ms_local ceph cwd unknown /proc/88109/task/88142/cwd (readlink: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88142 ms_local ceph rtd unknown /proc/88109/task/88142/root (readlink: Permission denied) 2024-03-30T00:37:29.026 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88142 ms_local ceph txt unknown /proc/88109/task/88142/exe (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88142 ms_local ceph NOFD /proc/88109/task/88142/fd (opendir: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88143 safe_time ceph cwd unknown /proc/88109/task/88143/cwd (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88143 safe_time ceph rtd unknown /proc/88109/task/88143/root (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88143 safe_time ceph txt unknown /proc/88109/task/88143/exe (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88143 safe_time ceph NOFD /proc/88109/task/88143/fd (opendir: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88144 safe_time ceph cwd unknown /proc/88109/task/88144/cwd (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88144 safe_time ceph rtd unknown /proc/88109/task/88144/root (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88144 safe_time ceph txt unknown /proc/88109/task/88144/exe (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88144 safe_time ceph NOFD /proc/88109/task/88144/fd (opendir: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88145 ceph-mds ceph cwd unknown /proc/88109/task/88145/cwd (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88145 ceph-mds ceph rtd unknown /proc/88109/task/88145/root (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88145 ceph-mds ceph txt unknown /proc/88109/task/88145/exe (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88145 ceph-mds ceph NOFD /proc/88109/task/88145/fd (opendir: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88176 safe_time ceph cwd unknown /proc/88109/task/88176/cwd (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88176 safe_time ceph rtd unknown /proc/88109/task/88176/root (readlink: Permission denied) 2024-03-30T00:37:29.027 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88176 safe_time ceph txt unknown /proc/88109/task/88176/exe (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 88176 safe_time ceph NOFD /proc/88109/task/88176/fd (opendir: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89602 ceph_time ceph cwd unknown /proc/88109/task/89602/cwd (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89602 ceph_time ceph rtd unknown /proc/88109/task/89602/root (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89602 ceph_time ceph txt unknown /proc/88109/task/89602/exe (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89602 ceph_time ceph NOFD /proc/88109/task/89602/fd (opendir: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89604 OpHistory ceph cwd unknown /proc/88109/task/89604/cwd (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89604 OpHistory ceph rtd unknown /proc/88109/task/89604/root (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89604 OpHistory ceph txt unknown /proc/88109/task/89604/exe (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89604 OpHistory ceph NOFD /proc/88109/task/89604/fd (opendir: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89605 ms_dispat ceph cwd unknown /proc/88109/task/89605/cwd (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89605 ms_dispat ceph rtd unknown /proc/88109/task/89605/root (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89605 ms_dispat ceph txt unknown /proc/88109/task/89605/exe (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89605 ms_dispat ceph NOFD /proc/88109/task/89605/fd (opendir: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89606 mds_rank_ ceph cwd unknown /proc/88109/task/89606/cwd (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89606 mds_rank_ ceph rtd unknown /proc/88109/task/89606/root (readlink: Permission denied) 2024-03-30T00:37:29.028 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89606 mds_rank_ ceph txt unknown /proc/88109/task/89606/exe (readlink: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89606 mds_rank_ ceph NOFD /proc/88109/task/89606/fd (opendir: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89607 PQ_Finish ceph cwd unknown /proc/88109/task/89607/cwd (readlink: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89607 PQ_Finish ceph rtd unknown /proc/88109/task/89607/root (readlink: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89607 PQ_Finish ceph txt unknown /proc/88109/task/89607/exe (readlink: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89607 PQ_Finish ceph NOFD /proc/88109/task/89607/fd (opendir: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89608 safe_time ceph cwd unknown /proc/88109/task/89608/cwd (readlink: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89608 safe_time ceph rtd unknown /proc/88109/task/89608/root (readlink: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89608 safe_time ceph txt unknown /proc/88109/task/89608/exe (readlink: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89608 safe_time ceph NOFD /proc/88109/task/89608/fd (opendir: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89609 MR_Finish ceph cwd unknown /proc/88109/task/89609/cwd (readlink: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89609 MR_Finish ceph rtd unknown /proc/88109/task/89609/root (readlink: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89609 MR_Finish ceph txt unknown /proc/88109/task/89609/exe (readlink: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89609 MR_Finish ceph NOFD /proc/88109/task/89609/fd (opendir: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89610 md_submit ceph cwd unknown /proc/88109/task/89610/cwd (readlink: Permission denied) 2024-03-30T00:37:29.029 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89610 md_submit ceph rtd unknown /proc/88109/task/89610/root (readlink: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89610 md_submit ceph txt unknown /proc/88109/task/89610/exe (readlink: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89610 md_submit ceph NOFD /proc/88109/task/89610/fd (opendir: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89655 ms_dispat ceph cwd unknown /proc/88109/task/89655/cwd (readlink: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89655 ms_dispat ceph rtd unknown /proc/88109/task/89655/root (readlink: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89655 ms_dispat ceph txt unknown /proc/88109/task/89655/exe (readlink: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-mds 88109 89655 ms_dispat ceph NOFD /proc/88109/task/89655/fd (opendir: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 90684 root cwd unknown /proc/90684/cwd (readlink: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 90684 root rtd unknown /proc/90684/root (readlink: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 90684 root txt unknown /proc/90684/exe (readlink: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 90684 root NOFD /proc/90684/fd (opendir: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 91763 root cwd unknown /proc/91763/cwd (readlink: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 91763 root rtd unknown /proc/91763/root (readlink: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 91763 root txt unknown /proc/91763/exe (readlink: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/4 91763 root NOFD /proc/91763/fd (opendir: Permission denied) 2024-03-30T00:37:29.030 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 92548 root cwd unknown /proc/92548/cwd (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 92548 root rtd unknown /proc/92548/root (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 92548 root txt unknown /proc/92548/exe (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/2 92548 root NOFD /proc/92548/fd (opendir: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92676 root cwd unknown /proc/92676/cwd (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92676 root rtd unknown /proc/92676/root (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92676 root txt unknown /proc/92676/exe (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92676 root NOFD /proc/92676/fd (opendir: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92677 root cwd unknown /proc/92677/cwd (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92677 root rtd unknown /proc/92677/root (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92677 root txt unknown /proc/92677/exe (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92677 root NOFD /proc/92677/fd (opendir: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92678 root cwd unknown /proc/92678/cwd (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92678 root rtd unknown /proc/92678/root (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92678 root txt unknown /proc/92678/exe (readlink: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 92678 root NOFD /proc/92678/fd (opendir: Permission denied) 2024-03-30T00:37:29.031 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 93747 root cwd unknown /proc/93747/cwd (readlink: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 93747 root rtd unknown /proc/93747/root (readlink: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 93747 root txt unknown /proc/93747/exe (readlink: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/u 93747 root NOFD /proc/93747/fd (opendir: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 94126 root cwd unknown /proc/94126/cwd (readlink: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 94126 root rtd unknown /proc/94126/root (readlink: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 94126 root txt unknown /proc/94126/exe (readlink: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/6 94126 root NOFD /proc/94126/fd (opendir: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94380 root cwd unknown /proc/94380/cwd (readlink: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94380 root rtd unknown /proc/94380/root (readlink: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94380 root txt unknown /proc/94380/exe (readlink: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94380 root NOFD /proc/94380/fd (opendir: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94381 root cwd unknown /proc/94381/cwd (readlink: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94381 root rtd unknown /proc/94381/root (readlink: Permission denied) 2024-03-30T00:37:29.032 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94381 root txt unknown /proc/94381/exe (readlink: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94381 root NOFD /proc/94381/fd (opendir: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94382 root cwd unknown /proc/94382/cwd (readlink: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94382 root rtd unknown /proc/94382/root (readlink: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94382 root txt unknown /proc/94382/exe (readlink: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94382 root NOFD /proc/94382/fd (opendir: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94383 root cwd unknown /proc/94383/cwd (readlink: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94383 root rtd unknown /proc/94383/root (readlink: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94383 root txt unknown /proc/94383/exe (readlink: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/1 94383 root NOFD /proc/94383/fd (opendir: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94735 root cwd unknown /proc/94735/cwd (readlink: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94735 root rtd unknown /proc/94735/root (readlink: Permission denied) 2024-03-30T00:37:29.033 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94735 root txt unknown /proc/94735/exe (readlink: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94735 root NOFD /proc/94735/fd (opendir: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94736 root cwd unknown /proc/94736/cwd (readlink: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94736 root rtd unknown /proc/94736/root (readlink: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94736 root txt unknown /proc/94736/exe (readlink: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94736 root NOFD /proc/94736/fd (opendir: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94737 root cwd unknown /proc/94737/cwd (readlink: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94737 root rtd unknown /proc/94737/root (readlink: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94737 root txt unknown /proc/94737/exe (readlink: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94737 root NOFD /proc/94737/fd (opendir: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94738 root cwd unknown /proc/94738/cwd (readlink: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94738 root rtd unknown /proc/94738/root (readlink: Permission denied) 2024-03-30T00:37:29.034 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94738 root txt unknown /proc/94738/exe (readlink: Permission denied) 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94738 root NOFD /proc/94738/fd (opendir: Permission denied) 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94739 root cwd unknown /proc/94739/cwd (readlink: Permission denied) 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94739 root rtd unknown /proc/94739/root (readlink: Permission denied) 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94739 root txt unknown /proc/94739/exe (readlink: Permission denied) 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/0 94739 root NOFD /proc/94739/fd (opendir: Permission denied) 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu cwd DIR 8,1 4096 73163 /home/ubuntu 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu rtd DIR 8,1 4096 2 / 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu txt REG 8,1 1150584 5206 /usr/bin/bash 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 2586930 18369 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 2088960 5167 /usr/lib64/libc-2.28.so 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 19584 5169 /usr/lib64/libdl-2.28.so 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 187552 4894 /usr/lib64/libtinfo.so.6.1 2024-03-30T00:37:29.035 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 1105744 5160 /usr/lib64/ld-2.28.so 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 337024 18370 /usr/lib/locale/en_US.utf8/LC_CTYPE 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 54 18374 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 3316 18324 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 286 18408 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 57 18372 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 34 18389 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 77 18373 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 167 18405 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 59 18409 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 23 18407 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 26998 5205 /usr/lib64/gconv/gconv-modules.cache 2024-03-30T00:37:29.036 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu mem REG 8,1 368 18406 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu 0r FIFO 0,13 0t0 367632 pipe 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu 1w FIFO 0,13 0t0 367633 pipe 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:bash 100923 ubuntu 2w FIFO 0,13 0t0 367634 pipe 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 100960 root cwd unknown /proc/100960/cwd (readlink: Permission denied) 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 100960 root rtd unknown /proc/100960/root (readlink: Permission denied) 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 100960 root txt unknown /proc/100960/exe (readlink: Permission denied) 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 100960 root NOFD /proc/100960/fd (opendir: Permission denied) 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 100964 root cwd unknown /proc/100964/cwd (readlink: Permission denied) 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 100964 root rtd unknown /proc/100964/root (readlink: Permission denied) 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 100964 root txt unknown /proc/100964/exe (readlink: Permission denied) 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 100964 root NOFD /proc/100964/fd (opendir: Permission denied) 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:daemon-he 100974 root cwd unknown /proc/100974/cwd (readlink: Permission denied) 2024-03-30T00:37:29.037 INFO:tasks.workunit.client.0.smithi099.stdout:daemon-he 100974 root rtd unknown /proc/100974/root (readlink: Permission denied) 2024-03-30T00:37:29.038 INFO:tasks.workunit.client.0.smithi099.stdout:daemon-he 100974 root txt unknown /proc/100974/exe (readlink: Permission denied) 2024-03-30T00:37:29.038 INFO:tasks.workunit.client.0.smithi099.stdout:daemon-he 100974 root NOFD /proc/100974/fd (opendir: Permission denied) 2024-03-30T00:37:29.038 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 root cwd unknown /proc/100990/cwd (readlink: Permission denied) 2024-03-30T00:37:29.038 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 root rtd unknown /proc/100990/root (readlink: Permission denied) 2024-03-30T00:37:29.038 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 root txt unknown /proc/100990/exe (readlink: Permission denied) 2024-03-30T00:37:29.038 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 root NOFD /proc/100990/fd (opendir: Permission denied) 2024-03-30T00:37:29.038 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101010 log root cwd unknown /proc/100990/task/101010/cwd (readlink: Permission denied) 2024-03-30T00:37:29.038 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101010 log root rtd unknown /proc/100990/task/101010/root (readlink: Permission denied) 2024-03-30T00:37:29.038 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101010 log root txt unknown /proc/100990/task/101010/exe (readlink: Permission denied) 2024-03-30T00:37:29.038 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101010 log root NOFD /proc/100990/task/101010/fd (opendir: Permission denied) 2024-03-30T00:37:29.038 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101012 msgr-work root cwd unknown /proc/100990/task/101012/cwd (readlink: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101012 msgr-work root rtd unknown /proc/100990/task/101012/root (readlink: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101012 msgr-work root txt unknown /proc/100990/task/101012/exe (readlink: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101012 msgr-work root NOFD /proc/100990/task/101012/fd (opendir: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101013 msgr-work root cwd unknown /proc/100990/task/101013/cwd (readlink: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101013 msgr-work root rtd unknown /proc/100990/task/101013/root (readlink: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101013 msgr-work root txt unknown /proc/100990/task/101013/exe (readlink: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101013 msgr-work root NOFD /proc/100990/task/101013/fd (opendir: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101014 msgr-work root cwd unknown /proc/100990/task/101014/cwd (readlink: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101014 msgr-work root rtd unknown /proc/100990/task/101014/root (readlink: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101014 msgr-work root txt unknown /proc/100990/task/101014/exe (readlink: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101014 msgr-work root NOFD /proc/100990/task/101014/fd (opendir: Permission denied) 2024-03-30T00:37:29.039 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101018 service root cwd unknown /proc/100990/task/101018/cwd (readlink: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101018 service root rtd unknown /proc/100990/task/101018/root (readlink: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101018 service root txt unknown /proc/100990/task/101018/exe (readlink: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101018 service root NOFD /proc/100990/task/101018/fd (opendir: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101019 admin_soc root cwd unknown /proc/100990/task/101019/cwd (readlink: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101019 admin_soc root rtd unknown /proc/100990/task/101019/root (readlink: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101019 admin_soc root txt unknown /proc/100990/task/101019/exe (readlink: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101019 admin_soc root NOFD /proc/100990/task/101019/fd (opendir: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101020 signal_ha root cwd unknown /proc/100990/task/101020/cwd (readlink: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101020 signal_ha root rtd unknown /proc/100990/task/101020/root (readlink: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101020 signal_ha root txt unknown /proc/100990/task/101020/exe (readlink: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101020 signal_ha root NOFD /proc/100990/task/101020/fd (opendir: Permission denied) 2024-03-30T00:37:29.040 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101021 io_contex root cwd unknown /proc/100990/task/101021/cwd (readlink: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101021 io_contex root rtd unknown /proc/100990/task/101021/root (readlink: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101021 io_contex root txt unknown /proc/100990/task/101021/exe (readlink: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101021 io_contex root NOFD /proc/100990/task/101021/fd (opendir: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101022 io_contex root cwd unknown /proc/100990/task/101022/cwd (readlink: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101022 io_contex root rtd unknown /proc/100990/task/101022/root (readlink: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101022 io_contex root txt unknown /proc/100990/task/101022/exe (readlink: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101022 io_contex root NOFD /proc/100990/task/101022/fd (opendir: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101023 ceph_time root cwd unknown /proc/100990/task/101023/cwd (readlink: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101023 ceph_time root rtd unknown /proc/100990/task/101023/root (readlink: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101023 ceph_time root txt unknown /proc/100990/task/101023/exe (readlink: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101023 ceph_time root NOFD /proc/100990/task/101023/fd (opendir: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101024 fn_anonym root cwd unknown /proc/100990/task/101024/cwd (readlink: Permission denied) 2024-03-30T00:37:29.041 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101024 fn_anonym root rtd unknown /proc/100990/task/101024/root (readlink: Permission denied) 2024-03-30T00:37:29.042 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101024 fn_anonym root txt unknown /proc/100990/task/101024/exe (readlink: Permission denied) 2024-03-30T00:37:29.042 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101024 fn_anonym root NOFD /proc/100990/task/101024/fd (opendir: Permission denied) 2024-03-30T00:37:29.042 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101025 fn_anonym root cwd unknown /proc/100990/task/101025/cwd (readlink: Permission denied) 2024-03-30T00:37:29.042 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101025 fn_anonym root rtd unknown /proc/100990/task/101025/root (readlink: Permission denied) 2024-03-30T00:37:29.042 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101025 fn_anonym root txt unknown /proc/100990/task/101025/exe (readlink: Permission denied) 2024-03-30T00:37:29.042 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101025 fn_anonym root NOFD /proc/100990/task/101025/fd (opendir: Permission denied) 2024-03-30T00:37:29.042 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101026 fn_anonym root cwd unknown /proc/100990/task/101026/cwd (readlink: Permission denied) 2024-03-30T00:37:29.042 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101026 fn_anonym root rtd unknown /proc/100990/task/101026/root (readlink: Permission denied) 2024-03-30T00:37:29.042 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101026 fn_anonym root txt unknown /proc/100990/task/101026/exe (readlink: Permission denied) 2024-03-30T00:37:29.042 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101026 fn_anonym root NOFD /proc/100990/task/101026/fd (opendir: Permission denied) 2024-03-30T00:37:29.042 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101027 fn_anonym root cwd unknown /proc/100990/task/101027/cwd (readlink: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101027 fn_anonym root rtd unknown /proc/100990/task/101027/root (readlink: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101027 fn_anonym root txt unknown /proc/100990/task/101027/exe (readlink: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101027 fn_anonym root NOFD /proc/100990/task/101027/fd (opendir: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101028 fn_anonym root cwd unknown /proc/100990/task/101028/cwd (readlink: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101028 fn_anonym root rtd unknown /proc/100990/task/101028/root (readlink: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101028 fn_anonym root txt unknown /proc/100990/task/101028/exe (readlink: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101028 fn_anonym root NOFD /proc/100990/task/101028/fd (opendir: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101029 safe_time root cwd unknown /proc/100990/task/101029/cwd (readlink: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101029 safe_time root rtd unknown /proc/100990/task/101029/root (readlink: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101029 safe_time root txt unknown /proc/100990/task/101029/exe (readlink: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101029 safe_time root NOFD /proc/100990/task/101029/fd (opendir: Permission denied) 2024-03-30T00:37:29.043 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101030 fn_anonym root cwd unknown /proc/100990/task/101030/cwd (readlink: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101030 fn_anonym root rtd unknown /proc/100990/task/101030/root (readlink: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101030 fn_anonym root txt unknown /proc/100990/task/101030/exe (readlink: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101030 fn_anonym root NOFD /proc/100990/task/101030/fd (opendir: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101031 flusher root cwd unknown /proc/100990/task/101031/cwd (readlink: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101031 flusher root rtd unknown /proc/100990/task/101031/root (readlink: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101031 flusher root txt unknown /proc/100990/task/101031/exe (readlink: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101031 flusher root NOFD /proc/100990/task/101031/fd (opendir: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101032 ms_dispat root cwd unknown /proc/100990/task/101032/cwd (readlink: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101032 ms_dispat root rtd unknown /proc/100990/task/101032/root (readlink: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101032 ms_dispat root txt unknown /proc/100990/task/101032/exe (readlink: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101032 ms_dispat root NOFD /proc/100990/task/101032/fd (opendir: Permission denied) 2024-03-30T00:37:29.044 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101033 ms_local root cwd unknown /proc/100990/task/101033/cwd (readlink: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101033 ms_local root rtd unknown /proc/100990/task/101033/root (readlink: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101033 ms_local root txt unknown /proc/100990/task/101033/exe (readlink: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101033 ms_local root NOFD /proc/100990/task/101033/fd (opendir: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101034 safe_time root cwd unknown /proc/100990/task/101034/cwd (readlink: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101034 safe_time root rtd unknown /proc/100990/task/101034/root (readlink: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101034 safe_time root txt unknown /proc/100990/task/101034/exe (readlink: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101034 safe_time root NOFD /proc/100990/task/101034/fd (opendir: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101035 ceph-fuse root cwd unknown /proc/100990/task/101035/cwd (readlink: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101035 ceph-fuse root rtd unknown /proc/100990/task/101035/root (readlink: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101035 ceph-fuse root txt unknown /proc/100990/task/101035/exe (readlink: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101035 ceph-fuse root NOFD /proc/100990/task/101035/fd (opendir: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101037 ceph-fuse root cwd unknown /proc/100990/task/101037/cwd (readlink: Permission denied) 2024-03-30T00:37:29.045 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101037 ceph-fuse root rtd unknown /proc/100990/task/101037/root (readlink: Permission denied) 2024-03-30T00:37:29.046 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101037 ceph-fuse root txt unknown /proc/100990/task/101037/exe (readlink: Permission denied) 2024-03-30T00:37:29.046 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101037 ceph-fuse root NOFD /proc/100990/task/101037/fd (opendir: Permission denied) 2024-03-30T00:37:29.046 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101041 ceph-fuse root cwd unknown /proc/100990/task/101041/cwd (readlink: Permission denied) 2024-03-30T00:37:29.046 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101041 ceph-fuse root rtd unknown /proc/100990/task/101041/root (readlink: Permission denied) 2024-03-30T00:37:29.046 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101041 ceph-fuse root txt unknown /proc/100990/task/101041/exe (readlink: Permission denied) 2024-03-30T00:37:29.046 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 101041 ceph-fuse root NOFD /proc/100990/task/101041/fd (opendir: Permission denied) 2024-03-30T00:37:29.046 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104083 ceph-fuse root cwd unknown /proc/100990/task/104083/cwd (readlink: Permission denied) 2024-03-30T00:37:29.046 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104083 ceph-fuse root rtd unknown /proc/100990/task/104083/root (readlink: Permission denied) 2024-03-30T00:37:29.046 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104083 ceph-fuse root txt unknown /proc/100990/task/104083/exe (readlink: Permission denied) 2024-03-30T00:37:29.046 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104083 ceph-fuse root NOFD /proc/100990/task/104083/fd (opendir: Permission denied) 2024-03-30T00:37:29.046 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104616 ceph-fuse root cwd unknown /proc/100990/task/104616/cwd (readlink: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104616 ceph-fuse root rtd unknown /proc/100990/task/104616/root (readlink: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104616 ceph-fuse root txt unknown /proc/100990/task/104616/exe (readlink: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104616 ceph-fuse root NOFD /proc/100990/task/104616/fd (opendir: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104732 ceph-fuse root cwd unknown /proc/100990/task/104732/cwd (readlink: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104732 ceph-fuse root rtd unknown /proc/100990/task/104732/root (readlink: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104732 ceph-fuse root txt unknown /proc/100990/task/104732/exe (readlink: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:ceph-fuse 100990 104732 ceph-fuse root NOFD /proc/100990/task/104732/fd (opendir: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 101680 root cwd unknown /proc/101680/cwd (readlink: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 101680 root rtd unknown /proc/101680/root (readlink: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 101680 root txt unknown /proc/101680/exe (readlink: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:kworker/5 101680 root NOFD /proc/101680/fd (opendir: Permission denied) 2024-03-30T00:37:29.047 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 102043 root cwd unknown /proc/102043/cwd (readlink: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 102043 root rtd unknown /proc/102043/root (readlink: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 102043 root txt unknown /proc/102043/exe (readlink: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:sudo 102043 root NOFD /proc/102043/fd (opendir: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:daemon-he 102102 root cwd unknown /proc/102102/cwd (readlink: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:daemon-he 102102 root rtd unknown /proc/102102/root (readlink: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:daemon-he 102102 root txt unknown /proc/102102/exe (readlink: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:daemon-he 102102 root NOFD /proc/102102/fd (opendir: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:bash 102106 root cwd unknown /proc/102106/cwd (readlink: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:bash 102106 root rtd unknown /proc/102106/root (readlink: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:bash 102106 root txt unknown /proc/102106/exe (readlink: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:bash 102106 root NOFD /proc/102106/fd (opendir: Permission denied) 2024-03-30T00:37:29.048 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu cwd DIR 0,122 195298064 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu rtd DIR 8,1 4096 2 / 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu txt REG 8,1 42840 7406 /usr/bin/timeout 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 2586930 18369 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 2088960 5167 /usr/lib64/libc-2.28.so 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 149976 5181 /usr/lib64/libpthread-2.28.so 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 42744 5185 /usr/lib64/librt-2.28.so 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 1105744 5160 /usr/lib64/ld-2.28.so 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 337024 18370 /usr/lib/locale/en_US.utf8/LC_CTYPE 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 54 18374 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 3316 18324 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 286 18408 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-30T00:37:29.049 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 57 18372 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-30T00:37:29.050 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 34 18389 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-30T00:37:29.050 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 77 18373 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-30T00:37:29.050 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 167 18405 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-30T00:37:29.050 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 59 18409 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-30T00:37:29.050 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 23 18407 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-30T00:37:29.050 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 26998 5205 /usr/lib64/gconv/gconv-modules.cache 2024-03-30T00:37:29.050 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu mem REG 8,1 368 18406 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-30T00:37:29.050 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu 0r FIFO 0,13 0t0 379238 pipe 2024-03-30T00:37:29.050 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu 1w FIFO 0,13 0t0 379239 pipe 2024-03-30T00:37:29.050 INFO:tasks.workunit.client.0.smithi099.stdout:timeout 104060 ubuntu 2w FIFO 0,13 0t0 379240 pipe 2024-03-30T00:37:29.050 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu cwd DIR 0,122 195298064 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-30T00:37:29.051 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu rtd DIR 8,1 4096 2 / 2024-03-30T00:37:29.051 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu txt REG 8,1 1150584 5206 /usr/bin/bash 2024-03-30T00:37:29.051 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 2586930 18369 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-03-30T00:37:29.051 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 2088960 5167 /usr/lib64/libc-2.28.so 2024-03-30T00:37:29.051 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 19584 5169 /usr/lib64/libdl-2.28.so 2024-03-30T00:37:29.051 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 187552 4894 /usr/lib64/libtinfo.so.6.1 2024-03-30T00:37:29.051 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 1105744 5160 /usr/lib64/ld-2.28.so 2024-03-30T00:37:29.051 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 337024 18370 /usr/lib/locale/en_US.utf8/LC_CTYPE 2024-03-30T00:37:29.051 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 54 18374 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-03-30T00:37:29.051 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 3316 18324 /usr/lib/locale/en_US.utf8/LC_TIME 2024-03-30T00:37:29.051 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 286 18408 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 57 18372 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 34 18389 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 77 18373 /usr/lib/locale/en_US.utf8/LC_NAME 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 167 18405 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 59 18409 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 23 18407 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 26998 5205 /usr/lib64/gconv/gconv-modules.cache 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu mem REG 8,1 368 18406 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu 0r FIFO 0,13 0t0 379238 pipe 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu 1w FIFO 0,13 0t0 379239 pipe 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu 2w FIFO 0,13 0t0 379240 pipe 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:fsync-tes 104081 ubuntu 255r REG 8,1 209 923379 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:sleep 105391 root cwd unknown /proc/105391/cwd (readlink: Permission denied) 2024-03-30T00:37:29.052 INFO:tasks.workunit.client.0.smithi099.stdout:sleep 105391 root rtd unknown /proc/105391/root (readlink: Permission denied) 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:sleep 105391 root txt unknown /proc/105391/exe (readlink: Permission denied) 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:sleep 105391 root NOFD /proc/105391/fd (opendir: Permission denied) 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:sleep 105644 root cwd unknown /proc/105644/cwd (readlink: Permission denied) 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:sleep 105644 root rtd unknown /proc/105644/root (readlink: Permission denied) 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:sleep 105644 root txt unknown /proc/105644/exe (readlink: Permission denied) 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:sleep 105644 root NOFD /proc/105644/fd (opendir: Permission denied) 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu cwd DIR 0,122 195298064 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu rtd DIR 8,1 4096 2 / 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu txt REG 8,1 179672 11587 /usr/bin/lsof 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 99600 5318 /usr/lib64/libz.so.1.2.11 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 88824 5183 /usr/lib64/libresolv-2.28.so 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 3087848 2383 /usr/lib64/libcrypto.so.1.1.1k 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 16192 5609 /usr/lib64/libkeyutils.so.1.6 2024-03-30T00:37:29.053 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 71464 7213 /usr/lib64/libkrb5support.so.0.1 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 149976 5181 /usr/lib64/libpthread-2.28.so 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 16792 5320 /usr/lib64/libcom_err.so.2.1 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 96272 7207 /usr/lib64/libk5crypto.so.3.1 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 975608 7212 /usr/lib64/libkrb5.so.3.3 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 355960 7204 /usr/lib64/libgssapi_krb5.so.2.2 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 19584 5169 /usr/lib64/libdl-2.28.so 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 543160 4872 /usr/lib64/libpcre2-8.so.0.7.1 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 2088960 5167 /usr/lib64/libc-2.28.so 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 213592 7222 /usr/lib64/libtirpc.so.3.0.0 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 168536 4876 /usr/lib64/libselinux.so.1 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 1105744 5160 /usr/lib64/ld-2.28.so 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 337024 18370 /usr/lib/locale/en_US.utf8/LC_CTYPE 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu mem REG 8,1 26998 5205 /usr/lib64/gconv/gconv-modules.cache 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu 0r FIFO 0,13 0t0 379238 pipe 2024-03-30T00:37:29.054 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu 1w FIFO 0,13 0t0 379239 pipe 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu 2w FIFO 0,13 0t0 379240 pipe 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu 3r DIR 0,21 0 1 /proc 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu 4r DIR 0,21 7 386305 /proc/105646/fd 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu 5w FIFO 0,13 0t0 386310 pipe 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105646 ubuntu 6r FIFO 0,13 0t0 386311 pipe 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu cwd DIR 0,122 195298064 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu rtd DIR 8,1 4096 2 / 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu txt REG 8,1 179672 11587 /usr/bin/lsof 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 99600 5318 /usr/lib64/libz.so.1.2.11 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 88824 5183 /usr/lib64/libresolv-2.28.so 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 3087848 2383 /usr/lib64/libcrypto.so.1.1.1k 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 16192 5609 /usr/lib64/libkeyutils.so.1.6 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 71464 7213 /usr/lib64/libkrb5support.so.0.1 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 149976 5181 /usr/lib64/libpthread-2.28.so 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 16792 5320 /usr/lib64/libcom_err.so.2.1 2024-03-30T00:37:29.055 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 96272 7207 /usr/lib64/libk5crypto.so.3.1 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 975608 7212 /usr/lib64/libkrb5.so.3.3 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 355960 7204 /usr/lib64/libgssapi_krb5.so.2.2 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 19584 5169 /usr/lib64/libdl-2.28.so 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 543160 4872 /usr/lib64/libpcre2-8.so.0.7.1 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 2088960 5167 /usr/lib64/libc-2.28.so 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 213592 7222 /usr/lib64/libtirpc.so.3.0.0 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 168536 4876 /usr/lib64/libselinux.so.1 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 1105744 5160 /usr/lib64/ld-2.28.so 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 337024 18370 /usr/lib/locale/en_US.utf8/LC_CTYPE 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu mem REG 8,1 26998 5205 /usr/lib64/gconv/gconv-modules.cache 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu 4r FIFO 0,13 0t0 386310 pipe 2024-03-30T00:37:29.056 INFO:tasks.workunit.client.0.smithi099.stdout:lsof 105647 ubuntu 7w FIFO 0,13 0t0 386311 pipe 2024-03-30T00:37:29.057 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-30T00:37:29.057 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-30T00:37:29.214 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-03-30T00:37:29.214 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-30T00:37:30.143 DEBUG:teuthology.parallel:result is None 2024-03-30T00:37:30.144 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-30T00:37:30.222 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-30T00:37:30.222 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-30T00:37:30.235 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi099.front.sepia.ceph.com 2024-03-30T00:37:30.247 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-30T00:37:30.257 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-30T00:37:30.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:30 smithi161 ceph-mon[42221]: pgmap v387: 97 pgs: 97 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 13 MiB/s wr, 2.95k op/s 2024-03-30T00:37:30.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:30 smithi099 ceph-mon[38079]: pgmap v387: 97 pgs: 97 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 13 MiB/s wr, 2.95k op/s 2024-03-30T00:37:30.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:30 smithi142 ceph-mon[41779]: pgmap v387: 97 pgs: 97 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 13 MiB/s wr, 2.95k op/s 2024-03-30T00:37:32.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T00:37:32.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:32 smithi161 ceph-mon[42221]: pgmap v388: 97 pgs: 97 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-03-30T00:37:32.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:32 smithi099 ceph-mon[38079]: pgmap v388: 97 pgs: 97 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-03-30T00:37:32.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.559+0000 7fe67fc00700 1 -- 172.21.15.99:0/3630843651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe678100d80 msgr2=0x7fe6781011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.559+0000 7fe67fc00700 1 --2- 172.21.15.99:0/3630843651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe678100d80 0x7fe6781011a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009920 tx=0x7fe66c009c30 comp rx=0 tx=0).stop 2024-03-30T00:37:32.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.559+0000 7fe67fc00700 1 -- 172.21.15.99:0/3630843651 shutdown_connections 2024-03-30T00:37:32.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.559+0000 7fe67fc00700 1 --2- 172.21.15.99:0/3630843651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe678102910 0x7fe6781091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.559+0000 7fe67fc00700 1 --2- 172.21.15.99:0/3630843651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe678100d80 0x7fe6781011a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.559+0000 7fe67fc00700 1 --2- 172.21.15.99:0/3630843651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe678101f70 0x7fe6781023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.559+0000 7fe67fc00700 1 -- 172.21.15.99:0/3630843651 >> 172.21.15.99:0/3630843651 conn(0x7fe6780fc2f0 msgr2=0x7fe6780fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:32.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.559+0000 7fe67fc00700 1 -- 172.21.15.99:0/3630843651 shutdown_connections 2024-03-30T00:37:32.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.559+0000 7fe67fc00700 1 -- 172.21.15.99:0/3630843651 wait complete. 2024-03-30T00:37:32.567 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:32 smithi142 ceph-mon[41779]: pgmap v388: 97 pgs: 97 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 2.66k op/s 2024-03-30T00:37:32.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.560+0000 7fe67fc00700 1 Processor -- start 2024-03-30T00:37:32.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.560+0000 7fe67fc00700 1 -- start start 2024-03-30T00:37:32.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.561+0000 7fe67fc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe678100d80 0x7fe678198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.561+0000 7fe67fc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe678101f70 0x7fe678199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.561+0000 7fe67fc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe678102910 0x7fe6781a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.561+0000 7fe67fc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781a0a40 con 0x7fe678100d80 2024-03-30T00:37:32.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.561+0000 7fe67fc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781a0b80 con 0x7fe678101f70 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.561+0000 7fe67fc00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781a0d10 con 0x7fe678102910 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.561+0000 7fe67d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe678100d80 0x7fe678198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.561+0000 7fe677e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe678101f70 0x7fe678199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.561+0000 7fe67d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe678100d80 0x7fe678198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57960/0 (socket says 172.21.15.99:57960) 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.561+0000 7fe67d200700 1 -- 172.21.15.99:0/3289006955 learned_addr learned my addr 172.21.15.99:0/3289006955 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.561+0000 7fe67dc00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe678102910 0x7fe6781a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe67d200700 1 -- 172.21.15.99:0/3289006955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe678102910 msgr2=0x7fe6781a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe67d200700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe678102910 0x7fe6781a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe67d200700 1 -- 172.21.15.99:0/3289006955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe678101f70 msgr2=0x7fe678199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe67d200700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe678101f70 0x7fe678199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe67d200700 1 -- 172.21.15.99:0/3289006955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c009520 con 0x7fe678100d80 2024-03-30T00:37:32.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe677e00700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe678101f70 0x7fe678199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:32.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe67d200700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe678100d80 0x7fe678198f10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009230 tx=0x7fe66c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:32.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe675600700 1 -- 172.21.15.99:0/3289006955 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c01c070 con 0x7fe678100d80 2024-03-30T00:37:32.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe675600700 1 -- 172.21.15.99:0/3289006955 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe66c00bec0 con 0x7fe678100d80 2024-03-30T00:37:32.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe67fc00700 1 -- 172.21.15.99:0/3289006955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6781a0f90 con 0x7fe678100d80 2024-03-30T00:37:32.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe675600700 1 -- 172.21.15.99:0/3289006955 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c004930 con 0x7fe678100d80 2024-03-30T00:37:32.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.562+0000 7fe67fc00700 1 -- 172.21.15.99:0/3289006955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6781a1300 con 0x7fe678100d80 2024-03-30T00:37:32.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.565+0000 7fe675600700 1 -- 172.21.15.99:0/3289006955 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fe66c020070 con 0x7fe678100d80 2024-03-30T00:37:32.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.566+0000 7fe675600700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6600a1120 0x7fe6600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.567+0000 7fe675600700 1 -- 172.21.15.99:0/3289006955 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fe66c0cd640 con 0x7fe678100d80 2024-03-30T00:37:32.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.567+0000 7fe693627b80 1 Processor -- start 2024-03-30T00:37:32.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.567+0000 7fe677e00700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6600a1120 0x7fe6600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.567+0000 7fe693627b80 1 -- start start 2024-03-30T00:37:32.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.567+0000 7fe693627b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.567+0000 7fe693627b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0130 0x55626c6b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.567+0000 7fe693627b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6b0fd0 0x55626c6b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.567+0000 7fe693627b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c607990 con 0x55626c6b0130 2024-03-30T00:37:32.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.567+0000 7fe693627b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c605350 con 0x55626c6af7d0 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.567+0000 7fe693627b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c600c40 con 0x55626c6b0fd0 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe677e00700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6600a1120 0x7fe6600a35e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe664005fd0 tx=0x7fe664009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe67d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe67d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:53780/0 (socket says 172.21.15.99:53780) 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe67d200700 1 -- 172.21.15.99:0/2407906041 learned_addr learned my addr 172.21.15.99:0/2407906041 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe677e00700 1 --2- 172.21.15.99:0/2407906041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0130 0x55626c6b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe67dc00700 1 --2- 172.21.15.99:0/2407906041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6b0fd0 0x55626c6b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe67d200700 1 -- 172.21.15.99:0/2407906041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6b0fd0 msgr2=0x55626c6b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe67d200700 1 --2- 172.21.15.99:0/2407906041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6b0fd0 0x55626c6b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe67d200700 1 -- 172.21.15.99:0/2407906041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0130 msgr2=0x55626c6b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe67d200700 1 --2- 172.21.15.99:0/2407906041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0130 0x55626c6b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe67d200700 1 -- 172.21.15.99:0/2407906041 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55626c4823b0 con 0x55626c6af7d0 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.568+0000 7fe67d200700 1 --2- 172.21.15.99:0/2407906041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6afbf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0058b0 tx=0x7fe66c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49cd6eb5750a3344 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.569+0000 7fe66a000700 1 -- 172.21.15.99:0/2407906041 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c09aa40 con 0x55626c6af7d0 2024-03-30T00:37:32.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.569+0000 7fe66a000700 1 -- 172.21.15.99:0/2407906041 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe66c09e870 con 0x55626c6af7d0 2024-03-30T00:37:32.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.569+0000 7fe66a000700 1 -- 172.21.15.99:0/2407906041 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c09af00 con 0x55626c6af7d0 2024-03-30T00:37:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.569+0000 7fe693627b80 1 -- 172.21.15.99:0/2407906041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 msgr2=0x55626c6afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.569+0000 7fe693627b80 1 --2- 172.21.15.99:0/2407906041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6afbf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0058b0 tx=0x7fe66c009c30 comp rx=0 tx=0).stop 2024-03-30T00:37:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.569+0000 7fe693627b80 1 -- 172.21.15.99:0/2407906041 shutdown_connections 2024-03-30T00:37:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.569+0000 7fe693627b80 1 --2- 172.21.15.99:0/2407906041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6b0fd0 0x55626c6b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.569+0000 7fe693627b80 1 --2- 172.21.15.99:0/2407906041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6afbf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.569+0000 7fe693627b80 1 --2- 172.21.15.99:0/2407906041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0130 0x55626c6b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.569+0000 7fe693627b80 1 -- 172.21.15.99:0/2407906041 >> 172.21.15.99:0/2407906041 conn(0x55626c60e6a0 msgr2=0x55626c6aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.569+0000 7fe693627b80 1 -- 172.21.15.99:0/2407906041 shutdown_connections 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.570+0000 7fe693627b80 1 -- 172.21.15.99:0/2407906041 wait complete. 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.570+0000 7fe693627b80 1 -- start start 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.570+0000 7fe693627b80 1 Processor -- start 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe693627b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 0x55626c6b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe693627b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 0x55626c6b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe693627b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe693627b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c59cd80 con 0x55626c6b0fd0 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe693627b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c59ca20 con 0x55626c6b0130 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe693627b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c4846a0 con 0x55626c6af7d0 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe693627b80 10 client.0 _ll_register_callbacks cb 0x55626c5a6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe677e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 0x55626c6b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe677e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 0x55626c6b98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:53794/0 (socket says 172.21.15.99:53794) 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe677e00700 1 -- 172.21.15.99:0/2890747069 learned_addr learned my addr 172.21.15.99:0/2890747069 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe67dc00700 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe67d200700 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 0x55626c6b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe677e00700 1 -- 172.21.15.99:0/2890747069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 msgr2=0x55626c6b93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe677e00700 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 0x55626c6b93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe677e00700 1 -- 172.21.15.99:0/2890747069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 msgr2=0x55626c6c0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe677e00700 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6c0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.571+0000 7fe677e00700 1 -- 172.21.15.99:0/2890747069 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6781a1300 con 0x55626c6b0130 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.572+0000 7fe677e00700 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 0x55626c6b98e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe6640039b0 tx=0x7fe6640047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957df324a9d65aef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.572+0000 7fe65ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.572+0000 7fe65ea00700 1 -- 172.21.15.99:0/2890747069 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66401f990 con 0x55626c6b0130 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.572+0000 7fe693627b80 1 -- 172.21.15.99:0/2890747069 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55626c5c1380 con 0x55626c6b0130 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.572+0000 7fe65ea00700 1 -- 172.21.15.99:0/2890747069 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fe66401faf0 con 0x55626c6b0130 2024-03-30T00:37:32.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.572+0000 7fe65ea00700 1 -- 172.21.15.99:0/2890747069 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe66401fcb0 con 0x55626c6b0130 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.572+0000 7fe693627b80 10 client.25832 fetch_fsmap learned FSMap version 27 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.572+0000 7fe693627b80 1 -- 172.21.15.99:0/2890747069 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55626c42eb60 con 0x55626c6b0130 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.572+0000 7fe65ea00700 1 -- 172.21.15.99:0/2890747069 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55626c5c1380 con 0x55626c6b0130 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.573+0000 7fe65ea00700 1 -- 172.21.15.99:0/2890747069 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe664028440 con 0x55626c6b0130 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.573+0000 7fe693627b80 10 client.25832 fetch_fsmap finished waiting for FSMap version 27 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.573+0000 7fe693627b80 10 client.25832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.573+0000 7fe693627b80 20 client.25832 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.573+0000 7fe693627b80 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55626c6c51d0 0x55626c6c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.573+0000 7fe693627b80 4 client.25832 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.573+0000 7fe693627b80 1 -- 172.21.15.99:0/2890747069 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55626c5f9700 con 0x55626c6c51d0 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.573+0000 7fe67d200700 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55626c6c51d0 0x55626c6c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.574+0000 7fe65ea00700 10 client.25832 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.574+0000 7fe67d200700 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55626c6c51d0 0x55626c6c75f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d33ca65418a87972 server_cookie=af5aacf416b99b97 in_seq=0 out_seq=0 2024-03-30T00:37:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.574+0000 7fe65ea00700 1 -- 172.21.15.99:0/2890747069 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55626c5f9700 con 0x55626c6c51d0 2024-03-30T00:37:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.574+0000 7fe65ea00700 10 client.25832 handle_command_reply: tid=0 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 1 client.25832 shutdown 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 1 -- 172.21.15.99:0/2890747069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 msgr2=0x55626c6b98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 0x55626c6b98e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe6640039b0 tx=0x7fe6640047c0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 1 -- 172.21.15.99:0/2890747069 shutdown_connections 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55626c6c51d0 0x55626c6c75f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 0x55626c6b93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 1 --2- 172.21.15.99:0/2890747069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 0x55626c6b98e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe65ea00700 0 client.25832 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 1 -- 172.21.15.99:0/2890747069 >> 172.21.15.99:0/2890747069 conn(0x55626c60e6a0 msgr2=0x55626c6ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 1 -- 172.21.15.99:0/2890747069 shutdown_connections 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 1 -- 172.21.15.99:0/2890747069 wait complete. 2024-03-30T00:37:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.575+0000 7fe693627b80 20 client.25832 trim_cache size 0 max 16384 2024-03-30T00:37:32.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.595+0000 7fe693627b80 1 Processor -- start 2024-03-30T00:37:32.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.595+0000 7fe693627b80 1 -- start start 2024-03-30T00:37:32.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe693627b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 0x55626c6b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe693627b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 0x55626c6c0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe693627b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe693627b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c6ba2a0 con 0x55626c6b0fd0 2024-03-30T00:37:32.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe693627b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c6b8f10 con 0x55626c6b0130 2024-03-30T00:37:32.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe693627b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c6c2680 con 0x55626c6af7d0 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe67dc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe67d200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 0x55626c6b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe677e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 0x55626c6c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe67d200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 0x55626c6b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:44344/0 (socket says 172.21.15.99:44344) 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe67d200700 1 -- 172.21.15.99:0/3030637098 learned_addr learned my addr 172.21.15.99:0/3030637098 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe67d200700 1 -- 172.21.15.99:0/3030637098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 msgr2=0x55626c6c0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe67d200700 1 --2- 172.21.15.99:0/3030637098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 0x55626c6c0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe67d200700 1 -- 172.21.15.99:0/3030637098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 msgr2=0x55626c6a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe67d200700 1 --2- 172.21.15.99:0/3030637098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.596+0000 7fe67d200700 1 -- 172.21.15.99:0/3030637098 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55626c5c6ea0 con 0x55626c6af7d0 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.597+0000 7fe67d200700 1 --2- 172.21.15.99:0/3030637098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 0x55626c6b13b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe66c09a9d0 tx=0x7fe66c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d6053cf88bc2abc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:32.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.597+0000 7fe66aa00700 1 -- 172.21.15.99:0/3030637098 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0a2eb0 con 0x55626c6af7d0 2024-03-30T00:37:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.597+0000 7fe66aa00700 1 -- 172.21.15.99:0/3030637098 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe66c098de0 con 0x55626c6af7d0 2024-03-30T00:37:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.597+0000 7fe66aa00700 1 -- 172.21.15.99:0/3030637098 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c09af00 con 0x55626c6af7d0 2024-03-30T00:37:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.597+0000 7fe693627b80 1 -- 172.21.15.99:0/3030637098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 msgr2=0x55626c6b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.597+0000 7fe693627b80 1 --2- 172.21.15.99:0/3030637098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 0x55626c6b13b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe66c09a9d0 tx=0x7fe66c009260 comp rx=0 tx=0).stop 2024-03-30T00:37:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.597+0000 7fe693627b80 1 -- 172.21.15.99:0/3030637098 shutdown_connections 2024-03-30T00:37:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.597+0000 7fe693627b80 1 --2- 172.21.15.99:0/3030637098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.597+0000 7fe693627b80 1 --2- 172.21.15.99:0/3030637098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6af7d0 0x55626c6b13b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.597+0000 7fe693627b80 1 --2- 172.21.15.99:0/3030637098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6b0130 0x55626c6c0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.597+0000 7fe693627b80 1 -- 172.21.15.99:0/3030637098 >> 172.21.15.99:0/3030637098 conn(0x55626c60e6a0 msgr2=0x55626c6b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.598+0000 7fe693627b80 1 -- 172.21.15.99:0/3030637098 shutdown_connections 2024-03-30T00:37:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.598+0000 7fe693627b80 1 -- 172.21.15.99:0/3030637098 wait complete. 2024-03-30T00:37:32.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.598+0000 7fe693627b80 1 -- start start 2024-03-30T00:37:32.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.598+0000 7fe693627b80 1 Processor -- start 2024-03-30T00:37:32.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe693627b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6beae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe693627b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6b0130 0x55626c6bf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe693627b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe693627b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c6c03e0 con 0x55626c6b0fd0 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe693627b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c6c0550 con 0x55626c6af7d0 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe693627b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55626c6c06c0 con 0x55626c6b0130 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe693627b80 10 client.0 _ll_register_callbacks cb 0x55626c5a6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe67d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6beae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe677e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6b0130 0x55626c6bf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe67d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6beae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:53824/0 (socket says 172.21.15.99:53824) 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe67d200700 1 -- 172.21.15.99:0/2370156525 learned_addr learned my addr 172.21.15.99:0/2370156525 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.599+0000 7fe67dc00700 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe67d200700 1 -- 172.21.15.99:0/2370156525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6b0130 msgr2=0x55626c6bf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:32.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe67d200700 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6b0130 0x55626c6bf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe67d200700 1 -- 172.21.15.99:0/2370156525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 msgr2=0x55626c6bfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe67d200700 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6bfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe67d200700 1 -- 172.21.15.99:0/2370156525 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55626c4823b0 con 0x55626c6af7d0 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe677e00700 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6b0130 0x55626c6bf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe67d200700 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6beae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0974b0 tx=0x7fe66c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36b288ce2b9df299 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe65ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe65ea00700 1 -- 172.21.15.99:0/2370156525 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c09a580 con 0x55626c6af7d0 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe65ea00700 1 -- 172.21.15.99:0/2370156525 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fe66c09aa40 con 0x55626c6af7d0 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe693627b80 1 -- 172.21.15.99:0/2370156525 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe66c09e870 con 0x55626c6af7d0 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe65ea00700 1 -- 172.21.15.99:0/2370156525 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe66c09aeb0 con 0x55626c6af7d0 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe693627b80 10 client.25838 fetch_fsmap learned FSMap version 27 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.600+0000 7fe693627b80 1 -- 172.21.15.99:0/2370156525 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55626c486cd0 con 0x55626c6af7d0 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.601+0000 7fe65ea00700 1 -- 172.21.15.99:0/2370156525 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe66c09e870 con 0x55626c6af7d0 2024-03-30T00:37:32.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.601+0000 7fe65ea00700 1 -- 172.21.15.99:0/2370156525 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe66c005040 con 0x55626c6af7d0 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.601+0000 7fe693627b80 10 client.25838 fetch_fsmap finished waiting for FSMap version 27 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.601+0000 7fe693627b80 10 client.25838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.601+0000 7fe693627b80 20 client.25838 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.601+0000 7fe693627b80 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55626c719c90 0x55626c71c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.601+0000 7fe693627b80 4 client.25838 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.601+0000 7fe693627b80 1 -- 172.21.15.99:0/2370156525 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55626c44fcf0 con 0x55626c719c90 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.601+0000 7fe677e00700 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55626c719c90 0x55626c71c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe677e00700 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55626c719c90 0x55626c71c080 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641f9f229f94566b server_cookie=79dae05dba16ba69 in_seq=0 out_seq=0 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe65ea00700 10 client.25838 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe65ea00700 1 -- 172.21.15.99:0/2370156525 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55626c44fcf0 con 0x55626c719c90 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe65ea00700 10 client.25838 handle_command_reply: tid=0 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe693627b80 1 client.25838 shutdown 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe693627b80 1 -- 172.21.15.99:0/2370156525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 msgr2=0x55626c6beae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe693627b80 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6beae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0974b0 tx=0x7fe66c005dc0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe693627b80 1 -- 172.21.15.99:0/2370156525 shutdown_connections 2024-03-30T00:37:32.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe693627b80 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x55626c719c90 0x55626c71c080 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe693627b80 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55626c6b0fd0 0x55626c6bfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe693627b80 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55626c6af7d0 0x55626c6beae0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe693627b80 1 --2- 172.21.15.99:0/2370156525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55626c6b0130 0x55626c6bf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe65ea00700 0 client.25838 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:32.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe693627b80 1 -- 172.21.15.99:0/2370156525 >> 172.21.15.99:0/2370156525 conn(0x55626c60e6a0 msgr2=0x55626c6b8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:32.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe693627b80 1 -- 172.21.15.99:0/2370156525 shutdown_connections 2024-03-30T00:37:32.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.602+0000 7fe693627b80 1 -- 172.21.15.99:0/2370156525 wait complete. 2024-03-30T00:37:32.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe693627b80 20 client.25838 trim_cache size 0 max 16384 2024-03-30T00:37:32.613 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:37:32.614 INFO:teuthology.orchestra.run.smithi099.stdout: "status": "no active scrubs running", 2024-03-30T00:37:32.614 INFO:teuthology.orchestra.run.smithi099.stdout: "scrubs": {} 2024-03-30T00:37:32.614 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:37:32.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 -- 172.21.15.99:0/3289006955 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6600a1120 msgr2=0x7fe6600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6600a1120 0x7fe6600a35e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe664005fd0 tx=0x7fe664009310 comp rx=0 tx=0).stop 2024-03-30T00:37:32.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 -- 172.21.15.99:0/3289006955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe678100d80 msgr2=0x7fe678198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe678100d80 0x7fe678198f10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009230 tx=0x7fe66c00b790 comp rx=0 tx=0).stop 2024-03-30T00:37:32.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 -- 172.21.15.99:0/3289006955 shutdown_connections 2024-03-30T00:37:32.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe678102910 0x7fe6781a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe678100d80 0x7fe678198f10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fe6600a1120 0x7fe6600a35e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 --2- 172.21.15.99:0/3289006955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe678101f70 0x7fe678199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 -- 172.21.15.99:0/3289006955 >> 172.21.15.99:0/3289006955 conn(0x7fe6780fc2f0 msgr2=0x7fe678107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:32.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 -- 172.21.15.99:0/3289006955 shutdown_connections 2024-03-30T00:37:32.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.603+0000 7fe67fc00700 1 -- 172.21.15.99:0/3289006955 wait complete. 2024-03-30T00:37:32.627 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T00:37:32.627 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T00:37:32.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.813+0000 7f8822c00700 1 -- 172.21.15.99:0/2751555639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f881c1016e0 msgr2=0x7f881c103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.813+0000 7f8822c00700 1 --2- 172.21.15.99:0/2751555639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f881c1016e0 0x7f881c103ad0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f880c009990 tx=0x7f880c009ca0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.813+0000 7f8822c00700 1 -- 172.21.15.99:0/2751555639 shutdown_connections 2024-03-30T00:37:32.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.813+0000 7f8822c00700 1 --2- 172.21.15.99:0/2751555639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f881c106940 0x7f881c108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.813+0000 7f8822c00700 1 --2- 172.21.15.99:0/2751555639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f881c1016e0 0x7f881c103ad0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.813+0000 7f8822c00700 1 --2- 172.21.15.99:0/2751555639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f881c104010 0x7f881c106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.813+0000 7f8822c00700 1 -- 172.21.15.99:0/2751555639 >> 172.21.15.99:0/2751555639 conn(0x7f881c0fb0d0 msgr2=0x7f881c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:32.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.814+0000 7f8822c00700 1 -- 172.21.15.99:0/2751555639 shutdown_connections 2024-03-30T00:37:32.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.814+0000 7f8822c00700 1 -- 172.21.15.99:0/2751555639 wait complete. 2024-03-30T00:37:32.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f8822c00700 1 Processor -- start 2024-03-30T00:37:32.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f8822c00700 1 -- start start 2024-03-30T00:37:32.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f8822c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f881c1016e0 0x7f881c198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f8822c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f881c104010 0x7f881c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f8822c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f881c106940 0x7f881c1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f8822c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c1a07c0 con 0x7f881c1016e0 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f8822c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c1a0930 con 0x7f881c106940 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f8822c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c1a0af0 con 0x7f881c104010 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f881be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f881c1016e0 0x7f881c198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f8820c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f881c106940 0x7f881c1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f881be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f881c1016e0 0x7f881c198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58008/0 (socket says 172.21.15.99:58008) 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.815+0000 7f881be00700 1 -- 172.21.15.99:0/1214790760 learned_addr learned my addr 172.21.15.99:0/1214790760 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f881b400700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f881c104010 0x7f881c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f8820c00700 1 -- 172.21.15.99:0/1214790760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f881c104010 msgr2=0x7f881c199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f8820c00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f881c104010 0x7f881c199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f8820c00700 1 -- 172.21.15.99:0/1214790760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f881c1016e0 msgr2=0x7f881c198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f8820c00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f881c1016e0 0x7f881c198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f8820c00700 1 -- 172.21.15.99:0/1214790760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c0095f0 con 0x7f881c106940 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f881be00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f881c1016e0 0x7f881c198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:32.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f8820c00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f881c106940 0x7f881c1a01f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f881400efc0 tx=0x7f881400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:32.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f8818c00700 1 -- 172.21.15.99:0/1214790760 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814018070 con 0x7f881c106940 2024-03-30T00:37:32.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f8818c00700 1 -- 172.21.15.99:0/1214790760 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f881400f040 con 0x7f881c106940 2024-03-30T00:37:32.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f8822c00700 1 -- 172.21.15.99:0/1214790760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881c1a0d70 con 0x7f881c106940 2024-03-30T00:37:32.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.816+0000 7f8818c00700 1 -- 172.21.15.99:0/1214790760 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881400cba0 con 0x7f881c106940 2024-03-30T00:37:32.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.817+0000 7f8822c00700 1 -- 172.21.15.99:0/1214790760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881c1a1210 con 0x7f881c106940 2024-03-30T00:37:32.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.818+0000 7f8822c00700 1 -- 172.21.15.99:0/1214790760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f881c108510 con 0x7f881c106940 2024-03-30T00:37:32.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.819+0000 7f8818c00700 1 -- 172.21.15.99:0/1214790760 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f8814003a00 con 0x7f881c106940 2024-03-30T00:37:32.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.821+0000 7f8818c00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88000a11a0 0x7f88000a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:32.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.821+0000 7f8818c00700 1 -- 172.21.15.99:0/1214790760 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f88140cc550 con 0x7f881c106940 2024-03-30T00:37:32.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.821+0000 7f881be00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88000a11a0 0x7f88000a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:32.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.821+0000 7f881be00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88000a11a0 0x7f88000a3660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f880c00b2e0 tx=0x7f880c00b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:32.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:32.824+0000 7f8818c00700 1 -- 172.21.15.99:0/1214790760 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8814095c90 con 0x7f881c106940 2024-03-30T00:37:33.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.090+0000 7f8822c00700 1 -- 172.21.15.99:0/1214790760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f881c067a10 con 0x7f881c106940 2024-03-30T00:37:33.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.091+0000 7f8818c00700 1 -- 172.21.15.99:0/1214790760 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10863 (secure 0 0 0) 0x7f88140992b0 con 0x7f881c106940 2024-03-30T00:37:33.097 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:37:33.097 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14607,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/380315573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":380315573},{"type":"v1","addr":"172.21.15.99:6835","nonce":380315573}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4277780379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4277780379},{"type":"v1","addr":"172.21.15.99:6837","nonce":4277780379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2189606703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2189606703},{"type":"v1","addr":"172.21.15.161:6837","nonce":2189606703}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":14667,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/364916855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":364916855},{"type":"v1","addr":"172.21.15.142:6837","nonce":364916855}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":14685,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/757502672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":757502672},{"type":"v1","addr":"172.21.15.99:6839","nonce":757502672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/3055946850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3055946850},{"type":"v1","addr":"172.21.15.161:6833","nonce":3055946850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T00:33:15.648899+0000","modified":"2024-03-30T00:37:32.178907+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24559,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/780858221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":780858221},{"type":"v1","addr":"172.21.15.142:6833","nonce":780858221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6835/80094727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":80094727},{"type":"v1","addr":"172.21.15.161:6835","nonce":80094727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24503":{"gid":24503,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/1641167461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1641167461},{"type":"v1","addr":"172.21.15.142:6835","nonce":1641167461}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24544":{"gid":24544,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6839/1486341215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":1486341215},{"type":"v1","addr":"172.21.15.161:6839","nonce":1486341215}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24559":{"gid":24559,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":62,"addr":"172.21.15.99:6841/1391823621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6840","nonce":1391823621},{"type":"v1","addr":"172.21.15.99:6841","nonce":1391823621}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6839/2862512313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":2862512313},{"type":"v1","addr":"172.21.15.142:6839","nonce":2862512313}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T00:37:33.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 -- 172.21.15.99:0/1214790760 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88000a11a0 msgr2=0x7f88000a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88000a11a0 0x7f88000a3660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f880c00b2e0 tx=0x7f880c00b610 comp rx=0 tx=0).stop 2024-03-30T00:37:33.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 -- 172.21.15.99:0/1214790760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f881c106940 msgr2=0x7f881c1a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f881c106940 0x7f881c1a01f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f881400efc0 tx=0x7f881400c3a0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 -- 172.21.15.99:0/1214790760 shutdown_connections 2024-03-30T00:37:33.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f88000a11a0 0x7f88000a3660 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f881c106940 0x7f881c1a01f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f881c1016e0 0x7f881c198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 --2- 172.21.15.99:0/1214790760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f881c104010 0x7f881c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 -- 172.21.15.99:0/1214790760 >> 172.21.15.99:0/1214790760 conn(0x7f881c0fb0d0 msgr2=0x7f881c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 -- 172.21.15.99:0/1214790760 shutdown_connections 2024-03-30T00:37:33.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.094+0000 7f8822c00700 1 -- 172.21.15.99:0/1214790760 wait complete. 2024-03-30T00:37:33.102 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-03-30T00:37:33.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T00:37:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.318+0000 7f6a33e00700 1 -- 172.21.15.99:0/4218490296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a34108a80 msgr2=0x7f6a3410af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.318+0000 7f6a33e00700 1 --2- 172.21.15.99:0/4218490296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a34108a80 0x7f6a3410af10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00b0a0 tx=0x7f6a2c00b3b0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.318+0000 7f6a33e00700 1 -- 172.21.15.99:0/4218490296 shutdown_connections 2024-03-30T00:37:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.318+0000 7f6a33e00700 1 --2- 172.21.15.99:0/4218490296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a34108a80 0x7f6a3410af10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.318+0000 7f6a33e00700 1 --2- 172.21.15.99:0/4218490296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a34074df0 0x7f6a34073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.318+0000 7f6a33e00700 1 --2- 172.21.15.99:0/4218490296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a34073810 0x7f6a34073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.318+0000 7f6a33e00700 1 -- 172.21.15.99:0/4218490296 >> 172.21.15.99:0/4218490296 conn(0x7f6a340fc110 msgr2=0x7f6a340fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.318+0000 7f6a33e00700 1 -- 172.21.15.99:0/4218490296 shutdown_connections 2024-03-30T00:37:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.318+0000 7f6a33e00700 1 -- 172.21.15.99:0/4218490296 wait complete. 2024-03-30T00:37:33.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.319+0000 7f6a33e00700 1 Processor -- start 2024-03-30T00:37:33.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.319+0000 7f6a33e00700 1 -- start start 2024-03-30T00:37:33.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.319+0000 7f6a33e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a34073810 0x7f6a34198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.319+0000 7f6a33e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a34074df0 0x7f6a341992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.319+0000 7f6a33e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a34108a80 0x7f6a341a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.319+0000 7f6a33e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341a0840 con 0x7f6a34108a80 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.319+0000 7f6a33e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341a09b0 con 0x7f6a34074df0 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.319+0000 7f6a33e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341a0b70 con 0x7f6a34073810 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a33400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a34108a80 0x7f6a341a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a32a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a34073810 0x7f6a34198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a33400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a34108a80 0x7f6a341a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58038/0 (socket says 172.21.15.99:58038) 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a32a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a34073810 0x7f6a34198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:44392/0 (socket says 172.21.15.99:44392) 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a33400700 1 -- 172.21.15.99:0/4239224090 learned_addr learned my addr 172.21.15.99:0/4239224090 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a32000700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a34074df0 0x7f6a341992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a33400700 1 -- 172.21.15.99:0/4239224090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a34073810 msgr2=0x7f6a34198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a33400700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a34073810 0x7f6a34198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a33400700 1 -- 172.21.15.99:0/4239224090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a34074df0 msgr2=0x7f6a341992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a33400700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a34074df0 0x7f6a341992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a33400700 1 -- 172.21.15.99:0/4239224090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c009d00 con 0x7f6a34108a80 2024-03-30T00:37:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a32000700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a34074df0 0x7f6a341992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.320+0000 7f6a33400700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a34108a80 0x7f6a341a0270 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00b780 tx=0x7f6a2c00bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.321+0000 7f6a2b400700 1 -- 172.21.15.99:0/4239224090 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00e040 con 0x7f6a34108a80 2024-03-30T00:37:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.321+0000 7f6a2b400700 1 -- 172.21.15.99:0/4239224090 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a2c009480 con 0x7f6a34108a80 2024-03-30T00:37:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.321+0000 7f6a33e00700 1 -- 172.21.15.99:0/4239224090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a341a0df0 con 0x7f6a34108a80 2024-03-30T00:37:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.321+0000 7f6a2b400700 1 -- 172.21.15.99:0/4239224090 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0052c0 con 0x7f6a34108a80 2024-03-30T00:37:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.321+0000 7f6a33e00700 1 -- 172.21.15.99:0/4239224090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a341a11c0 con 0x7f6a34108a80 2024-03-30T00:37:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.324+0000 7f6a2b400700 1 -- 172.21.15.99:0/4239224090 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f6a2c005420 con 0x7f6a34108a80 2024-03-30T00:37:33.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.325+0000 7f6a4d7b9b80 1 Processor -- start 2024-03-30T00:37:33.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.325+0000 7f6a4d7b9b80 1 -- start start 2024-03-30T00:37:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.325+0000 7f6a2b400700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6a180a11a0 0x7f6a180a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.325+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b799e0 0x558dc0b79e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.325+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7a340 0x558dc0b7abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.325+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 0x558dc0b7b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.325+0000 7f6a2b400700 1 -- 172.21.15.99:0/4239224090 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f6a2c0cdd40 con 0x7f6a34108a80 2024-03-30T00:37:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.325+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0ad1990 con 0x558dc0b7b190 2024-03-30T00:37:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.325+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0acf350 con 0x558dc0b7a340 2024-03-30T00:37:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.325+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0acac40 con 0x558dc0b799e0 2024-03-30T00:37:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.325+0000 7f6a32000700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6a180a11a0 0x7f6a180a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7a340 0x558dc0b7abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7a340 0x558dc0b7abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:53870/0 (socket says 172.21.15.99:53870) 2024-03-30T00:37:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a33400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 0x558dc0b7b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32000700 1 -- 172.21.15.99:0/3983855837 learned_addr learned my addr 172.21.15.99:0/3983855837 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32a00700 1 --2- 172.21.15.99:0/3983855837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b799e0 0x558dc0b79e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32a00700 1 -- 172.21.15.99:0/3983855837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7a340 msgr2=0x558dc0b7abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32a00700 1 --2- 172.21.15.99:0/3983855837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7a340 0x558dc0b7abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32a00700 1 -- 172.21.15.99:0/3983855837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 msgr2=0x558dc0b7b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32a00700 1 --2- 172.21.15.99:0/3983855837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 0x558dc0b7b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32a00700 1 -- 172.21.15.99:0/3983855837 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc094c3b0 con 0x558dc0b799e0 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a33400700 1 --2- 172.21.15.99:0/3983855837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 0x558dc0b7b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32000700 1 --2- 172.21.15.99:0/3983855837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7a340 0x558dc0b7abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32000700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6a180a11a0 0x7f6a180a3660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6a20014a80 tx=0x7f6a2000bcb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.326+0000 7f6a32a00700 1 --2- 172.21.15.99:0/3983855837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b799e0 0x558dc0b79e00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009c80 tx=0x7f6a24009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feec17e7807e370a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.327+0000 7f6a1fe00700 1 -- 172.21.15.99:0/3983855837 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24004310 con 0x558dc0b799e0 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.327+0000 7f6a1fe00700 1 -- 172.21.15.99:0/3983855837 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a24011dc0 con 0x558dc0b799e0 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.327+0000 7f6a1fe00700 1 -- 172.21.15.99:0/3983855837 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24003dc0 con 0x558dc0b799e0 2024-03-30T00:37:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.327+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/3983855837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b799e0 msgr2=0x558dc0b79e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.327+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/3983855837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b799e0 0x558dc0b79e00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009c80 tx=0x7f6a24009f90 comp rx=0 tx=0).stop 2024-03-30T00:37:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.328+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/3983855837 shutdown_connections 2024-03-30T00:37:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.328+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/3983855837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 0x558dc0b7b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.328+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/3983855837 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b799e0 0x558dc0b79e00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.328+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/3983855837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7a340 0x558dc0b7abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.328+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/3983855837 >> 172.21.15.99:0/3983855837 conn(0x558dc0ad86a0 msgr2=0x558dc0b74ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.328+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/3983855837 shutdown_connections 2024-03-30T00:37:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.328+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/3983855837 wait complete. 2024-03-30T00:37:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.328+0000 7f6a4d7b9b80 1 -- start start 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.328+0000 7f6a4d7b9b80 1 Processor -- start 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b799e0 0x558dc0b83820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7a340 0x558dc0b83d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7b190 0x558dc0b8ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0a66d80 con 0x558dc0b7a340 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0a66a20 con 0x558dc0b799e0 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc094e6a0 con 0x558dc0b7b190 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a4d7b9b80 10 client.0 _ll_register_callbacks cb 0x558dc0a70470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a32000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7a340 0x558dc0b83d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a32000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7a340 0x558dc0b83d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58058/0 (socket says 172.21.15.99:58058) 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a32000700 1 -- 172.21.15.99:0/2159749043 learned_addr learned my addr 172.21.15.99:0/2159749043 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a32a00700 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b799e0 0x558dc0b83820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a33400700 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7b190 0x558dc0b8ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a32000700 1 -- 172.21.15.99:0/2159749043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7b190 msgr2=0x558dc0b8ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a32000700 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7b190 0x558dc0b8ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a32000700 1 -- 172.21.15.99:0/2159749043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b799e0 msgr2=0x558dc0b83820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.329+0000 7f6a32000700 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b799e0 0x558dc0b83820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.330+0000 7f6a32000700 1 -- 172.21.15.99:0/2159749043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a341a11c0 con 0x558dc0b7a340 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.330+0000 7f6a32000700 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7a340 0x558dc0b83d60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6a200143c0 tx=0x7f6a200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629cbf9db75ea96c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.330+0000 7f6a1cc00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.330+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2159749043 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20004220 con 0x558dc0b7a340 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.330+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2159749043 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f6a20004380 con 0x558dc0b7a340 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.330+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2159749043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558dc0a8b380 con 0x558dc0b7a340 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.330+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2159749043 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a20004540 con 0x558dc0b7a340 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.330+0000 7f6a4d7b9b80 10 client.16161 fetch_fsmap learned FSMap version 27 2024-03-30T00:37:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.330+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2159749043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f6a240098e0 con 0x558dc0b7a340 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.331+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2159749043 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558dc0a8b380 con 0x558dc0b7a340 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.331+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2159749043 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6a20027030 con 0x558dc0b7a340 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.331+0000 7f6a4d7b9b80 10 client.16161 fetch_fsmap finished waiting for FSMap version 27 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.331+0000 7f6a4d7b9b80 10 client.16161 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.331+0000 7f6a4d7b9b80 20 client.16161 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.331+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x558dc0b8f570 0x558dc0b91990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.331+0000 7f6a4d7b9b80 4 client.16161 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.331+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2159749043 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dc0ac3700 con 0x558dc0b8f570 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.331+0000 7f6a32a00700 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x558dc0b8f570 0x558dc0b91990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.332+0000 7f6a32a00700 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x558dc0b8f570 0x558dc0b91990 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2831cf2c9b6a54e4 server_cookie=d0f36d017728adc1 in_seq=0 out_seq=0 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.332+0000 7f6a1cc00700 10 client.16161 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2159749043 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558dc0ac3700 con 0x558dc0b8f570 2024-03-30T00:37:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a1cc00700 10 client.16161 handle_command_reply: tid=0 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 1 client.16161 shutdown 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2159749043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7a340 msgr2=0x558dc0b83d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7a340 0x558dc0b83d60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6a200143c0 tx=0x7f6a200047c0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2159749043 shutdown_connections 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x558dc0b8f570 0x558dc0b91990 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7b190 0x558dc0b8ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b799e0 0x558dc0b83820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a1cc00700 0 client.16161 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2159749043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7a340 0x558dc0b83d60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2159749043 >> 172.21.15.99:0/2159749043 conn(0x558dc0ad86a0 msgr2=0x558dc0b783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2159749043 shutdown_connections 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2159749043 wait complete. 2024-03-30T00:37:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.333+0000 7f6a4d7b9b80 20 client.16161 trim_cache size 0 max 16384 2024-03-30T00:37:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.346+0000 7f6a4d7b9b80 1 Processor -- start 2024-03-30T00:37:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a4d7b9b80 1 -- start start 2024-03-30T00:37:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b799e0 0x558dc0b82d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7a340 0x558dc0b8ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7b190 0x558dc0b7b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0b7b000 con 0x558dc0b799e0 2024-03-30T00:37:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0b8bf20 con 0x558dc0b7b190 2024-03-30T00:37:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0b833c0 con 0x558dc0b7a340 2024-03-30T00:37:33.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a32a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b799e0 0x558dc0b82d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a32a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b799e0 0x558dc0b82d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58068/0 (socket says 172.21.15.99:58068) 2024-03-30T00:37:33.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a32a00700 1 -- 172.21.15.99:0/165682222 learned_addr learned my addr 172.21.15.99:0/165682222 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a32000700 1 --2- 172.21.15.99:0/165682222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7a340 0x558dc0b8ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a33400700 1 --2- 172.21.15.99:0/165682222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7b190 0x558dc0b7b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a32a00700 1 -- 172.21.15.99:0/165682222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7a340 msgr2=0x558dc0b8ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a32a00700 1 --2- 172.21.15.99:0/165682222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7a340 0x558dc0b8ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a32a00700 1 -- 172.21.15.99:0/165682222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7b190 msgr2=0x558dc0b7b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a32a00700 1 --2- 172.21.15.99:0/165682222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7b190 0x558dc0b7b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.347+0000 7f6a32a00700 1 -- 172.21.15.99:0/165682222 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc08f8b60 con 0x558dc0b799e0 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.348+0000 7f6a32a00700 1 --2- 172.21.15.99:0/165682222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b799e0 0x558dc0b82d30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x558dc0b76050 tx=0x7f6a24009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f34a6e416a0a438 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.348+0000 7f6a28c00700 1 -- 172.21.15.99:0/165682222 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24011a30 con 0x558dc0b799e0 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.348+0000 7f6a28c00700 1 -- 172.21.15.99:0/165682222 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a24011b90 con 0x558dc0b799e0 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.348+0000 7f6a28c00700 1 -- 172.21.15.99:0/165682222 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2401a6a0 con 0x558dc0b799e0 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.349+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/165682222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b799e0 msgr2=0x558dc0b82d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.349+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/165682222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b799e0 0x558dc0b82d30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x558dc0b76050 tx=0x7f6a24009260 comp rx=0 tx=0).stop 2024-03-30T00:37:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.349+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/165682222 shutdown_connections 2024-03-30T00:37:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.349+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/165682222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b7b190 0x558dc0b7b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.349+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/165682222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b799e0 0x558dc0b82d30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.349+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/165682222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7a340 0x558dc0b8ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.349+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/165682222 >> 172.21.15.99:0/165682222 conn(0x558dc0ad86a0 msgr2=0x558dc0b6d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.349+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/165682222 shutdown_connections 2024-03-30T00:37:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.349+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/165682222 wait complete. 2024-03-30T00:37:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.349+0000 7f6a4d7b9b80 1 -- start start 2024-03-30T00:37:33.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.349+0000 7f6a4d7b9b80 1 Processor -- start 2024-03-30T00:37:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b799e0 0x558dc0b8a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7a340 0x558dc0b8ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a4d7b9b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 0x558dc0b89040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0b89720 con 0x558dc0b7b190 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0b89890 con 0x558dc0b799e0 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a4d7b9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc0b89a50 con 0x558dc0b7a340 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a4d7b9b80 10 client.0 _ll_register_callbacks cb 0x558dc0a70470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a33400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 0x558dc0b89040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a32000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7a340 0x558dc0b8ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a33400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 0x558dc0b89040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58084/0 (socket says 172.21.15.99:58084) 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a33400700 1 -- 172.21.15.99:0/2630857269 learned_addr learned my addr 172.21.15.99:0/2630857269 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a32a00700 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b799e0 0x558dc0b8a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a33400700 1 -- 172.21.15.99:0/2630857269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7a340 msgr2=0x558dc0b8ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a33400700 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7a340 0x558dc0b8ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a33400700 1 -- 172.21.15.99:0/2630857269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b799e0 msgr2=0x558dc0b8a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a33400700 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b799e0 0x558dc0b8a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a33400700 1 -- 172.21.15.99:0/2630857269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2401e040 con 0x558dc0b7b190 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.350+0000 7f6a32a00700 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b799e0 0x558dc0b8a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a33400700 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 0x558dc0b89040 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0a3000 tx=0x7f6a2c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609a106fc2f37a10 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a1cc00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2630857269 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c09b5a0 con 0x558dc0b7b190 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2630857269 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f6a2c0a3530 con 0x558dc0b7b190 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2630857269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a24011dc0 con 0x558dc0b7b190 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2630857269 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a2c003e10 con 0x558dc0b7b190 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a4d7b9b80 10 client.16167 fetch_fsmap learned FSMap version 27 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2630857269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558dc0a90ea0 con 0x558dc0b7b190 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2630857269 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a24011dc0 con 0x558dc0b7b190 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2630857269 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6a2c0a31c0 con 0x558dc0b7b190 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a4d7b9b80 10 client.16167 fetch_fsmap finished waiting for FSMap version 27 2024-03-30T00:37:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a4d7b9b80 10 client.16167 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a4d7b9b80 20 client.16167 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x558dc0be4740 0x558dc0be6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a4d7b9b80 4 client.16167 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.351+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2630857269 --> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558dc0919cf0 con 0x558dc0be4740 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.352+0000 7f6a32a00700 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x558dc0be4740 0x558dc0be6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.352+0000 7f6a32a00700 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x558dc0be4740 0x558dc0be6b60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b690e7fc6c9c0cf2 server_cookie=291fb0769a76533c in_seq=0 out_seq=0 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.352+0000 7f6a1cc00700 10 client.16167 ms_handle_connect on v2:172.21.15.161:6834/80094727 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.352+0000 7f6a1cc00700 1 -- 172.21.15.99:0/2630857269 <== mds.2 v2:172.21.15.161:6834/80094727 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558dc0919cf0 con 0x558dc0be4740 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.352+0000 7f6a1cc00700 10 client.16167 handle_command_reply: tid=0 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.352+0000 7f6a4d7b9b80 1 client.16167 shutdown 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2630857269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 msgr2=0x558dc0b89040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 0x558dc0b89040 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0a3000 tx=0x7f6a2c00b3b0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2630857269 shutdown_connections 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.161:6834/80094727,v1:172.21.15.161:6835/80094727] conn(0x558dc0be4740 0x558dc0be6b60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dc0b7b190 0x558dc0b89040 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558dc0b799e0 0x558dc0b8a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a1cc00700 0 client.16167 ms_handle_reset on v2:172.21.15.161:6834/80094727 2024-03-30T00:37:33.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a4d7b9b80 1 --2- 172.21.15.99:0/2630857269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558dc0b7a340 0x558dc0b8ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2630857269 >> 172.21.15.99:0/2630857269 conn(0x558dc0ad86a0 msgr2=0x558dc0b7e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2630857269 shutdown_connections 2024-03-30T00:37:33.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a4d7b9b80 1 -- 172.21.15.99:0/2630857269 wait complete. 2024-03-30T00:37:33.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.353+0000 7f6a4d7b9b80 20 client.16167 trim_cache size 0 max 16384 2024-03-30T00:37:33.364 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:37:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 -- 172.21.15.99:0/4239224090 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6a180a11a0 msgr2=0x7f6a180a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6a180a11a0 0x7f6a180a3660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6a20014a80 tx=0x7f6a2000bcb0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 -- 172.21.15.99:0/4239224090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a34108a80 msgr2=0x7f6a341a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a34108a80 0x7f6a341a0270 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00b780 tx=0x7f6a2c00bcc0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 -- 172.21.15.99:0/4239224090 shutdown_connections 2024-03-30T00:37:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a34108a80 0x7f6a341a0270 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6a34073810 0x7f6a34198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f6a180a11a0 0x7f6a180a3660 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 --2- 172.21.15.99:0/4239224090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a34074df0 0x7f6a341992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 -- 172.21.15.99:0/4239224090 >> 172.21.15.99:0/4239224090 conn(0x7f6a340fc110 msgr2=0x7f6a34104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 -- 172.21.15.99:0/4239224090 shutdown_connections 2024-03-30T00:37:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.354+0000 7f6a33e00700 1 -- 172.21.15.99:0/4239224090 wait complete. 2024-03-30T00:37:33.375 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:33 smithi099 ceph-mon[38079]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:37:33.375 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:33 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1214790760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:37:33.378 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T00:37:33.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:33 smithi161 ceph-mon[42221]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:37:33.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:33 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1214790760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:37:33.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:33 smithi142 ceph-mon[41779]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-03-30T00:37:33.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:33 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1214790760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T00:37:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.577+0000 7f3ed7a00700 1 -- 172.21.15.99:0/3802432954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0100d80 msgr2=0x7f3ed01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.577+0000 7f3ed7a00700 1 --2- 172.21.15.99:0/3802432954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0100d80 0x7f3ed01011a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40099c0 tx=0x7f3ec4009cd0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.577+0000 7f3ed7a00700 1 -- 172.21.15.99:0/3802432954 shutdown_connections 2024-03-30T00:37:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.577+0000 7f3ed7a00700 1 --2- 172.21.15.99:0/3802432954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed0102910 0x7f3ed01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.577+0000 7f3ed7a00700 1 --2- 172.21.15.99:0/3802432954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0100d80 0x7f3ed01011a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.577+0000 7f3ed7a00700 1 --2- 172.21.15.99:0/3802432954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed0101f70 0x7f3ed01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.577+0000 7f3ed7a00700 1 -- 172.21.15.99:0/3802432954 >> 172.21.15.99:0/3802432954 conn(0x7f3ed00fc2f0 msgr2=0x7f3ed00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.577+0000 7f3ed7a00700 1 -- 172.21.15.99:0/3802432954 shutdown_connections 2024-03-30T00:37:33.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.578+0000 7f3ed7a00700 1 -- 172.21.15.99:0/3802432954 wait complete. 2024-03-30T00:37:33.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.578+0000 7f3ed7a00700 1 Processor -- start 2024-03-30T00:37:33.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ed7a00700 1 -- start start 2024-03-30T00:37:33.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ed7a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed0100d80 0x7f3ed0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ed7a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0101f70 0x7f3ed00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ed7a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed0102910 0x7f3ed00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ed7a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed0073e80 con 0x7f3ed0101f70 2024-03-30T00:37:33.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ed7a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed0073ff0 con 0x7f3ed0100d80 2024-03-30T00:37:33.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ed7a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed01a0d00 con 0x7f3ed0102910 2024-03-30T00:37:33.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ecfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0101f70 0x7f3ed00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ed5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed0100d80 0x7f3ed0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ecfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0101f70 0x7f3ed00731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58106/0 (socket says 172.21.15.99:58106) 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ecfe00700 1 -- 172.21.15.99:0/92223056 learned_addr learned my addr 172.21.15.99:0/92223056 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.579+0000 7f3ed5a00700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed0102910 0x7f3ed00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ed5000700 1 -- 172.21.15.99:0/92223056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed0102910 msgr2=0x7f3ed00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ed5000700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed0102910 0x7f3ed00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ed5000700 1 -- 172.21.15.99:0/92223056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0101f70 msgr2=0x7f3ed00731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ed5000700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0101f70 0x7f3ed00731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ed5000700 1 -- 172.21.15.99:0/92223056 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec40095f0 con 0x7f3ed0100d80 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ecfe00700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0101f70 0x7f3ed00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ed5a00700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed0102910 0x7f3ed00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ed5000700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed0100d80 0x7f3ed0074b10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40098e0 tx=0x7f3ec4003a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ecd600700 1 -- 172.21.15.99:0/92223056 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec401d070 con 0x7f3ed0100d80 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ecd600700 1 -- 172.21.15.99:0/92223056 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ec400bda0 con 0x7f3ed0100d80 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ecd600700 1 -- 172.21.15.99:0/92223056 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4022b60 con 0x7f3ed0100d80 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ed7a00700 1 -- 172.21.15.99:0/92223056 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed01a0f30 con 0x7f3ed0100d80 2024-03-30T00:37:33.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.580+0000 7f3ed7a00700 1 -- 172.21.15.99:0/92223056 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed01a12a0 con 0x7f3ed0100d80 2024-03-30T00:37:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.585+0000 7f3ecd600700 1 -- 172.21.15.99:0/92223056 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f3ec4022470 con 0x7f3ed0100d80 2024-03-30T00:37:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.585+0000 7f3eeb4d1b80 1 Processor -- start 2024-03-30T00:37:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3eeb4d1b80 1 -- start start 2024-03-30T00:37:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3ecd600700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3eb80a1120 0x7f3eb80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e4ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4f130 0x5624f0e4f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4ffd0 0x5624f0e50450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0da6990 con 0x5624f0e4ffd0 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0da4350 con 0x5624f0e4e7d0 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0d9fc40 con 0x5624f0e4f130 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3ecd600700 1 -- 172.21.15.99:0/92223056 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f3ec40cde80 con 0x7f3ed0100d80 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3ed5a00700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3eb80a1120 0x7f3eb80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3ecfe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4f130 0x5624f0e4f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3ed5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e4ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3ed5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e4ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:53924/0 (socket says 172.21.15.99:53924) 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.586+0000 7f3ed5000700 1 -- 172.21.15.99:0/410746574 learned_addr learned my addr 172.21.15.99:0/410746574 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.587+0000 7f3ed5a00700 1 --2- 172.21.15.99:0/410746574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4ffd0 0x5624f0e50450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.587+0000 7f3ed5000700 1 -- 172.21.15.99:0/410746574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4f130 msgr2=0x5624f0e4f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.587+0000 7f3ed5000700 1 --2- 172.21.15.99:0/410746574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4f130 0x5624f0e4f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.587+0000 7f3ed5000700 1 -- 172.21.15.99:0/410746574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4ffd0 msgr2=0x5624f0e50450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.587+0000 7f3ed5000700 1 --2- 172.21.15.99:0/410746574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4ffd0 0x5624f0e50450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.587+0000 7f3ed5000700 1 -- 172.21.15.99:0/410746574 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f0c213b0 con 0x5624f0e4e7d0 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.587+0000 7f3ed5a00700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3eb80a1120 0x7f3eb80a35e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009fd0 tx=0x7f3ec800ab00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.587+0000 7f3ed5000700 1 --2- 172.21.15.99:0/410746574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e4ebf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400b580 tx=0x7f3ec4005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=585a13dba234b2d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.587+0000 7f3ec2000700 1 -- 172.21.15.99:0/410746574 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40adb00 con 0x5624f0e4e7d0 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.587+0000 7f3ec2000700 1 -- 172.21.15.99:0/410746574 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ec40ad920 con 0x5624f0e4e7d0 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.588+0000 7f3ec2000700 1 -- 172.21.15.99:0/410746574 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40ad510 con 0x5624f0e4e7d0 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.588+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/410746574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 msgr2=0x5624f0e4ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.588+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/410746574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e4ebf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400b580 tx=0x7f3ec4005f10 comp rx=0 tx=0).stop 2024-03-30T00:37:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.588+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/410746574 shutdown_connections 2024-03-30T00:37:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.588+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/410746574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4ffd0 0x5624f0e50450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.588+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/410746574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e4ebf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.588+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/410746574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4f130 0x5624f0e4f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.588+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/410746574 >> 172.21.15.99:0/410746574 conn(0x5624f0dad6a0 msgr2=0x5624f0e499a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.588+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/410746574 shutdown_connections 2024-03-30T00:37:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.588+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/410746574 wait complete. 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.589+0000 7f3eeb4d1b80 1 -- start start 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.589+0000 7f3eeb4d1b80 1 Processor -- start 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.589+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e58290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.590+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4f130 0x5624f0e587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.590+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4ffd0 0x5624f0e5f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.590+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0d3bd80 con 0x5624f0e4ffd0 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.590+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0d3ba20 con 0x5624f0e4e7d0 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.590+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0c236a0 con 0x5624f0e4f130 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.590+0000 7f3eeb4d1b80 10 client.0 _ll_register_callbacks cb 0x5624f0d45470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.590+0000 7f3ecfe00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4f130 0x5624f0e587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.590+0000 7f3ed5a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4ffd0 0x5624f0e5f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.590+0000 7f3ed5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e58290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.590+0000 7f3ed5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e58290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:53938/0 (socket says 172.21.15.99:53938) 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.590+0000 7f3ed5000700 1 -- 172.21.15.99:0/3360611921 learned_addr learned my addr 172.21.15.99:0/3360611921 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.591+0000 7f3ecfe00700 1 -- 172.21.15.99:0/3360611921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 msgr2=0x5624f0e58290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.591+0000 7f3ecfe00700 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e58290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.591+0000 7f3ecfe00700 1 -- 172.21.15.99:0/3360611921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4ffd0 msgr2=0x5624f0e5f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.591+0000 7f3ecfe00700 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4ffd0 0x5624f0e5f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.591+0000 7f3ecfe00700 1 -- 172.21.15.99:0/3360611921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed01a12a0 con 0x5624f0e4f130 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.591+0000 7f3ed5000700 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e58290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.591+0000 7f3ecfe00700 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4f130 0x5624f0e587d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc011820 tx=0x7f3ebc011b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e67f5e694e24c3eb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.591+0000 7f3eb6a00700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.591+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3360611921 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc016070 con 0x5624f0e4f130 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.591+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3360611921 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f3ebc004320 con 0x5624f0e4f130 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.591+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3360611921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624f0d60380 con 0x5624f0e4f130 2024-03-30T00:37:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.592+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3360611921 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ebc0044e0 con 0x5624f0e4f130 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.592+0000 7f3eeb4d1b80 10 client.25888 fetch_fsmap learned FSMap version 27 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.592+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3360611921 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5624f0bcdb60 con 0x5624f0e4f130 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.592+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3360611921 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5624f0d60380 con 0x5624f0e4f130 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.592+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3360611921 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ebc010870 con 0x5624f0e4f130 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.592+0000 7f3eeb4d1b80 10 client.25888 fetch_fsmap finished waiting for FSMap version 27 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.592+0000 7f3eeb4d1b80 10 client.25888 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.592+0000 7f3eeb4d1b80 20 client.25888 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.592+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5624f0e64020 0x5624f0e66440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.592+0000 7f3eeb4d1b80 4 client.25888 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.592+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3360611921 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624f0d98700 con 0x5624f0e64020 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.593+0000 7f3ed5000700 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5624f0e64020 0x5624f0e66440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.593+0000 7f3eb6a00700 10 client.25888 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.593+0000 7f3ed5000700 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5624f0e64020 0x5624f0e66440 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9a321a583622589 server_cookie=46d8836045d16904 in_seq=0 out_seq=0 2024-03-30T00:37:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3360611921 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5624f0d98700 con 0x5624f0e64020 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eb6a00700 10 client.25888 handle_command_reply: tid=0 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 1 client.25888 shutdown 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3360611921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4f130 msgr2=0x5624f0e587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4f130 0x5624f0e587d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc011820 tx=0x7f3ebc011b30 comp rx=0 tx=0).stop 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3360611921 shutdown_connections 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5624f0e64020 0x5624f0e66440 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4ffd0 0x5624f0e5f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e58290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eb6a00700 0 client.25888 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3360611921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4f130 0x5624f0e587d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3360611921 >> 172.21.15.99:0/3360611921 conn(0x5624f0dad6a0 msgr2=0x5624f0e4d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3360611921 shutdown_connections 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3360611921 wait complete. 2024-03-30T00:37:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.594+0000 7f3eeb4d1b80 20 client.25888 trim_cache size 0 max 16384 2024-03-30T00:37:33.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.609+0000 7f3eeb4d1b80 1 Processor -- start 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.609+0000 7f3eeb4d1b80 1 -- start start 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.609+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e5fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.609+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.609+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4ffd0 0x5624f0e503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.609+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0e57290 con 0x5624f0e4f130 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.609+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0e60990 con 0x5624f0e4e7d0 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.609+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0e4e450 con 0x5624f0e4ffd0 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.610+0000 7f3ecfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.610+0000 7f3ecfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58128/0 (socket says 172.21.15.99:58128) 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.610+0000 7f3ecfe00700 1 -- 172.21.15.99:0/1682814003 learned_addr learned my addr 172.21.15.99:0/1682814003 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.610+0000 7f3ed5a00700 1 --2- 172.21.15.99:0/1682814003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4ffd0 0x5624f0e503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.610+0000 7f3ed5000700 1 --2- 172.21.15.99:0/1682814003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e5fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.610+0000 7f3ecfe00700 1 -- 172.21.15.99:0/1682814003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4ffd0 msgr2=0x5624f0e503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.610+0000 7f3ecfe00700 1 --2- 172.21.15.99:0/1682814003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4ffd0 0x5624f0e503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.610+0000 7f3ecfe00700 1 -- 172.21.15.99:0/1682814003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 msgr2=0x5624f0e5fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.610+0000 7f3ecfe00700 1 --2- 172.21.15.99:0/1682814003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e5fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.610+0000 7f3ecfe00700 1 -- 172.21.15.99:0/1682814003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f0d65ea0 con 0x5624f0e4f130 2024-03-30T00:37:33.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.610+0000 7f3ed5a00700 1 --2- 172.21.15.99:0/1682814003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4ffd0 0x5624f0e503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:33.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.611+0000 7f3ecfe00700 1 --2- 172.21.15.99:0/1682814003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5f730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc005950 tx=0x7f3ebc00b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb13ed93c69604 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.611+0000 7f3ec2a00700 1 -- 172.21.15.99:0/1682814003 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc016070 con 0x5624f0e4f130 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.611+0000 7f3ec2a00700 1 -- 172.21.15.99:0/1682814003 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ebc017040 con 0x5624f0e4f130 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.611+0000 7f3ec2a00700 1 -- 172.21.15.99:0/1682814003 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc003a00 con 0x5624f0e4f130 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.611+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/1682814003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 msgr2=0x5624f0e5f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.611+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/1682814003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5f730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc005950 tx=0x7f3ebc00b6f0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.612+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/1682814003 shutdown_connections 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.612+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/1682814003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4ffd0 0x5624f0e503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.612+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/1682814003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4e7d0 0x5624f0e5fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.612+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/1682814003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5f730 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.612+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/1682814003 >> 172.21.15.99:0/1682814003 conn(0x5624f0dad6a0 msgr2=0x5624f0e425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.612+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/1682814003 shutdown_connections 2024-03-30T00:37:33.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.612+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/1682814003 wait complete. 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.612+0000 7f3eeb4d1b80 1 -- start start 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.612+0000 7f3eeb4d1b80 1 Processor -- start 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4e7d0 0x5624f0e5d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3eeb4d1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4ffd0 0x5624f0e5e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0e5ecf0 con 0x5624f0e4f130 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0e5ee60 con 0x5624f0e4ffd0 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3eeb4d1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0e5f020 con 0x5624f0e4e7d0 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3eeb4d1b80 10 client.0 _ll_register_callbacks cb 0x5624f0d45470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ecfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ecfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58142/0 (socket says 172.21.15.99:58142) 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ed5000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4e7d0 0x5624f0e5d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ecfe00700 1 -- 172.21.15.99:0/3429636151 learned_addr learned my addr 172.21.15.99:0/3429636151 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ed5a00700 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4ffd0 0x5624f0e5e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ecfe00700 1 -- 172.21.15.99:0/3429636151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4e7d0 msgr2=0x5624f0e5d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ecfe00700 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4e7d0 0x5624f0e5d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ecfe00700 1 -- 172.21.15.99:0/3429636151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4ffd0 msgr2=0x5624f0e5e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ecfe00700 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4ffd0 0x5624f0e5e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ecfe00700 1 -- 172.21.15.99:0/3429636151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec8012040 con 0x5624f0e4f130 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ed5000700 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4e7d0 0x5624f0e5d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.613+0000 7f3ed5a00700 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4ffd0 0x5624f0e5e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.614+0000 7f3ecfe00700 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5d7f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc011820 tx=0x7f3ebc0039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54aba567cf436583 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.614+0000 7f3eb6a00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.614+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3429636151 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc016050 con 0x5624f0e4f130 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.614+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3429636151 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f3ebc010a00 con 0x5624f0e4f130 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.614+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3429636151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ec40ad920 con 0x5624f0e4f130 2024-03-30T00:37:33.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.614+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3429636151 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ebc035070 con 0x5624f0e4f130 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.614+0000 7f3eeb4d1b80 10 client.16185 fetch_fsmap learned FSMap version 27 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.614+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3429636151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5624f0bcdb60 con 0x5624f0e4f130 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.615+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3429636151 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ec40ad920 con 0x5624f0e4f130 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.615+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3429636151 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ebc0259f0 con 0x5624f0e4f130 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.615+0000 7f3eeb4d1b80 10 client.16185 fetch_fsmap finished waiting for FSMap version 27 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.615+0000 7f3eeb4d1b80 10 client.16185 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.615+0000 7f3eeb4d1b80 20 client.16185 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.615+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5624f0eb8780 0x5624f0ebaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.615+0000 7f3eeb4d1b80 4 client.16185 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.615+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3429636151 --> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5624f0beecf0 con 0x5624f0eb8780 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.615+0000 7f3ed5000700 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5624f0eb8780 0x5624f0ebaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.615+0000 7f3ed5000700 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5624f0eb8780 0x5624f0ebaba0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f41e44a70de519d0 server_cookie=7fce7059952c535c in_seq=0 out_seq=0 2024-03-30T00:37:33.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.615+0000 7f3eb6a00700 10 client.16185 ms_handle_connect on v2:172.21.15.99:6840/1391823621 2024-03-30T00:37:33.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eb6a00700 1 -- 172.21.15.99:0/3429636151 <== mds.1 v2:172.21.15.99:6840/1391823621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+834 (crc 0 0 0) 0x5624f0beecf0 con 0x5624f0eb8780 2024-03-30T00:37:33.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eb6a00700 10 client.16185 handle_command_reply: tid=0 2024-03-30T00:37:33.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 1 client.16185 shutdown 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3429636151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 msgr2=0x5624f0e5d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5d7f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc011820 tx=0x7f3ebc0039b0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3429636151 shutdown_connections 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.99:6840/1391823621,v1:172.21.15.99:6841/1391823621] conn(0x5624f0eb8780 0x5624f0ebaba0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5624f0e4ffd0 0x5624f0e5e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5624f0e4e7d0 0x5624f0e5d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 1 --2- 172.21.15.99:0/3429636151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624f0e4f130 0x5624f0e5d7f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eb6a00700 0 client.16185 ms_handle_reset on v2:172.21.15.99:6840/1391823621 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3429636151 >> 172.21.15.99:0/3429636151 conn(0x5624f0dad6a0 msgr2=0x5624f0e425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3429636151 shutdown_connections 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 1 -- 172.21.15.99:0/3429636151 wait complete. 2024-03-30T00:37:33.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.616+0000 7f3eeb4d1b80 20 client.16185 trim_cache size 0 max 16384 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: "damage_type": "backtrace", 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 571126927, 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 1099511627783, 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.c" 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: "damage_type": "backtrace", 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 1151261223, 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 1099511628289, 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester" 2024-03-30T00:37:33.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: "damage_type": "backtrace", 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 2104435737, 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 1099511628291, 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.tst-file" 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: "damage_type": "backtrace", 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4020592589, 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 1099511628290, 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: "path": "/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.rnd-file" 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:33.629 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-03-30T00:37:33.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 -- 172.21.15.99:0/92223056 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3eb80a1120 msgr2=0x7f3eb80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3eb80a1120 0x7f3eb80a35e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009fd0 tx=0x7f3ec800ab00 comp rx=0 tx=0).stop 2024-03-30T00:37:33.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 -- 172.21.15.99:0/92223056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed0100d80 msgr2=0x7f3ed0074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed0100d80 0x7f3ed0074b10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40098e0 tx=0x7f3ec4003a50 comp rx=0 tx=0).stop 2024-03-30T00:37:33.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 -- 172.21.15.99:0/92223056 shutdown_connections 2024-03-30T00:37:33.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ed0102910 0x7f3ed00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f3eb80a1120 0x7f3eb80a35e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed0100d80 0x7f3ed0074b10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 --2- 172.21.15.99:0/92223056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ed0101f70 0x7f3ed00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 -- 172.21.15.99:0/92223056 >> 172.21.15.99:0/92223056 conn(0x7f3ed00fc2f0 msgr2=0x7f3ed0105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 -- 172.21.15.99:0/92223056 shutdown_connections 2024-03-30T00:37:33.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.617+0000 7f3ed7a00700 1 -- 172.21.15.99:0/92223056 wait complete. 2024-03-30T00:37:33.642 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T00:37:33.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.831+0000 7f8a50000700 1 -- 172.21.15.99:0/713921223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a481017e0 msgr2=0x7f8a48103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.831+0000 7f8a50000700 1 --2- 172.21.15.99:0/713921223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a481017e0 0x7f8a48103bd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009990 tx=0x7f8a3c009ca0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.831+0000 7f8a50000700 1 -- 172.21.15.99:0/713921223 shutdown_connections 2024-03-30T00:37:33.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.831+0000 7f8a50000700 1 --2- 172.21.15.99:0/713921223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a48106a40 0x7f8a48108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.831+0000 7f8a50000700 1 --2- 172.21.15.99:0/713921223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a481017e0 0x7f8a48103bd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.831+0000 7f8a50000700 1 --2- 172.21.15.99:0/713921223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a48104110 0x7f8a48106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.831+0000 7f8a50000700 1 -- 172.21.15.99:0/713921223 >> 172.21.15.99:0/713921223 conn(0x7f8a480fb1d0 msgr2=0x7f8a480fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.832+0000 7f8a50000700 1 -- 172.21.15.99:0/713921223 shutdown_connections 2024-03-30T00:37:33.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.832+0000 7f8a50000700 1 -- 172.21.15.99:0/713921223 wait complete. 2024-03-30T00:37:33.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.832+0000 7f8a50000700 1 Processor -- start 2024-03-30T00:37:33.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a50000700 1 -- start start 2024-03-30T00:37:33.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a50000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a481017e0 0x7f8a48196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a50000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a48104110 0x7f8a48197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a50000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a48106a40 0x7f8a4819e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a50000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4819e7b0 con 0x7f8a481017e0 2024-03-30T00:37:33.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a50000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4819e920 con 0x7f8a48106a40 2024-03-30T00:37:33.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a50000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4819eae0 con 0x7f8a48104110 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a4d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a481017e0 0x7f8a48196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a4d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a481017e0 0x7f8a48196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58160/0 (socket says 172.21.15.99:58160) 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a4d600700 1 -- 172.21.15.99:0/3244898874 learned_addr learned my addr 172.21.15.99:0/3244898874 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a48104110 0x7f8a48197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.833+0000 7f8a4e000700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a48106a40 0x7f8a4819e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a4d600700 1 -- 172.21.15.99:0/3244898874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a48104110 msgr2=0x7f8a48197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a4d600700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a48104110 0x7f8a48197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a4d600700 1 -- 172.21.15.99:0/3244898874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a48106a40 msgr2=0x7f8a4819e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a4d600700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a48106a40 0x7f8a4819e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a4d600700 1 -- 172.21.15.99:0/3244898874 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c0095f0 con 0x7f8a481017e0 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a4e000700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a48106a40 0x7f8a4819e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:33.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a4d600700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a481017e0 0x7f8a48196cc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0098e0 tx=0x7f8a3c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a46000700 1 -- 172.21.15.99:0/3244898874 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c01d070 con 0x7f8a481017e0 2024-03-30T00:37:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a46000700 1 -- 172.21.15.99:0/3244898874 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a3c004000 con 0x7f8a481017e0 2024-03-30T00:37:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a50000700 1 -- 172.21.15.99:0/3244898874 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4819ed60 con 0x7f8a481017e0 2024-03-30T00:37:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a46000700 1 -- 172.21.15.99:0/3244898874 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c022ca0 con 0x7f8a481017e0 2024-03-30T00:37:33.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.834+0000 7f8a50000700 1 -- 172.21.15.99:0/3244898874 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4819f130 con 0x7f8a481017e0 2024-03-30T00:37:33.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.837+0000 7f8a46000700 1 -- 172.21.15.99:0/3244898874 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f8a3c022470 con 0x7f8a481017e0 2024-03-30T00:37:33.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.838+0000 7f8a46000700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a300a1120 0x7f8a300a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.838+0000 7f8a63b0db80 1 Processor -- start 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a46000700 1 -- 172.21.15.99:0/3244898874 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f8a3c0cdea0 con 0x7f8a481017e0 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a63b0db80 1 -- start start 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a300a1120 0x7f8a300a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x56542319afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319b4e0 0x56542319bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x56542319c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654230ee990 con 0x56542319c3d0 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654230ec350 con 0x56542319ab80 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654230e7c40 con 0x56542319b4e0 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a4d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x56542319afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a4e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x56542319c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a4d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x56542319afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:53972/0 (socket says 172.21.15.99:53972) 2024-03-30T00:37:33.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a4d600700 1 -- 172.21.15.99:0/1833973035 learned_addr learned my addr 172.21.15.99:0/1833973035 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.839+0000 7f8a4e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x56542319c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58176/0 (socket says 172.21.15.99:58176) 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.840+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/1833973035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319b4e0 0x56542319bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.840+0000 7f8a4d600700 1 -- 172.21.15.99:0/1833973035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319b4e0 msgr2=0x56542319bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.840+0000 7f8a4d600700 1 --2- 172.21.15.99:0/1833973035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319b4e0 0x56542319bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.840+0000 7f8a4d600700 1 -- 172.21.15.99:0/1833973035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 msgr2=0x56542319c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.840+0000 7f8a4d600700 1 --2- 172.21.15.99:0/1833973035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x56542319c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.840+0000 7f8a4d600700 1 -- 172.21.15.99:0/1833973035 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565422f693b0 con 0x56542319ab80 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.840+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a300a1120 0x7f8a300a35e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8a34007bf0 tx=0x7f8a340103d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.840+0000 7f8a4d600700 1 --2- 172.21.15.99:0/1833973035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x56542319afa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0977a0 tx=0x7f8a3c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dbfd68f84413d36 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.840+0000 7f8a3a600700 1 -- 172.21.15.99:0/1833973035 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c09b9a0 con 0x56542319ab80 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.840+0000 7f8a3a600700 1 -- 172.21.15.99:0/1833973035 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a3c09b2a0 con 0x56542319ab80 2024-03-30T00:37:33.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.840+0000 7f8a3a600700 1 -- 172.21.15.99:0/1833973035 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c099760 con 0x56542319ab80 2024-03-30T00:37:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.841+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1833973035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 msgr2=0x56542319afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.841+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1833973035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x56542319afa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0977a0 tx=0x7f8a3c009ca0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.841+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1833973035 shutdown_connections 2024-03-30T00:37:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.841+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1833973035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x56542319c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.841+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1833973035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x56542319afa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.841+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1833973035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319b4e0 0x56542319bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.841+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1833973035 >> 172.21.15.99:0/1833973035 conn(0x5654230f56a0 msgr2=0x565423191880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.841+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1833973035 shutdown_connections 2024-03-30T00:37:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.841+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1833973035 wait complete. 2024-03-30T00:37:33.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.842+0000 7f8a63b0db80 1 -- start start 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.842+0000 7f8a63b0db80 1 Processor -- start 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x5654231a4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319b4e0 0x5654231a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x5654231abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x565423083d80 con 0x56542319c3d0 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565423083a20 con 0x56542319ab80 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x565422f6b6a0 con 0x56542319b4e0 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a63b0db80 10 client.0 _ll_register_callbacks cb 0x56542308d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x5654231a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x5654231abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x5654231a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.99:53982/0 (socket says 172.21.15.99:53982) 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4d600700 1 -- 172.21.15.99:0/1592062737 learned_addr learned my addr 172.21.15.99:0/1592062737 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319b4e0 0x5654231a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4d600700 1 -- 172.21.15.99:0/1592062737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319b4e0 msgr2=0x5654231a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4d600700 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319b4e0 0x5654231a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4d600700 1 -- 172.21.15.99:0/1592062737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 msgr2=0x5654231abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4d600700 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x5654231abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4d600700 1 -- 172.21.15.99:0/1592062737 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4819f130 con 0x56542319ab80 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319b4e0 0x5654231a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4e000700 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x5654231abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.843+0000 7f8a4d600700 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x5654231a4720 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c09ab90 tx=0x7f8a3c0a34e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b58ca8e7f51a527a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a2ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a2ea00700 1 -- 172.21.15.99:0/1592062737 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c09b9a0 con 0x56542319ab80 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a2ea00700 1 -- 172.21.15.99:0/1592062737 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f8a3c0b3050 con 0x56542319ab80 2024-03-30T00:37:33.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1592062737 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654230a8380 con 0x56542319ab80 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a2ea00700 1 -- 172.21.15.99:0/1592062737 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a3c09f0d0 con 0x56542319ab80 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a63b0db80 10 client.25889 fetch_fsmap learned FSMap version 27 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1592062737 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565422f15b60 con 0x56542319ab80 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a2ea00700 1 -- 172.21.15.99:0/1592062737 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5654230a8380 con 0x56542319ab80 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a2ea00700 1 -- 172.21.15.99:0/1592062737 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a3c09d030 con 0x56542319ab80 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a63b0db80 10 client.25889 fetch_fsmap finished waiting for FSMap version 27 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a63b0db80 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a63b0db80 20 client.25889 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.844+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5654231b00b0 0x5654231b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.845+0000 7f8a63b0db80 4 client.25889 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.845+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1592062737 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654230e0700 con 0x5654231b00b0 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.845+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5654231b00b0 0x5654231b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5654231b00b0 0x5654231b24d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9490da73151206 server_cookie=23c0e5788b2fa64e in_seq=0 out_seq=0 2024-03-30T00:37:33.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a2ea00700 10 client.25889 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a2ea00700 1 -- 172.21.15.99:0/1592062737 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5654230e0700 con 0x5654231b00b0 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a2ea00700 10 client.25889 handle_command_reply: tid=0 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a63b0db80 1 client.25889 shutdown 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1592062737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 msgr2=0x5654231a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x5654231a4720 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c09ab90 tx=0x7f8a3c0a34e0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1592062737 shutdown_connections 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x5654231b00b0 0x5654231b24d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x5654231abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x5654231a4720 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1592062737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319b4e0 0x5654231a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a2ea00700 0 client.25889 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.846+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1592062737 >> 172.21.15.99:0/1592062737 conn(0x5654230f56a0 msgr2=0x565423199580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.847+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1592062737 shutdown_connections 2024-03-30T00:37:33.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.847+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1592062737 wait complete. 2024-03-30T00:37:33.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.847+0000 7f8a63b0db80 20 client.25889 trim_cache size 0 max 16384 2024-03-30T00:37:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.860+0000 7f8a63b0db80 1 Processor -- start 2024-03-30T00:37:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.860+0000 7f8a63b0db80 1 -- start start 2024-03-30T00:37:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.860+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x5654231b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.860+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319b4e0 0x56542319af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.860+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319c3d0 0x5654231ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.860+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x56542319bc50 con 0x56542319b4e0 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.860+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565423192410 con 0x56542319ab80 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.860+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654231a5910 con 0x56542319c3d0 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.860+0000 7f8a4cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319b4e0 0x56542319af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a4cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319b4e0 0x56542319af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58184/0 (socket says 172.21.15.99:58184) 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a4cc00700 1 -- 172.21.15.99:0/1791819136 learned_addr learned my addr 172.21.15.99:0/1791819136 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a4e000700 1 --2- 172.21.15.99:0/1791819136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319c3d0 0x5654231ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a4d600700 1 --2- 172.21.15.99:0/1791819136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x5654231b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a4cc00700 1 -- 172.21.15.99:0/1791819136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319c3d0 msgr2=0x5654231ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/1791819136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319c3d0 0x5654231ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a4cc00700 1 -- 172.21.15.99:0/1791819136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 msgr2=0x5654231b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/1791819136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x5654231b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a4cc00700 1 -- 172.21.15.99:0/1791819136 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654231a3710 con 0x56542319b4e0 2024-03-30T00:37:33.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/1791819136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319b4e0 0x56542319af60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8a340079d0 tx=0x7f8a34015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b003a162bf24a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a3b000700 1 -- 172.21.15.99:0/1791819136 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34024040 con 0x56542319b4e0 2024-03-30T00:37:33.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a3b000700 1 -- 172.21.15.99:0/1791819136 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a34026ca0 con 0x56542319b4e0 2024-03-30T00:37:33.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.861+0000 7f8a3b000700 1 -- 172.21.15.99:0/1791819136 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34013e50 con 0x56542319b4e0 2024-03-30T00:37:33.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.862+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1791819136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319b4e0 msgr2=0x56542319af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.862+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1791819136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319b4e0 0x56542319af60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8a340079d0 tx=0x7f8a34015000 comp rx=0 tx=0).stop 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.862+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1791819136 shutdown_connections 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.862+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1791819136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319c3d0 0x5654231ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.862+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1791819136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319ab80 0x5654231b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.862+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/1791819136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319b4e0 0x56542319af60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.862+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1791819136 >> 172.21.15.99:0/1791819136 conn(0x5654230f56a0 msgr2=0x56542318af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.862+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1791819136 shutdown_connections 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.862+0000 7f8a63b0db80 1 -- 172.21.15.99:0/1791819136 wait complete. 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.863+0000 7f8a63b0db80 1 -- start start 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.863+0000 7f8a63b0db80 1 Processor -- start 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.863+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319ab80 0x5654231a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.863+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319b4e0 0x5654231a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.863+0000 7f8a63b0db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x5654231aab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.863+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654231ab0a0 con 0x56542319c3d0 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.863+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654231ab210 con 0x56542319b4e0 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.863+0000 7f8a63b0db80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654231ab3d0 con 0x56542319ab80 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.863+0000 7f8a63b0db80 10 client.0 _ll_register_callbacks cb 0x56542308d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x5654231aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x5654231aab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58200/0 (socket says 172.21.15.99:58200) 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4e000700 1 -- 172.21.15.99:0/2230965207 learned_addr learned my addr 172.21.15.99:0/2230965207 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4d600700 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319ab80 0x5654231a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319b4e0 0x5654231a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4e000700 1 -- 172.21.15.99:0/2230965207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319ab80 msgr2=0x5654231a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4e000700 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319ab80 0x5654231a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4e000700 1 -- 172.21.15.99:0/2230965207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319b4e0 msgr2=0x5654231a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4e000700 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319b4e0 0x5654231a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4e000700 1 -- 172.21.15.99:0/2230965207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a34010b70 con 0x56542319c3d0 2024-03-30T00:37:33.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4cc00700 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319b4e0 0x5654231a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a4e000700 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x5654231aab60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8a40007fb0 tx=0x7f8a40009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9e580ff8d9fb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.864+0000 7f8a2ea00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a2ea00700 1 -- 172.21.15.99:0/2230965207 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40017ac0 con 0x56542319c3d0 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a2ea00700 1 -- 172.21.15.99:0/2230965207 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f8a40017c20 con 0x56542319c3d0 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a63b0db80 1 -- 172.21.15.99:0/2230965207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a3c09b2a0 con 0x56542319c3d0 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a2ea00700 1 -- 172.21.15.99:0/2230965207 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a40017de0 con 0x56542319c3d0 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a63b0db80 10 client.16203 fetch_fsmap learned FSMap version 27 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a63b0db80 1 -- 172.21.15.99:0/2230965207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5654230adea0 con 0x56542319c3d0 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a2ea00700 1 -- 172.21.15.99:0/2230965207 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a3c09b2a0 con 0x56542319c3d0 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a2ea00700 1 -- 172.21.15.99:0/2230965207 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a40021550 con 0x56542319c3d0 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a63b0db80 10 client.16203 fetch_fsmap finished waiting for FSMap version 27 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a63b0db80 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a63b0db80 20 client.16203 populate_metadata read hostname 'smithi099' 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565423204a70 0x565423206e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:33.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a63b0db80 4 client.16203 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T00:37:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.865+0000 7f8a63b0db80 1 -- 172.21.15.99:0/2230965207 --> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565422f36cf0 con 0x565423204a70 2024-03-30T00:37:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a4d600700 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565423204a70 0x565423206e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a2ea00700 10 client.16203 ms_handle_connect on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:33.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a4d600700 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565423204a70 0x565423206e90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f22af74d7cfbef71 server_cookie=5d8ea026072f238c in_seq=0 out_seq=0 2024-03-30T00:37:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a2ea00700 1 -- 172.21.15.99:0/2230965207 <== mds.0 v2:172.21.15.142:6838/2862512313 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565422f36cf0 con 0x565423204a70 2024-03-30T00:37:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a2ea00700 10 client.16203 handle_command_reply: tid=0 2024-03-30T00:37:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a63b0db80 1 client.16203 shutdown 2024-03-30T00:37:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a63b0db80 1 -- 172.21.15.99:0/2230965207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 msgr2=0x5654231aab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x5654231aab60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8a40007fb0 tx=0x7f8a40009e60 comp rx=0 tx=0).stop 2024-03-30T00:37:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a63b0db80 1 -- 172.21.15.99:0/2230965207 shutdown_connections 2024-03-30T00:37:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.142:6838/2862512313,v1:172.21.15.142:6839/2862512313] conn(0x565423204a70 0x565423206e90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56542319c3d0 0x5654231aab60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a2ea00700 0 client.16203 ms_handle_reset on v2:172.21.15.142:6838/2862512313 2024-03-30T00:37:33.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.866+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56542319ab80 0x5654231a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a63b0db80 1 --2- 172.21.15.99:0/2230965207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56542319b4e0 0x5654231a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a63b0db80 1 -- 172.21.15.99:0/2230965207 >> 172.21.15.99:0/2230965207 conn(0x5654230f56a0 msgr2=0x56542318a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a63b0db80 1 -- 172.21.15.99:0/2230965207 shutdown_connections 2024-03-30T00:37:33.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a63b0db80 1 -- 172.21.15.99:0/2230965207 wait complete. 2024-03-30T00:37:33.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a63b0db80 20 client.16203 trim_cache size 0 max 16384 2024-03-30T00:37:33.878 INFO:teuthology.orchestra.run.smithi099.stdout:[] 2024-03-30T00:37:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a50000700 1 -- 172.21.15.99:0/3244898874 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a300a1120 msgr2=0x7f8a300a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a50000700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a300a1120 0x7f8a300a35e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8a34007bf0 tx=0x7f8a340103d0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a50000700 1 -- 172.21.15.99:0/3244898874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a481017e0 msgr2=0x7f8a48196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a50000700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a481017e0 0x7f8a48196cc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0098e0 tx=0x7f8a3c005f10 comp rx=0 tx=0).stop 2024-03-30T00:37:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a50000700 1 -- 172.21.15.99:0/3244898874 shutdown_connections 2024-03-30T00:37:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a50000700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8a300a1120 0x7f8a300a35e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a50000700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a48106a40 0x7f8a4819e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a50000700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a481017e0 0x7f8a48196cc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a50000700 1 --2- 172.21.15.99:0/3244898874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a48104110 0x7f8a48197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:33.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.867+0000 7f8a50000700 1 -- 172.21.15.99:0/3244898874 >> 172.21.15.99:0/3244898874 conn(0x7f8a480fb1d0 msgr2=0x7f8a481037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:33.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.868+0000 7f8a50000700 1 -- 172.21.15.99:0/3244898874 shutdown_connections 2024-03-30T00:37:33.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:33.868+0000 7f8a50000700 1 -- 172.21.15.99:0/3244898874 wait complete. 2024-03-30T00:37:33.892 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 571126927, 'ino': 1099511627783, 'path': '/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.c'}, {'damage_type': 'backtrace', 'id': 1151261223, 'ino': 1099511628289, 'path': '/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester'}, {'damage_type': 'backtrace', 'id': 2104435737, 'ino': 1099511628291, 'path': '/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.tst-file'}, {'damage_type': 'backtrace', 'id': 4020592589, 'ino': 1099511628290, 'path': '/volumes/qa/sv_0/53f81470-29f1-4e8a-82e0-10b602b921af/client.0/tmp/fsync-tester.rnd-file'}] 2024-03-30T00:37:33.892 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-03-30T00:37:33.893 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-30T00:37:33.912 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-30T00:37:33.923 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:37:34.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.114+0000 7f1461800700 1 -- 172.21.15.99:0/375751715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c100c80 msgr2=0x7f145c1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:34.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.114+0000 7f1461800700 1 --2- 172.21.15.99:0/375751715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c100c80 0x7f145c1010a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f144c0099c0 tx=0x7f144c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.114+0000 7f1461800700 1 -- 172.21.15.99:0/375751715 shutdown_connections 2024-03-30T00:37:34.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.114+0000 7f1461800700 1 --2- 172.21.15.99:0/375751715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c102810 0x7f145c1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.114+0000 7f1461800700 1 --2- 172.21.15.99:0/375751715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c100c80 0x7f145c1010a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.114+0000 7f1461800700 1 --2- 172.21.15.99:0/375751715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f145c101e70 0x7f145c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.114+0000 7f1461800700 1 -- 172.21.15.99:0/375751715 >> 172.21.15.99:0/375751715 conn(0x7f145c0fc1f0 msgr2=0x7f145c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:34.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.114+0000 7f1461800700 1 -- 172.21.15.99:0/375751715 shutdown_connections 2024-03-30T00:37:34.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.114+0000 7f1461800700 1 -- 172.21.15.99:0/375751715 wait complete. 2024-03-30T00:37:34.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.115+0000 7f1461800700 1 Processor -- start 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.115+0000 7f1461800700 1 -- start start 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f1461800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f145c100c80 0x7f145c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f1461800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c101e70 0x7f145c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f1461800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c102810 0x7f145c1a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f1461800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1a0820 con 0x7f145c102810 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f1461800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1a0990 con 0x7f145c101e70 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f1461800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1a0b50 con 0x7f145c100c80 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f145b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c102810 0x7f145c1a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f145b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c102810 0x7f145c1a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58212/0 (socket says 172.21.15.99:58212) 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f145b400700 1 -- 172.21.15.99:0/1836188957 learned_addr learned my addr 172.21.15.99:0/1836188957 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f145a000700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c101e70 0x7f145c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f145aa00700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f145c100c80 0x7f145c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f145b400700 1 -- 172.21.15.99:0/1836188957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f145c100c80 msgr2=0x7f145c198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.116+0000 7f145b400700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f145c100c80 0x7f145c198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.117+0000 7f145b400700 1 -- 172.21.15.99:0/1836188957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c101e70 msgr2=0x7f145c199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:34.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.117+0000 7f145b400700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c101e70 0x7f145c199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.117+0000 7f145b400700 1 -- 172.21.15.99:0/1836188957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c0095f0 con 0x7f145c102810 2024-03-30T00:37:34.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.117+0000 7f145a000700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c101e70 0x7f145c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:37:34.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.117+0000 7f145b400700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c102810 0x7f145c1a0250 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1454007d70 tx=0x7f145400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:34.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.117+0000 7f1453400700 1 -- 172.21.15.99:0/1836188957 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400ad40 con 0x7f145c102810 2024-03-30T00:37:34.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.117+0000 7f1453400700 1 -- 172.21.15.99:0/1836188957 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14540128b0 con 0x7f145c102810 2024-03-30T00:37:34.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.117+0000 7f1461800700 1 -- 172.21.15.99:0/1836188957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145c1a0e30 con 0x7f145c102810 2024-03-30T00:37:34.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.117+0000 7f1453400700 1 -- 172.21.15.99:0/1836188957 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540116b0 con 0x7f145c102810 2024-03-30T00:37:34.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.117+0000 7f1461800700 1 -- 172.21.15.99:0/1836188957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c1a12b0 con 0x7f145c102810 2024-03-30T00:37:34.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.118+0000 7f1461800700 1 -- 172.21.15.99:0/1836188957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145c067a10 con 0x7f145c102810 2024-03-30T00:37:34.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.120+0000 7f1453400700 1 -- 172.21.15.99:0/1836188957 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1454012a20 con 0x7f145c102810 2024-03-30T00:37:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.121+0000 7f1453400700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14400a11a0 0x7f14400a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.122+0000 7f1453400700 1 -- 172.21.15.99:0/1836188957 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f14540ccce0 con 0x7f145c102810 2024-03-30T00:37:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.122+0000 7f145aa00700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14400a11a0 0x7f14400a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:34.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.122+0000 7f145aa00700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14400a11a0 0x7f14400a3660 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f144c00b580 tx=0x7f144c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:34.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.126+0000 7f1453400700 1 -- 172.21.15.99:0/1836188957 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1454096390 con 0x7f145c102810 2024-03-30T00:37:34.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:34 smithi099 ceph-mon[38079]: pgmap v389: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 2.40k op/s 2024-03-30T00:37:34.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.376+0000 7f1461800700 1 -- 172.21.15.99:0/1836188957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f145c10c390 con 0x7f145c102810 2024-03-30T00:37:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.378+0000 7f1453400700 1 -- 172.21.15.99:0/1836188957 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22859 (secure 0 0 0) 0x7f1454016090 con 0x7f145c102810 2024-03-30T00:37:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:37:34.385 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:35:10.474499+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8},{"from":5,"to":7}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:37:34.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.380+0000 7f1461800700 1 -- 172.21.15.99:0/1836188957 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14400a11a0 msgr2=0x7f14400a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:34.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.380+0000 7f1461800700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14400a11a0 0x7f14400a3660 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f144c00b580 tx=0x7f144c005fb0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.380+0000 7f1461800700 1 -- 172.21.15.99:0/1836188957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c102810 msgr2=0x7f145c1a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:34.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.380+0000 7f1461800700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c102810 0x7f145c1a0250 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1454007d70 tx=0x7f145400a340 comp rx=0 tx=0).stop 2024-03-30T00:37:34.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.380+0000 7f1461800700 1 -- 172.21.15.99:0/1836188957 shutdown_connections 2024-03-30T00:37:34.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.381+0000 7f1461800700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c102810 0x7f145c1a0250 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.381+0000 7f1461800700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f145c100c80 0x7f145c198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.381+0000 7f1461800700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f14400a11a0 0x7f14400a3660 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.381+0000 7f1461800700 1 --2- 172.21.15.99:0/1836188957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c101e70 0x7f145c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.381+0000 7f1461800700 1 -- 172.21.15.99:0/1836188957 >> 172.21.15.99:0/1836188957 conn(0x7f145c0fc1f0 msgr2=0x7f145c107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:34.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.381+0000 7f1461800700 1 -- 172.21.15.99:0/1836188957 shutdown_connections 2024-03-30T00:37:34.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.381+0000 7f1461800700 1 -- 172.21.15.99:0/1836188957 wait complete. 2024-03-30T00:37:34.406 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:33:14.085184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:33:15.090699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:37:34.406 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:37:34.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:34 smithi161 ceph-mon[42221]: pgmap v389: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 2.40k op/s 2024-03-30T00:37:34.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:34 smithi142 ceph-mon[41779]: pgmap v389: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 2.40k op/s 2024-03-30T00:37:34.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.596+0000 7ffba2c00700 1 -- 172.21.15.99:0/761353982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9c102910 msgr2=0x7ffb9c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:34.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.596+0000 7ffba2c00700 1 --2- 172.21.15.99:0/761353982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9c102910 0x7ffb9c1091c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400b0d0 tx=0x7ffb9400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.596+0000 7ffba2c00700 1 -- 172.21.15.99:0/761353982 shutdown_connections 2024-03-30T00:37:34.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.596+0000 7ffba2c00700 1 --2- 172.21.15.99:0/761353982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9c102910 0x7ffb9c1091c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.596+0000 7ffba2c00700 1 --2- 172.21.15.99:0/761353982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb9c100d80 0x7ffb9c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.596+0000 7ffba2c00700 1 --2- 172.21.15.99:0/761353982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9c101f70 0x7ffb9c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.596+0000 7ffba2c00700 1 -- 172.21.15.99:0/761353982 >> 172.21.15.99:0/761353982 conn(0x7ffb9c0fc2f0 msgr2=0x7ffb9c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:34.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.596+0000 7ffba2c00700 1 -- 172.21.15.99:0/761353982 shutdown_connections 2024-03-30T00:37:34.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.597+0000 7ffba2c00700 1 -- 172.21.15.99:0/761353982 wait complete. 2024-03-30T00:37:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.597+0000 7ffba2c00700 1 Processor -- start 2024-03-30T00:37:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffba2c00700 1 -- start start 2024-03-30T00:37:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffba2c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb9c100d80 0x7ffb9c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffba2c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9c101f70 0x7ffb9c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffba2c00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9c102910 0x7ffb9c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffba2c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c073e80 con 0x7ffb9c101f70 2024-03-30T00:37:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffba2c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c073ff0 con 0x7ffb9c100d80 2024-03-30T00:37:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffba2c00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c1a0d30 con 0x7ffb9c102910 2024-03-30T00:37:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffb9b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9c101f70 0x7ffb9c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffb9b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9c101f70 0x7ffb9c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58242/0 (socket says 172.21.15.99:58242) 2024-03-30T00:37:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffb9b400700 1 -- 172.21.15.99:0/157053273 learned_addr learned my addr 172.21.15.99:0/157053273 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffb9be00700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb9c100d80 0x7ffb9c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.598+0000 7ffba0c00700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9c102910 0x7ffb9c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.599+0000 7ffb9b400700 1 -- 172.21.15.99:0/157053273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9c102910 msgr2=0x7ffb9c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:34.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.599+0000 7ffb9b400700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9c102910 0x7ffb9c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.599+0000 7ffb9b400700 1 -- 172.21.15.99:0/157053273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb9c100d80 msgr2=0x7ffb9c074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:34.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.599+0000 7ffb9b400700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb9c100d80 0x7ffb9c074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.599+0000 7ffb9b400700 1 -- 172.21.15.99:0/157053273 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb94009d00 con 0x7ffb9c101f70 2024-03-30T00:37:34.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.599+0000 7ffb9b400700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9c101f70 0x7ffb9c0731a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800c9b0 tx=0x7ffb8800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:34.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.599+0000 7ffb98c00700 1 -- 172.21.15.99:0/157053273 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88007aa0 con 0x7ffb9c101f70 2024-03-30T00:37:34.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.599+0000 7ffb98c00700 1 -- 172.21.15.99:0/157053273 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb88007c00 con 0x7ffb9c101f70 2024-03-30T00:37:34.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.599+0000 7ffba2c00700 1 -- 172.21.15.99:0/157053273 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c1a0fc0 con 0x7ffb9c101f70 2024-03-30T00:37:34.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.599+0000 7ffb98c00700 1 -- 172.21.15.99:0/157053273 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88018420 con 0x7ffb9c101f70 2024-03-30T00:37:34.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.599+0000 7ffba2c00700 1 -- 172.21.15.99:0/157053273 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c1a1440 con 0x7ffb9c101f70 2024-03-30T00:37:34.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.600+0000 7ffba2c00700 1 -- 172.21.15.99:0/157053273 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9c108130 con 0x7ffb9c101f70 2024-03-30T00:37:34.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.602+0000 7ffb98c00700 1 -- 172.21.15.99:0/157053273 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ffb8800f450 con 0x7ffb9c101f70 2024-03-30T00:37:34.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.603+0000 7ffb98c00700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ffb800a1120 0x7ffb800a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:34.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.603+0000 7ffb98c00700 1 -- 172.21.15.99:0/157053273 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7ffb880ccba0 con 0x7ffb9c101f70 2024-03-30T00:37:34.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.604+0000 7ffb9be00700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ffb800a1120 0x7ffb800a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:34.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.604+0000 7ffb9be00700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ffb800a1120 0x7ffb800a35e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c101dd0 tx=0x7ffb8c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:34.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.606+0000 7ffb98c00700 1 -- 172.21.15.99:0/157053273 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffb880962e0 con 0x7ffb9c101f70 2024-03-30T00:37:34.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.812+0000 7ffba2c00700 1 -- 172.21.15.99:0/157053273 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb9c10c4a0 con 0x7ffb9c101f70 2024-03-30T00:37:34.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.812+0000 7ffb98c00700 1 -- 172.21.15.99:0/157053273 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb8800f700 con 0x7ffb9c101f70 2024-03-30T00:37:34.819 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:37:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.814+0000 7ffba2c00700 1 -- 172.21.15.99:0/157053273 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ffb800a1120 msgr2=0x7ffb800a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.814+0000 7ffba2c00700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ffb800a1120 0x7ffb800a35e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c101dd0 tx=0x7ffb8c00b410 comp rx=0 tx=0).stop 2024-03-30T00:37:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.814+0000 7ffba2c00700 1 -- 172.21.15.99:0/157053273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9c101f70 msgr2=0x7ffb9c0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.814+0000 7ffba2c00700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9c101f70 0x7ffb9c0731a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800c9b0 tx=0x7ffb8800ccc0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.815+0000 7ffba2c00700 1 -- 172.21.15.99:0/157053273 shutdown_connections 2024-03-30T00:37:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.815+0000 7ffba2c00700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb9c102910 0x7ffb9c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.815+0000 7ffba2c00700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ffb800a1120 0x7ffb800a35e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.815+0000 7ffba2c00700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb9c100d80 0x7ffb9c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.815+0000 7ffba2c00700 1 --2- 172.21.15.99:0/157053273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb9c101f70 0x7ffb9c0731a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.815+0000 7ffba2c00700 1 -- 172.21.15.99:0/157053273 >> 172.21.15.99:0/157053273 conn(0x7ffb9c0fc2f0 msgr2=0x7ffb9c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.815+0000 7ffba2c00700 1 -- 172.21.15.99:0/157053273 shutdown_connections 2024-03-30T00:37:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:34.815+0000 7ffba2c00700 1 -- 172.21.15.99:0/157053273 wait complete. 2024-03-30T00:37:34.839 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:37:35.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.027+0000 7fbbff400700 1 -- 172.21.15.99:0/1563962868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8102910 msgr2=0x7fbbf81091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.027+0000 7fbbff400700 1 --2- 172.21.15.99:0/1563962868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8102910 0x7fbbf81091c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000b0d0 tx=0x7fbbf000b3e0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.027+0000 7fbbff400700 1 -- 172.21.15.99:0/1563962868 shutdown_connections 2024-03-30T00:37:35.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.027+0000 7fbbff400700 1 --2- 172.21.15.99:0/1563962868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8102910 0x7fbbf81091c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.027+0000 7fbbff400700 1 --2- 172.21.15.99:0/1563962868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf8100d80 0x7fbbf81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.027+0000 7fbbff400700 1 --2- 172.21.15.99:0/1563962868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbf8101f70 0x7fbbf81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.027+0000 7fbbff400700 1 -- 172.21.15.99:0/1563962868 >> 172.21.15.99:0/1563962868 conn(0x7fbbf80fc2f0 msgr2=0x7fbbf80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:35.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.027+0000 7fbbff400700 1 -- 172.21.15.99:0/1563962868 shutdown_connections 2024-03-30T00:37:35.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.027+0000 7fbbff400700 1 -- 172.21.15.99:0/1563962868 wait complete. 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.028+0000 7fbbff400700 1 Processor -- start 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.028+0000 7fbbff400700 1 -- start start 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbff400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbf8100d80 0x7fbbf8198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbff400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8101f70 0x7fbbf81994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbff400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf8102910 0x7fbbf81a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbff400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf81a0a80 con 0x7fbbf8101f70 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbff400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf81a0bc0 con 0x7fbbf8102910 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbff400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf81a0d50 con 0x7fbbf8100d80 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbf7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8101f70 0x7fbbf81994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbf7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8101f70 0x7fbbf81994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55818/0 (socket says 172.21.15.99:55818) 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbf7e00700 1 -- 172.21.15.99:0/2047279986 learned_addr learned my addr 172.21.15.99:0/2047279986 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:35.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbfca00700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbf8100d80 0x7fbbf8198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbfd400700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf8102910 0x7fbbf81a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbf7e00700 1 -- 172.21.15.99:0/2047279986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbf8100d80 msgr2=0x7fbbf8198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbf7e00700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbf8100d80 0x7fbbf8198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbf7e00700 1 -- 172.21.15.99:0/2047279986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf8102910 msgr2=0x7fbbf81a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbf7e00700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf8102910 0x7fbbf81a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbf7e00700 1 -- 172.21.15.99:0/2047279986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf0009d00 con 0x7fbbf8101f70 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.029+0000 7fbbfd400700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf8102910 0x7fbbf81a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.030+0000 7fbbf7e00700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8101f70 0x7fbbf81994e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400e9a0 tx=0x7fbbe400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.030+0000 7fbbf5600700 1 -- 172.21.15.99:0/2047279986 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4009790 con 0x7fbbf8101f70 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.030+0000 7fbbf5600700 1 -- 172.21.15.99:0/2047279986 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbe400cbd0 con 0x7fbbf8101f70 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.030+0000 7fbbff400700 1 -- 172.21.15.99:0/2047279986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf81a0fd0 con 0x7fbbf8101f70 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.030+0000 7fbbf5600700 1 -- 172.21.15.99:0/2047279986 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4018680 con 0x7fbbf8101f70 2024-03-30T00:37:35.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.030+0000 7fbbff400700 1 -- 172.21.15.99:0/2047279986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf81a14b0 con 0x7fbbf8101f70 2024-03-30T00:37:35.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.031+0000 7fbbff400700 1 -- 172.21.15.99:0/2047279986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf8067af0 con 0x7fbbf8101f70 2024-03-30T00:37:35.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.033+0000 7fbbf5600700 1 -- 172.21.15.99:0/2047279986 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fbbe400cd40 con 0x7fbbf8101f70 2024-03-30T00:37:35.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.034+0000 7fbbf5600700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbe00a1120 0x7fbbe00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.034+0000 7fbbf5600700 1 -- 172.21.15.99:0/2047279986 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fbbe4014070 con 0x7fbbf8101f70 2024-03-30T00:37:35.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.034+0000 7fbbfca00700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbe00a1120 0x7fbbe00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.035+0000 7fbbfca00700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbe00a1120 0x7fbbe00a35e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8101dd0 tx=0x7fbbec007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:35.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.038+0000 7fbbf5600700 1 -- 172.21.15.99:0/2047279986 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbbe40529f0 con 0x7fbbf8101f70 2024-03-30T00:37:35.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.246+0000 7fbbff400700 1 -- 172.21.15.99:0/2047279986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbbf810c450 con 0x7fbbf8101f70 2024-03-30T00:37:35.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.246+0000 7fbbf5600700 1 -- 172.21.15.99:0/2047279986 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbbe4096220 con 0x7fbbf8101f70 2024-03-30T00:37:35.253 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-30T00:37:35.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 -- 172.21.15.99:0/2047279986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbe00a1120 msgr2=0x7fbbe00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbe00a1120 0x7fbbe00a35e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8101dd0 tx=0x7fbbec007400 comp rx=0 tx=0).stop 2024-03-30T00:37:35.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 -- 172.21.15.99:0/2047279986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8101f70 msgr2=0x7fbbf81994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8101f70 0x7fbbf81994e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400e9a0 tx=0x7fbbe400ecb0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 -- 172.21.15.99:0/2047279986 shutdown_connections 2024-03-30T00:37:35.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbe00a1120 0x7fbbe00a35e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf8102910 0x7fbbf81a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbf8100d80 0x7fbbf8198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 --2- 172.21.15.99:0/2047279986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8101f70 0x7fbbf81994e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 -- 172.21.15.99:0/2047279986 >> 172.21.15.99:0/2047279986 conn(0x7fbbf80fc2f0 msgr2=0x7fbbf8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:35.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 -- 172.21.15.99:0/2047279986 shutdown_connections 2024-03-30T00:37:35.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.249+0000 7fbbff400700 1 -- 172.21.15.99:0/2047279986 wait complete. 2024-03-30T00:37:35.274 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:37:35.468 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:35 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1836188957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:37:35.469 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:35 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/157053273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:37:35.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.463+0000 7f71b5000700 1 -- 172.21.15.99:0/2271457277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b0101f70 msgr2=0x7f71b01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.463+0000 7f71b5000700 1 --2- 172.21.15.99:0/2271457277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b0101f70 0x7f71b01023d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f719c0099c0 tx=0x7f719c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.463+0000 7f71b5000700 1 -- 172.21.15.99:0/2271457277 shutdown_connections 2024-03-30T00:37:35.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.463+0000 7f71b5000700 1 --2- 172.21.15.99:0/2271457277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b0102910 0x7f71b01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.463+0000 7f71b5000700 1 --2- 172.21.15.99:0/2271457277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71b0100d80 0x7f71b01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.463+0000 7f71b5000700 1 --2- 172.21.15.99:0/2271457277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b0101f70 0x7f71b01023d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.463+0000 7f71b5000700 1 -- 172.21.15.99:0/2271457277 >> 172.21.15.99:0/2271457277 conn(0x7f71b00fc2f0 msgr2=0x7f71b00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:35.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.463+0000 7f71b5000700 1 -- 172.21.15.99:0/2271457277 shutdown_connections 2024-03-30T00:37:35.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.463+0000 7f71b5000700 1 -- 172.21.15.99:0/2271457277 wait complete. 2024-03-30T00:37:35.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.464+0000 7f71b5000700 1 Processor -- start 2024-03-30T00:37:35.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.465+0000 7f71b5000700 1 -- start start 2024-03-30T00:37:35.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.465+0000 7f71b5000700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71b0100d80 0x7f71b0196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.465+0000 7f71b5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b0101f70 0x7f71b0197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.465+0000 7f71b5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b0102910 0x7f71b019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.465+0000 7f71b5000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b019e720 con 0x7f71b0101f70 2024-03-30T00:37:35.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.465+0000 7f71b5000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b019e860 con 0x7f71b0102910 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.465+0000 7f71b5000700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b019e9f0 con 0x7f71b0100d80 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.465+0000 7f71ad400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b0101f70 0x7f71b0197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.465+0000 7f71ad400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b0101f70 0x7f71b0197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55840/0 (socket says 172.21.15.99:55840) 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.465+0000 7f71ad400700 1 -- 172.21.15.99:0/3473375595 learned_addr learned my addr 172.21.15.99:0/3473375595 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.465+0000 7f71ae800700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b0102910 0x7f71b019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71ade00700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71b0100d80 0x7f71b0196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71ad400700 1 -- 172.21.15.99:0/3473375595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71b0100d80 msgr2=0x7f71b0196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71ad400700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71b0100d80 0x7f71b0196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71ad400700 1 -- 172.21.15.99:0/3473375595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b0102910 msgr2=0x7f71b019e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71ad400700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b0102910 0x7f71b019e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71ad400700 1 -- 172.21.15.99:0/3473375595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c0095f0 con 0x7f71b0101f70 2024-03-30T00:37:35.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71ad400700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b0101f70 0x7f71b0197130 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f719c0098e0 tx=0x7f719c003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:35.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71a6a00700 1 -- 172.21.15.99:0/3473375595 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c01d070 con 0x7f71b0101f70 2024-03-30T00:37:35.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71a6a00700 1 -- 172.21.15.99:0/3473375595 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f719c00bd50 con 0x7f71b0101f70 2024-03-30T00:37:35.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71b5000700 1 -- 172.21.15.99:0/3473375595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b019ec70 con 0x7f71b0101f70 2024-03-30T00:37:35.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71a6a00700 1 -- 172.21.15.99:0/3473375595 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c022c80 con 0x7f71b0101f70 2024-03-30T00:37:35.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.466+0000 7f71b5000700 1 -- 172.21.15.99:0/3473375595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b019f120 con 0x7f71b0101f70 2024-03-30T00:37:35.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.467+0000 7f71b5000700 1 -- 172.21.15.99:0/3473375595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b0108250 con 0x7f71b0101f70 2024-03-30T00:37:35.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.469+0000 7f71a6a00700 1 -- 172.21.15.99:0/3473375595 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f719c022470 con 0x7f71b0101f70 2024-03-30T00:37:35.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.470+0000 7f71a6a00700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71940a1120 0x7f71940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.471+0000 7f71a6a00700 1 -- 172.21.15.99:0/3473375595 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f719c0cdec0 con 0x7f71b0101f70 2024-03-30T00:37:35.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.471+0000 7f71ade00700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71940a1120 0x7f71940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.471+0000 7f71ade00700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71940a1120 0x7f71940a35e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f71a0005fd0 tx=0x7f71a0005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:35.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.475+0000 7f71a6a00700 1 -- 172.21.15.99:0/3473375595 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f719c097600 con 0x7f71b0101f70 2024-03-30T00:37:35.486 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:35 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1836188957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:37:35.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:35 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/157053273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:37:35.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:35 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1836188957' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:37:35.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:35 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/157053273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:37:35.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.681+0000 7f71b5000700 1 -- 172.21.15.99:0/3473375595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f71b002cd00 con 0x7f71b0101f70 2024-03-30T00:37:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.682+0000 7f71a6a00700 1 -- 172.21.15.99:0/3473375595 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f719c0270a0 con 0x7f71b0101f70 2024-03-30T00:37:35.688 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-30T00:37:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 -- 172.21.15.99:0/3473375595 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71940a1120 msgr2=0x7f71940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71940a1120 0x7f71940a35e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f71a0005fd0 tx=0x7f71a0005f60 comp rx=0 tx=0).stop 2024-03-30T00:37:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 -- 172.21.15.99:0/3473375595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b0101f70 msgr2=0x7f71b0197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b0101f70 0x7f71b0197130 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f719c0098e0 tx=0x7f719c003ad0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 -- 172.21.15.99:0/3473375595 shutdown_connections 2024-03-30T00:37:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f71940a1120 0x7f71940a35e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71b0102910 0x7f71b019e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71b0100d80 0x7f71b0196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 --2- 172.21.15.99:0/3473375595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71b0101f70 0x7f71b0197130 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 -- 172.21.15.99:0/3473375595 >> 172.21.15.99:0/3473375595 conn(0x7f71b00fc2f0 msgr2=0x7f71b0105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:35.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 -- 172.21.15.99:0/3473375595 shutdown_connections 2024-03-30T00:37:35.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.684+0000 7f71b5000700 1 -- 172.21.15.99:0/3473375595 wait complete. 2024-03-30T00:37:35.709 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-30T00:37:35.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.893+0000 7f7514400700 1 -- 172.21.15.99:0/987864125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750c102810 msgr2=0x7f750c1090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.893+0000 7f7514400700 1 --2- 172.21.15.99:0/987864125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750c102810 0x7f750c1090c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f750400b0d0 tx=0x7f750400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.893+0000 7f7514400700 1 -- 172.21.15.99:0/987864125 shutdown_connections 2024-03-30T00:37:35.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.893+0000 7f7514400700 1 --2- 172.21.15.99:0/987864125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750c102810 0x7f750c1090c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.893+0000 7f7514400700 1 --2- 172.21.15.99:0/987864125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750c100c80 0x7f750c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.893+0000 7f7514400700 1 --2- 172.21.15.99:0/987864125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f750c101e70 0x7f750c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.893+0000 7f7514400700 1 -- 172.21.15.99:0/987864125 >> 172.21.15.99:0/987864125 conn(0x7f750c0fc1f0 msgr2=0x7f750c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:35.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.893+0000 7f7514400700 1 -- 172.21.15.99:0/987864125 shutdown_connections 2024-03-30T00:37:35.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.893+0000 7f7514400700 1 -- 172.21.15.99:0/987864125 wait complete. 2024-03-30T00:37:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.894+0000 7f7514400700 1 Processor -- start 2024-03-30T00:37:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7514400700 1 -- start start 2024-03-30T00:37:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7514400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750c100c80 0x7f750c198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7514400700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f750c101e70 0x7f750c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7514400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750c102810 0x7f750c1a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7514400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c1a0990 con 0x7f750c100c80 2024-03-30T00:37:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7514400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c1a0ad0 con 0x7f750c102810 2024-03-30T00:37:35.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7514400700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750c1a0c60 con 0x7f750c101e70 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7511a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750c100c80 0x7f750c198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7511a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750c100c80 0x7f750c198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55860/0 (socket says 172.21.15.99:55860) 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7511a00700 1 -- 172.21.15.99:0/4127265247 learned_addr learned my addr 172.21.15.99:0/4127265247 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7512400700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750c102810 0x7f750c1a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.895+0000 7f7511000700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f750c101e70 0x7f750c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.896+0000 7f7511a00700 1 -- 172.21.15.99:0/4127265247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f750c101e70 msgr2=0x7f750c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.896+0000 7f7511a00700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f750c101e70 0x7f750c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.896+0000 7f7511a00700 1 -- 172.21.15.99:0/4127265247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750c102810 msgr2=0x7f750c1a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.896+0000 7f7511a00700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750c102810 0x7f750c1a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.896+0000 7f7511a00700 1 -- 172.21.15.99:0/4127265247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7504009d00 con 0x7f750c100c80 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.896+0000 7f7511a00700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750c100c80 0x7f750c198f00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f750000d980 tx=0x7f750000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.896+0000 7f750a000700 1 -- 172.21.15.99:0/4127265247 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750000f890 con 0x7f750c100c80 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.896+0000 7f750a000700 1 -- 172.21.15.99:0/4127265247 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f750000fe80 con 0x7f750c100c80 2024-03-30T00:37:35.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.896+0000 7f7514400700 1 -- 172.21.15.99:0/4127265247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f750c1a0f40 con 0x7f750c100c80 2024-03-30T00:37:35.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.896+0000 7f750a000700 1 -- 172.21.15.99:0/4127265247 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500019360 con 0x7f750c100c80 2024-03-30T00:37:35.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.896+0000 7f7514400700 1 -- 172.21.15.99:0/4127265247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750c1a1360 con 0x7f750c100c80 2024-03-30T00:37:35.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.898+0000 7f7514400700 1 -- 172.21.15.99:0/4127265247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f750c067a10 con 0x7f750c100c80 2024-03-30T00:37:35.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.899+0000 7f750a000700 1 -- 172.21.15.99:0/4127265247 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f750000f9f0 con 0x7f750c100c80 2024-03-30T00:37:35.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.901+0000 7f750a000700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74f40a1120 0x7f74f40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:37:35.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.901+0000 7f750a000700 1 -- 172.21.15.99:0/4127265247 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f75000cc3a0 con 0x7f750c100c80 2024-03-30T00:37:35.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.901+0000 7f7511000700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74f40a1120 0x7f74f40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:37:35.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.901+0000 7f7511000700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74f40a1120 0x7f74f40a35e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f74f8005950 tx=0x7f74f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:37:35.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:35.905+0000 7f750a000700 1 -- 172.21.15.99:0/4127265247 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7500095a50 con 0x7f750c100c80 2024-03-30T00:37:36.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.169+0000 7f7514400700 1 -- 172.21.15.99:0/4127265247 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f750c10c3a0 con 0x7f750c100c80 2024-03-30T00:37:36.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.180+0000 7f750a000700 1 -- 172.21.15.99:0/4127265247 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733393 (secure 0 0 0) 0x7f7500099070 con 0x7f750c100c80 2024-03-30T00:37:36.197 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:37:36.197 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:37:36.197 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 18, 2024-03-30T00:37:36.197 INFO:teuthology.orchestra.run.smithi099.stdout: "active_gid": 14150, 2024-03-30T00:37:36.197 INFO:teuthology.orchestra.run.smithi099.stdout: "active_name": "x", 2024-03-30T00:37:36.197 INFO:teuthology.orchestra.run.smithi099.stdout: "active_addrs": { 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1501731896 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1501731896 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: "active_addr": "172.21.15.99:6801/1501731896", 2024-03-30T00:37:36.198 INFO:teuthology.orchestra.run.smithi099.stdout: "active_change": "2024-03-30T00:27:08.933104+0000", 2024-03-30T00:37:36.199 INFO:teuthology.orchestra.run.smithi099.stdout: "active_mgr_features": 4540138322906710015, 2024-03-30T00:37:36.199 INFO:teuthology.orchestra.run.smithi099.stdout: "available": true, 2024-03-30T00:37:36.199 INFO:teuthology.orchestra.run.smithi099.stdout: "standbys": [ 2024-03-30T00:37:36.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.199 INFO:teuthology.orchestra.run.smithi099.stdout: "gid": 14220, 2024-03-30T00:37:36.199 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "y", 2024-03-30T00:37:36.199 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr_features": 4540138322906710015, 2024-03-30T00:37:36.199 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2024-03-30T00:37:36.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.199 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2024-03-30T00:37:36.200 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.201 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.202 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.203 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.206 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2024-03-30T00:37:36.207 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2024-03-30T00:37:36.208 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.209 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.209 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.209 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2024-03-30T00:37:36.209 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2024-03-30T00:37:36.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.209 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.209 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.209 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.209 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.210 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.211 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.212 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.212 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.212 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2024-03-30T00:37:36.212 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.212 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.212 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.212 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2024-03-30T00:37:36.212 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2024-03-30T00:37:36.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.213 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.215 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2024-03-30T00:37:36.215 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.215 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.215 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.215 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2024-03-30T00:37:36.215 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2024-03-30T00:37:36.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.215 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.215 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.215 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2024-03-30T00:37:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-30T00:37:36.217 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:36.218 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.219 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.220 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-30T00:37:36.220 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-30T00:37:36.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.220 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.220 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2024-03-30T00:37:36.220 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2024-03-30T00:37:36.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.220 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2024-03-30T00:37:36.221 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2024-03-30T00:37:36.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.224 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.225 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.226 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.227 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2024-03-30T00:37:36.228 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-30T00:37:36.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap" 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2024-03-30T00:37:36.230 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-30T00:37:36.231 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.233 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.235 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.236 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.237 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.238 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-30T00:37:36.239 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.240 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2024-03-30T00:37:36.243 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.244 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-30T00:37:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-30T00:37:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2024-03-30T00:37:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2024-03-30T00:37:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "hw_monitoring": { 2024-03-30T00:37:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hw_monitoring", 2024-03-30T00:37:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "inventory_list_all": { 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "inventory_list_all", 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-30T00:37:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-30T00:37:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-30T00:37:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-30T00:37:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-30T00:37:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2024-03-30T00:37:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2024-03-30T00:37:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2024-03-30T00:37:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "oob_default_addr": { 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "oob_default_addr", 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "169.254.1.1", 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2024-03-30T00:37:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2024-03-30T00:37:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-30T00:37:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2024-03-30T00:37:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2024-03-30T00:37:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2024-03-30T00:37:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2024-03-30T00:37:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-30T00:37:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2024-03-30T00:37:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-30T00:37:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-30T00:37:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-30T00:37:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-30T00:37:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-30T00:37:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2024-03-30T00:37:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-30T00:37:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-30T00:37:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-30T00:37:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-30T00:37:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-30T00:37:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-30T00:37:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-30T00:37:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-30T00:37:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-30T00:37:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-30T00:37:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-30T00:37:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-30T00:37:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-30T00:37:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-30T00:37:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-30T00:37:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-30T00:37:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-30T00:37:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-30T00:37:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-30T00:37:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-30T00:37:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-30T00:37:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-30T00:37:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-30T00:37:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-30T00:37:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2024-03-30T00:37:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-30T00:37:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-30T00:37:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "UNSAFE_TLS_v1_2": { 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-30T00:37:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-30T00:37:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2024-03-30T00:37:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2024-03-30T00:37:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2024-03-30T00:37:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2024-03-30T00:37:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.370 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.371 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.374 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.375 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.376 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.376 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-03-30T00:37:36.376 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.376 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.376 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.377 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2024-03-30T00:37:36.377 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2024-03-30T00:37:36.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.377 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.377 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.377 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.377 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.377 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.377 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.378 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2024-03-30T00:37:36.378 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.378 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.378 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.378 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.379 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.379 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.379 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.379 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.379 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.379 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.379 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.379 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.379 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.379 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.380 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.380 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.380 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.380 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.380 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.380 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.381 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.381 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.381 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.381 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.381 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.382 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.382 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.382 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.382 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.382 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.383 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.383 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.383 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.383 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.383 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.383 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.383 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.384 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2024-03-30T00:37:36.385 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2024-03-30T00:37:36.386 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2024-03-30T00:37:36.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.387 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.387 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.387 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2024-03-30T00:37:36.387 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.387 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.387 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.387 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-30T00:37:36.387 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.388 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.389 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.390 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.391 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.392 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.393 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.394 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.395 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.396 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.399 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.399 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2024-03-30T00:37:36.399 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2024-03-30T00:37:36.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.399 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.399 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.399 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-30T00:37:36.399 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2024-03-30T00:37:36.400 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.402 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2024-03-30T00:37:36.403 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2024-03-30T00:37:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:36.405 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2024-03-30T00:37:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2024-03-30T00:37:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2024-03-30T00:37:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2024-03-30T00:37:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-30T00:37:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-30T00:37:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2024-03-30T00:37:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-30T00:37:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2024-03-30T00:37:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2024-03-30T00:37:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-30T00:37:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2024-03-30T00:37:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2024-03-30T00:37:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2024-03-30T00:37:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2024-03-30T00:37:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-03-30T00:37:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2024-03-30T00:37:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2024-03-30T00:37:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2024-03-30T00:37:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2024-03-30T00:37:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2024-03-30T00:37:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.504 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-30T00:37:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-30T00:37:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.507 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.508 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.509 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.510 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.511 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.512 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-30T00:37:36.513 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-30T00:37:36.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.515 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2024-03-30T00:37:36.516 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2024-03-30T00:37:36.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.516 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.516 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.516 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2024-03-30T00:37:36.516 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.516 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.516 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.516 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.516 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.516 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.517 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-30T00:37:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.520 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.521 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.522 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.525 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.527 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.527 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.527 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.527 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.527 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.527 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.527 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.528 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.529 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.530 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.533 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.533 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.533 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.533 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.533 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.533 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.533 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.533 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.535 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.536 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.537 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2024-03-30T00:37:36.538 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.538 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.538 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.538 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.538 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.538 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.538 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.538 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.539 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.542 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.543 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.544 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.545 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.546 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.548 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.548 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.548 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2024-03-30T00:37:36.548 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2024-03-30T00:37:36.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.548 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.548 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.548 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2024-03-30T00:37:36.548 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.549 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.549 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.549 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-30T00:37:36.549 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.549 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.550 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.551 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.551 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.551 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.551 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.551 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.551 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.551 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.551 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.551 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.551 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.551 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.552 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.553 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.554 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.554 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.554 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.554 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.554 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.554 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.554 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.554 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.554 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.554 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.554 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.555 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.556 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2024-03-30T00:37:36.557 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.558 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.558 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.558 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.558 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.558 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.558 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.558 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2024-03-30T00:37:36.558 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2024-03-30T00:37:36.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.558 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2024-03-30T00:37:36.559 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.561 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.562 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2024-03-30T00:37:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2024-03-30T00:37:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.564 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.565 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.565 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.565 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.565 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.565 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.565 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.565 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2024-03-30T00:37:36.565 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2024-03-30T00:37:36.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.566 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.566 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.566 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:36.566 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2024-03-30T00:37:36.566 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.566 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.566 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.566 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.567 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.568 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.569 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.570 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.572 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.573 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.574 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.576 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.579 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.580 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.581 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.582 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.583 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.584 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.587 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.587 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.587 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.587 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.587 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.587 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.587 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.587 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.587 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.589 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.590 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.591 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.592 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.593 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.594 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.595 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.596 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.596 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.596 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.596 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.596 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.596 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.596 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.596 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.596 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.597 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.599 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.600 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.601 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.602 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.602 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.602 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.602 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.602 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.602 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-30T00:37:36.602 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.602 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.602 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.602 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.604 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2024-03-30T00:37:36.604 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2024-03-30T00:37:36.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.604 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.604 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.604 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.604 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.604 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.604 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-30T00:37:36.604 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.605 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-30T00:37:36.606 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.606 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.606 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.606 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2024-03-30T00:37:36.606 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2024-03-30T00:37:36.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.606 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.606 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.606 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.606 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.607 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2024-03-30T00:37:36.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-30T00:37:36.609 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.610 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.611 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.612 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.613 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.614 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.615 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.616 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.617 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.618 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.619 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.620 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.621 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.622 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.623 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.624 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.625 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.627 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.628 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.629 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.630 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.631 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.632 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.633 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.634 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.635 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.636 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.637 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.638 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_no_wait": { 2024-03-30T00:37:36.639 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_no_wait", 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2024-03-30T00:37:36.641 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.642 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:36.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.644 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.645 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.645 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:36.645 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.645 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.645 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.645 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.645 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.646 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.647 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.648 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.649 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.651 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.652 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.653 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.654 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: "gid": 24109, 2024-03-30T00:37:36.655 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "z", 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr_features": 4540138322906710015, 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.656 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.657 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.658 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.658 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.658 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.658 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.658 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.658 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.658 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.658 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.658 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.658 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.658 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.659 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.660 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.660 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.660 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.660 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.660 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.660 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.660 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.660 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.660 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.661 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.663 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2024-03-30T00:37:36.664 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2024-03-30T00:37:36.667 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.668 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.669 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.670 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.671 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.673 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2024-03-30T00:37:36.674 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.675 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:36.676 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2024-03-30T00:37:36.677 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.679 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2024-03-30T00:37:36.680 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.681 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.684 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.686 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.687 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2024-03-30T00:37:36.689 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap" 2024-03-30T00:37:36.690 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.690 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2024-03-30T00:37:36.690 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.690 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.690 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2024-03-30T00:37:36.690 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2024-03-30T00:37:36.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.690 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.690 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:36.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.692 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.694 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2024-03-30T00:37:36.694 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2024-03-30T00:37:36.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:36.694 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.694 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.694 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.694 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.694 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.694 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.694 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-30T00:37:36.694 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2024-03-30T00:37:36.695 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2024-03-30T00:37:36.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.696 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.696 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-30T00:37:36.696 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.696 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.696 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.696 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-30T00:37:36.696 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.696 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.697 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.698 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.698 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2024-03-30T00:37:36.698 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2024-03-30T00:37:36.698 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.698 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.698 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.698 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2024-03-30T00:37:36.698 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.698 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.698 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.699 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.700 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2024-03-30T00:37:36.701 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.702 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.704 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.705 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.705 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2024-03-30T00:37:36.705 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2024-03-30T00:37:36.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.705 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.705 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.705 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.705 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.705 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.705 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.706 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-30T00:37:36.706 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.706 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.706 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.706 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2024-03-30T00:37:36.706 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2024-03-30T00:37:36.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.706 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.706 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.706 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.707 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2024-03-30T00:37:36.708 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2024-03-30T00:37:36.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.709 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.709 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.709 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-30T00:37:36.709 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.709 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.709 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.709 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2024-03-30T00:37:36.709 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.709 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.709 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:36.710 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.711 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.711 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2024-03-30T00:37:36.711 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2024-03-30T00:37:36.711 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.711 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.711 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.711 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-30T00:37:36.711 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.711 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.712 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.713 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2024-03-30T00:37:36.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.715 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.718 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2024-03-30T00:37:36.718 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2024-03-30T00:37:36.718 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.718 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.718 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.718 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-30T00:37:36.718 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.718 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.718 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.718 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:36.718 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.719 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.721 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.721 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:36.721 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.721 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.721 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.721 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2024-03-30T00:37:36.721 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2024-03-30T00:37:36.721 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.721 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.721 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2024-03-30T00:37:36.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.723 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.725 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.726 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.727 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.728 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.729 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2024-03-30T00:37:36.730 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2024-03-30T00:37:36.731 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2024-03-30T00:37:36.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.732 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.732 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.732 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:36.732 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.732 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.732 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.732 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2024-03-30T00:37:36.732 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.732 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "hw_monitoring": { 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hw_monitoring", 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-30T00:37:36.733 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: "inventory_list_all": { 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "inventory_list_all", 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.734 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.735 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.736 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.737 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.738 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.739 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.741 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-30T00:37:36.742 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.743 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.744 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.745 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.746 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.746 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-30T00:37:36.746 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.746 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.746 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.746 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2024-03-30T00:37:36.746 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2024-03-30T00:37:36.746 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.746 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.746 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.747 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.748 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "oob_default_addr": { 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "oob_default_addr", 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "169.254.1.1", 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.750 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.751 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.752 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.753 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.754 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2024-03-30T00:37:36.755 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.756 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.758 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.759 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.760 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.761 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.762 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.765 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.767 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.767 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.767 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.767 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.767 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.767 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.767 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.767 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.767 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.767 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.768 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.770 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.771 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.771 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2024-03-30T00:37:36.771 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2024-03-30T00:37:36.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.771 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.771 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2024-03-30T00:37:36.771 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.771 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.771 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.772 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.774 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.775 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.775 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.775 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2024-03-30T00:37:36.775 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-30T00:37:36.775 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.775 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.775 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.775 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.775 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.775 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.776 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.777 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.777 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.777 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.777 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.777 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.777 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.777 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.777 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2024-03-30T00:37:36.777 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2024-03-30T00:37:36.777 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.778 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.778 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.778 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.778 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.778 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.778 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.778 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.778 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.778 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.779 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-30T00:37:36.779 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-30T00:37:36.779 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.779 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.779 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.779 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.779 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.779 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.779 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.779 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.779 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.780 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.781 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.782 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.783 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-30T00:37:36.784 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.785 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-30T00:37:36.786 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-30T00:37:36.786 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.786 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.786 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.786 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.786 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.786 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.786 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.786 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.786 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.786 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.787 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.788 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.789 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.789 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.789 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.789 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.789 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.789 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-30T00:37:36.789 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-30T00:37:36.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.789 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-30T00:37:36.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.791 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.791 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.791 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-30T00:37:36.791 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.791 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.791 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.791 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.791 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.791 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.791 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.793 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.794 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.795 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.796 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-30T00:37:36.797 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.800 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.801 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.802 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.802 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.802 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-30T00:37:36.802 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-30T00:37:36.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.802 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.802 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.802 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.803 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.804 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.804 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.804 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.804 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.804 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.804 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.804 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.804 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.804 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-30T00:37:36.804 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.805 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-30T00:37:36.806 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-30T00:37:36.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.806 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.806 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.806 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.806 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.806 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.806 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.806 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.807 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.808 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.809 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.809 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.809 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.809 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.809 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.809 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-30T00:37:36.809 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-30T00:37:36.809 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.809 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.809 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.810 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.810 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.810 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.810 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.810 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.810 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.810 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.810 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2024-03-30T00:37:36.810 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.813 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.814 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.814 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.814 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-30T00:37:36.814 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-30T00:37:36.814 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.814 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.814 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.814 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2024-03-30T00:37:36.814 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.814 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.815 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.816 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-30T00:37:36.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2024-03-30T00:37:36.818 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.821 INFO:teuthology.orchestra.run.smithi099.stdout: "UNSAFE_TLS_v1_2": { 2024-03-30T00:37:36.821 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-30T00:37:36.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.821 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.821 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.821 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.821 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.821 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.821 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.821 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.822 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.822 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.822 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-30T00:37:36.822 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-30T00:37:36.822 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.822 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.822 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.822 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:36.822 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.823 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.823 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.823 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.823 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.823 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.823 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.823 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-30T00:37:36.823 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-30T00:37:36.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.823 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-30T00:37:36.824 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.825 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.826 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.827 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2024-03-30T00:37:36.827 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2024-03-30T00:37:36.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.827 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.827 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.827 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.827 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.827 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.827 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.827 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.828 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.828 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.828 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2024-03-30T00:37:36.828 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2024-03-30T00:37:36.828 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.828 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.828 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.828 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.828 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2024-03-30T00:37:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2024-03-30T00:37:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2024-03-30T00:37:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-30T00:37:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-30T00:37:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2024-03-30T00:37:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2024-03-30T00:37:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2024-03-30T00:37:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-30T00:37:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-30T00:37:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2024-03-30T00:37:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-30T00:37:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-03-30T00:37:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2024-03-30T00:37:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2024-03-30T00:37:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-30T00:37:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2024-03-30T00:37:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2024-03-30T00:37:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2024-03-30T00:37:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2024-03-30T00:37:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-30T00:37:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2024-03-30T00:37:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2024-03-30T00:37:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2024-03-30T00:37:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2024-03-30T00:37:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2024-03-30T00:37:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2024-03-30T00:37:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2024-03-30T00:37:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-30T00:37:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2024-03-30T00:37:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2024-03-30T00:37:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2024-03-30T00:37:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-30T00:37:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2024-03-30T00:37:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2024-03-30T00:37:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2024-03-30T00:37:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2024-03-30T00:37:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2024-03-30T00:37:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2024-03-30T00:37:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2024-03-30T00:37:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2024-03-30T00:37:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2024-03-30T00:37:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2024-03-30T00:37:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2024-03-30T00:37:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2024-03-30T00:37:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-30T00:37:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2024-03-30T00:37:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2024-03-30T00:37:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2024-03-30T00:37:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2024-03-30T00:37:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2024-03-30T00:37:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2024-03-30T00:37:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2024-03-30T00:37:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2024-03-30T00:37:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2024-03-30T00:37:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.992 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.993 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.994 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.995 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.996 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.997 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.998 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:36.999 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2024-03-30T00:37:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2024-03-30T00:37:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2024-03-30T00:37:37.000 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-30T00:37:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-30T00:37:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.001 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-30T00:37:37.002 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.003 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.004 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.005 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-30T00:37:37.006 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.007 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.008 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.009 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.010 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.011 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.012 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.013 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2024-03-30T00:37:37.014 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.015 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.016 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.017 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.017 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.017 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2024-03-30T00:37:37.017 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.017 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.017 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.017 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2024-03-30T00:37:37.017 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2024-03-30T00:37:37.017 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.018 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-30T00:37:37.019 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-30T00:37:37.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.019 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.019 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.019 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.019 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.019 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.019 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.019 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.019 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.020 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.021 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.023 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.024 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-30T00:37:37.025 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.028 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2024-03-30T00:37:37.028 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.028 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.028 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.028 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.028 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.028 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.029 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.030 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.031 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.032 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.032 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.032 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.032 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.032 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.032 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.032 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.032 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.032 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.032 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.032 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.033 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2024-03-30T00:37:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2024-03-30T00:37:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2024-03-30T00:37:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2024-03-30T00:37:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2024-03-30T00:37:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.060 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.061 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2024-03-30T00:37:37.062 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.064 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.065 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.065 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.065 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.065 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2024-03-30T00:37:37.065 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.065 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.065 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.065 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2024-03-30T00:37:37.065 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2024-03-30T00:37:37.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.066 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.067 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.068 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.069 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.069 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.069 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.069 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.069 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.069 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.069 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.069 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.069 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.070 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.071 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.071 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.071 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.071 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.071 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.071 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.071 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.071 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.071 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.073 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.074 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.075 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.076 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.077 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.077 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.077 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.077 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.077 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.077 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.077 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.077 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.077 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.077 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.078 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.079 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.080 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.081 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.081 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.081 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.081 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.081 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.081 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.081 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.081 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.081 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.082 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.083 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.084 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.086 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2024-03-30T00:37:37.087 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.088 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.089 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.090 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.090 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.090 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.090 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.090 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.090 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.090 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.091 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.091 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.091 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.091 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.091 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.091 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.091 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.091 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.092 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.093 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.094 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.095 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2024-03-30T00:37:37.095 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2024-03-30T00:37:37.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.095 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.095 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.095 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.095 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.095 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.095 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-30T00:37:37.095 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.096 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.097 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-30T00:37:37.097 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.097 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.097 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.097 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2024-03-30T00:37:37.097 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2024-03-30T00:37:37.097 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.097 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.097 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.097 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.098 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2024-03-30T00:37:37.099 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.101 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.102 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2024-03-30T00:37:37.102 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2024-03-30T00:37:37.102 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.102 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.102 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.102 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.102 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.102 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.102 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.102 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.103 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.104 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.105 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.106 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2024-03-30T00:37:37.107 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.108 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.110 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.111 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.112 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.113 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.114 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2024-03-30T00:37:37.115 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.117 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.118 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.119 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.120 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.121 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.122 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.123 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.124 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.126 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.127 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.127 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.127 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.127 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.127 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.127 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.127 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.127 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.128 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.129 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.130 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2024-03-30T00:37:37.132 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_no_wait": { 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_no_wait", 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.134 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2024-03-30T00:37:37.135 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2024-03-30T00:37:37.136 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.137 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:37.138 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:37.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.138 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.138 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.138 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:37.138 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.138 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.138 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.138 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.138 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.138 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.139 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.140 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.141 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.142 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.144 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.146 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.147 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: "modules": [ 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: "dashboard", 2024-03-30T00:37:37.148 INFO:teuthology.orchestra.run.smithi099.stdout: "iostat", 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: "nfs", 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: "restful" 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: "available_modules": [ 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "alerts", 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.149 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How frequently to reexamine health status", 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.150 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.151 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.152 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.153 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.154 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.155 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.156 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.156 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.156 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.156 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.156 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_destination": { 2024-03-30T00:37:37.156 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_destination", 2024-03-30T00:37:37.156 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.156 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.156 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.156 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email address to send alerts to", 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_from_name": { 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_from_name", 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.157 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "Ceph", 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Email From: name", 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_host": { 2024-03-30T00:37:37.158 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_host", 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP server", 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_password": { 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_password", 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Password to authenticate with", 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.160 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_port": { 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_port", 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "465", 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.161 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP port", 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_sender": { 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_sender", 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SMTP envelope sender", 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_ssl": { 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_ssl", 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.163 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: "smtp_user": { 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "smtp_user", 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.164 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "User to authenticate as", 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "balancer", 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: "active": { 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "active", 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.166 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically balance PGs across cluster", 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_time": { 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_time", 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0000", 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "beginning time of day to automatically balance", 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "begin_weekday": { 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "begin_weekday", 2024-03-30T00:37:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_max_iterations": { 2024-03-30T00:37:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_max_iterations", 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "25", 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "250", 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_metrics": { 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_metrics", 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "pgs,objects,bytes", 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-30T00:37:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_compat_step": { 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crush_compat_step", 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.5", 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0.001", 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "0.999", 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "aggressiveness of optimization", 2024-03-30T00:37:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "end_time": { 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_time", 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2359", 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ending time of day to automatically balance", 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "end_weekday": { 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "end_weekday", 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "0", 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "6", 2024-03-30T00:37:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "min_score": { 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_score", 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-30T00:37:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-30T00:37:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-30T00:37:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "upmap", 2024-03-30T00:37:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-compat", 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "none", 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap" 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Balancer mode", 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_ids": { 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_ids", 2024-03-30T00:37:37.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:37.184 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_deviation": { 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_deviation", 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-30T00:37:37.186 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "upmap_max_optimizations": { 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "upmap_max_optimizations", 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.187 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephadm", 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_down_multiplier": { 2024-03-30T00:37:37.188 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_down_multiplier", 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_refresh_rate": { 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_refresh_rate", 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "20", 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.190 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "agent_starting_port": { 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "agent_starting_port", 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4721", 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.191 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_ptrace": { 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_ptrace", 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.192 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_interval": { 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_interval", 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.193 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "autotune_memory_target_ratio": { 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "autotune_memory_target_ratio", 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.194 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0.7", 2024-03-30T00:37:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-30T00:37:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "cgroups_split": { 2024-03-30T00:37:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cgroups_split", 2024-03-30T00:37:37.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.196 INFO:teuthology.orchestra.run.smithi099.stdout: "config_checks_enabled": { 2024-03-30T00:37:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_checks_enabled", 2024-03-30T00:37:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-30T00:37:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.197 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "config_dashboard": { 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_dashboard", 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.198 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_alertmanager": { 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_alertmanager", 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.199 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_base": { 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_base", 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.200 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-30T00:37:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Container image name, without the tag", 2024-03-30T00:37:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_elasticsearch": { 2024-03-30T00:37:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_elasticsearch", 2024-03-30T00:37:37.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "elasticsearch container image", 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.202 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_grafana": { 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_grafana", 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.203 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_haproxy": { 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_haproxy", 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "HAproxy container image", 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_agent": { 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_agent", 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger agent container image", 2024-03-30T00:37:37.205 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_collector": { 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_collector", 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.206 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger collector container image", 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_jaeger_query": { 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_jaeger_query", 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.207 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Jaeger query container image", 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_keepalived": { 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_keepalived", 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-30T00:37:37.208 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Keepalived container image", 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_loki": { 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_loki", 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Loki container image", 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_node_exporter": { 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_node_exporter", 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.210 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_nvmeof": { 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_nvmeof", 2024-03-30T00:37:37.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.212 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.212 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-30T00:37:37.212 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.212 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.212 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.212 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Nvme-of container image", 2024-03-30T00:37:37.212 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.212 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.212 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_prometheus": { 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_prometheus", 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Prometheus container image", 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_promtail": { 2024-03-30T00:37:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_promtail", 2024-03-30T00:37:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-30T00:37:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.214 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Promtail container image", 2024-03-30T00:37:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "container_image_snmp_gateway": { 2024-03-30T00:37:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_image_snmp_gateway", 2024-03-30T00:37:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.215 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "SNMP Gateway container image", 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "container_init": { 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "container_init", 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Run podman/docker with `--init`", 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "daemon_cache_timeout": { 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "daemon_cache_timeout", 2024-03-30T00:37:37.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-30T00:37:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "default_cephadm_command_timeout": { 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_cephadm_command_timeout", 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "900", 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "default_registry": { 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "default_registry", 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "docker.io", 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.220 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "device_cache_timeout": { 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_cache_timeout", 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1800", 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.221 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache device inventory", 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "device_enhanced_scan": { 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_enhanced_scan", 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "facts_cache_timeout": { 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "facts_cache_timeout", 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.223 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "seconds to cache host facts data", 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "host_check_interval": { 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "host_check_interval", 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.224 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to perform a host check", 2024-03-30T00:37:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.225 INFO:teuthology.orchestra.run.smithi099.stdout: "hw_monitoring": { 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hw_monitoring", 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.226 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "inventory_list_all": { 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "inventory_list_all", 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.227 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.228 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "log_refresh_metadata": { 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_refresh_metadata", 2024-03-30T00:37:37.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-30T00:37:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.231 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.232 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.233 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.234 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-30T00:37:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-30T00:37:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.235 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "*", 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.236 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "max_count_per_host": { 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_count_per_host", 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:37.237 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of daemons per service per host", 2024-03-30T00:37:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd_draining_count": { 2024-03-30T00:37:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_osd_draining_count", 2024-03-30T00:37:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.238 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "migration_current": { 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "migration_current", 2024-03-30T00:37:37.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "internal - do not modify", 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "mode": { 2024-03-30T00:37:37.240 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mode", 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "root", 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm-package", 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "root" 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "mode for remote execution of cephadm", 2024-03-30T00:37:37.241 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "oob_default_addr": { 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "oob_default_addr", 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "169.254.1.1", 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.242 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "prometheus_alerts_path": { 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus_alerts_path", 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-30T00:37:37.243 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_insecure": { 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_insecure", 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.244 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_password": { 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_password", 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_url": { 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_url", 2024-03-30T00:37:37.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.247 INFO:teuthology.orchestra.run.smithi099.stdout: "registry_username": { 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "registry_username", 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.248 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "secure_monitoring_stack": { 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "secure_monitoring_stack", 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-30T00:37:37.249 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "service_discovery_port": { 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "service_discovery_port", 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8765", 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.250 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "cephadm service discovery port", 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "ssh_config_file": { 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssh_config_file", 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.251 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "use_agent": { 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_agent", 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.252 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "use_repo_digest": { 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "use_repo_digest", 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.253 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_failed_host_check": { 2024-03-30T00:37:37.254 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_failed_host_check", 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if the host check fails", 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.255 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_daemons": { 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_daemons", 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.256 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_on_stray_hosts": { 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_on_stray_hosts", 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-30T00:37:37.257 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crash", 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.258 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.259 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.260 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.261 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.262 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.263 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "retain_interval": { 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retain_interval", 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "31536000", 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.266 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_recent_interval": { 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_recent_interval", 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1209600", 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-30T00:37:37.267 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dashboard", 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-30T00:37:37.268 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_HOST": { 2024-03-30T00:37:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-30T00:37:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.270 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_ENABLED": { 2024-03-30T00:37:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_ENABLED", 2024-03-30T00:37:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.272 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-30T00:37:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-30T00:37:37.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.274 INFO:teuthology.orchestra.run.smithi099.stdout: "ENABLE_BROWSABLE_API": { 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.275 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.276 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.277 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-30T00:37:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-30T00:37:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.278 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.279 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-30T00:37:37.280 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.282 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.283 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.284 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_PASSWORD": { 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-30T00:37:37.285 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_URL": { 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_URL", 2024-03-30T00:37:37.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_API_USERNAME": { 2024-03-30T00:37:37.288 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-30T00:37:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-30T00:37:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.289 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-30T00:37:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-30T00:37:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.290 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.291 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.292 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.293 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_HOST": { 2024-03-30T00:37:37.294 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.295 INFO:teuthology.orchestra.run.smithi099.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.296 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.298 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.299 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.300 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.301 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-30T00:37:37.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-30T00:37:37.303 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_ENABLED": { 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.305 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-30T00:37:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-30T00:37:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-30T00:37:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.306 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:37.307 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.308 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8", 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.309 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "45", 2024-03-30T00:37:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.310 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ACCESS_KEY": { 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "admin", 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SECRET_KEY": { 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.313 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "RGW_API_SSL_VERIFY": { 2024-03-30T00:37:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-30T00:37:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.314 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "UNSAFE_TLS_v1_2": { 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.315 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.316 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-30T00:37:37.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.318 INFO:teuthology.orchestra.run.smithi099.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "cross_origin_url": { 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cross_origin_url", 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.320 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "crt_file": { 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "crt_file", 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.321 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "debug": { 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "debug", 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.322 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Enable/disable debug options", 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "jwt_token_ttl": { 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "jwt_token_ttl", 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "28800", 2024-03-30T00:37:37.323 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.325 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.326 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.327 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.329 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.330 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "motd": { 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "motd", 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.331 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "The message of the day", 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect_resolve_ip_addr": { 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "redirect_resolve_ip_addr", 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.332 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.333 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-30T00:37:37.334 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8080", 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-30T00:37:37.335 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.336 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl_server_port": { 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl_server_port", 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8443", 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.337 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "redirect", 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.338 INFO:teuthology.orchestra.run.smithi099.stdout: "redirect" 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-30T00:37:37.339 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: "url_prefix": { 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url_prefix", 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.340 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.342 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-03-30T00:37:37.342 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.342 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.342 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.342 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_monitoring": { 2024-03-30T00:37:37.342 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_monitoring", 2024-03-30T00:37:37.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.342 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.342 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.342 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "monitor device health metrics", 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.343 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.344 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.345 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.346 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "mark_out_threshold": { 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mark_out_threshold", 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "2419200", 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": { 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pool_name", 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "device_health_metrics", 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "retention_period": { 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "retention_period", 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15552000", 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long to retain device health metrics", 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_frequency": { 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_frequency", 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "self_heal": { 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "self_heal", 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how frequently to wake up and check device health", 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "warn_threshold": { 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "warn_threshold", 2024-03-30T00:37:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7257600", 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "diskprediction_local", 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "predict_interval": { 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predict_interval", 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "86400", 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "predictor_model": { 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "predictor_model", 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "prophetstor", 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "600", 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "influx", 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": false, 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "influxdb python module not found", 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "batch_size": { 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "batch_size", 2024-03-30T00:37:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5000", 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "database": { 2024-03-30T00:37:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "database", 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "ceph", 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": { 2024-03-30T00:37:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "hostname", 2024-03-30T00:37:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server hostname", 2024-03-30T00:37:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "30", 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "5", 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-30T00:37:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "password": { 2024-03-30T00:37:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "password", 2024-03-30T00:37:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "password of InfluxDB server user", 2024-03-30T00:37:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "port": { 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "port", 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "8086", 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "InfluxDB server port", 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "ssl": { 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ssl", 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "false", 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "threads": { 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threads", 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "32", 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "username": { 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "username", 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "username of InfluxDB server user", 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "verify_ssl": { 2024-03-30T00:37:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "verify_ssl", 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "true", 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "insights", 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "iostat", 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "k8sevents", 2024-03-30T00:37:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_event_retention_days": { 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "ceph_event_retention_days", 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "7", 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-30T00:37:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "config_check_secs": { 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "config_check_secs", 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "10", 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "localpool", 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "failure_domain": { 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "failure_domain", 2024-03-30T00:37:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "host", 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "failure domain for any created local pool", 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": { 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "min_size", 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default min_size for any created local pool", 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "num_rep": { 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "num_rep", 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3", 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default replica count for any created local pool", 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": { 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_num", 2024-03-30T00:37:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "128", 2024-03-30T00:37:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "default pg_num for any created local pool", 2024-03-30T00:37:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "prefix": { 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prefix", 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "name prefix for any created local pool", 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "subtree": { 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "subtree", 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "rack", 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-30T00:37:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mds_autoscaler", 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirroring", 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "nfs", 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "fail_fs": { 2024-03-30T00:37:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "fail_fs", 2024-03-30T00:37:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-30T00:37:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.441 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.442 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator": { 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "orchestrator", 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "cephadm", 2024-03-30T00:37:37.445 INFO:teuthology.orchestra.run.smithi099.stdout: "rook", 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: "test_orchestrator" 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Orchestrator backend", 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_perf_query", 2024-03-30T00:37:37.446 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.447 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.449 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.450 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.451 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "osd_support", 2024-03-30T00:37:37.452 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.453 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.454 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.454 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.454 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.454 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.454 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.454 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.454 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.454 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.454 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.455 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.456 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.456 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.456 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.456 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.456 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.456 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.456 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.456 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.457 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.458 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.458 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.458 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.458 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.458 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.458 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.458 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.458 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.459 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "pg_autoscaler", 2024-03-30T00:37:37.459 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.459 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.459 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.459 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.459 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.460 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.461 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.464 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: "threshold": { 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "threshold", 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "3.0", 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1.0", 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.466 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "scaling threshold", 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "progress", 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.467 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_pg_recovery_event": { 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_pg_recovery_event", 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.468 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.469 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.470 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.471 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.472 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.473 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.474 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.474 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.474 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.474 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.474 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.474 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.474 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.474 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.474 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: "max_completed_events": { 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_completed_events", 2024-03-30T00:37:37.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.476 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.476 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.476 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "50", 2024-03-30T00:37:37.476 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.476 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.476 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.476 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "number of past completed events to remember", 2024-03-30T00:37:37.476 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.476 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "sleep_interval": { 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "sleep_interval", 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "5", 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "how long the module is going to sleep", 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.477 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "prometheus", 2024-03-30T00:37:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: "cache": { 2024-03-30T00:37:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cache", 2024-03-30T00:37:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.479 INFO:teuthology.orchestra.run.smithi099.stdout: "exclude_perf_counters": { 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "exclude_perf_counters", 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.480 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.481 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.483 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.484 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools": { 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools", 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.486 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.487 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "300", 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "scrape_interval": { 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "scrape_interval", 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:37.488 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15.0", 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-30T00:37:37.489 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-30T00:37:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "::", 2024-03-30T00:37:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-30T00:37:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.490 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "9283", 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-30T00:37:37.491 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "stale_cache_strategy": { 2024-03-30T00:37:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stale_cache_strategy", 2024-03-30T00:37:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "log", 2024-03-30T00:37:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.492 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_behaviour": { 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_behaviour", 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.493 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "default", 2024-03-30T00:37:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "default", 2024-03-30T00:37:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "error" 2024-03-30T00:37:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "standby_error_status_code": { 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "standby_error_status_code", 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "500", 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "400", 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "599", 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.495 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.496 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.497 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.498 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.499 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.500 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.501 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_snap_create": { 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_snap_create", 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10", 2024-03-30T00:37:37.502 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "mirror_snapshot_schedule": { 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "mirror_snapshot_schedule", 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "trash_purge_schedule": { 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "trash_purge_schedule", 2024-03-30T00:37:37.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "restful", 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "enable_auth": { 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enable_auth", 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.506 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "key_file": { 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "key_file", 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.507 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.509 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.510 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.512 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.513 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "server_addr": { 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_addr", 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.514 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "server_port": { 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "server_port", 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rgw", 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.517 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.518 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.521 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rook", 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.523 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.524 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.525 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.526 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.528 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "storage_class": { 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "storage_class", 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "local", 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-30T00:37:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "selftest", 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.530 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.531 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.534 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.535 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "roption1": { 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption1", 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.536 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "roption2": { 2024-03-30T00:37:37.537 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "roption2", 2024-03-30T00:37:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "xyz", 2024-03-30T00:37:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.538 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption1": { 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption1", 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.539 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption2": { 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption2", 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.540 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption3": { 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption3", 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "float", 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.541 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption4": { 2024-03-30T00:37:37.542 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption4", 2024-03-30T00:37:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.543 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption5": { 2024-03-30T00:37:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption5", 2024-03-30T00:37:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.545 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.546 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption6": { 2024-03-30T00:37:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption6", 2024-03-30T00:37:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.547 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "rwoption7": { 2024-03-30T00:37:37.548 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rwoption7", 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "1", 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "42", 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "testkey": { 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testkey", 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.550 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "testlkey": { 2024-03-30T00:37:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testlkey", 2024-03-30T00:37:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.551 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "testnewline": { 2024-03-30T00:37:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "testnewline", 2024-03-30T00:37:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.552 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snap_schedule", 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.553 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_m_granularity": { 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "allow_m_granularity", 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "allow minute scheduled snapshots", 2024-03-30T00:37:37.554 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "dump_on_update": { 2024-03-30T00:37:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "dump_on_update", 2024-03-30T00:37:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.555 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "dump database to debug log on update", 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.556 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.558 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.559 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.561 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "stats", 2024-03-30T00:37:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.562 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.563 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.564 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.566 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.567 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "status", 2024-03-30T00:37:37.568 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.569 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.570 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.573 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telegraf", 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "address": { 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "address", 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.575 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "15", 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.576 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.577 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.578 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.579 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.580 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.581 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "telemetry", 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_basic": { 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_basic", 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.583 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_crash": { 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_crash", 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.584 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_device": { 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_device", 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.585 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_ident": { 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_ident", 2024-03-30T00:37:37.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.587 INFO:teuthology.orchestra.run.smithi099.stdout: "channel_perf": { 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "channel_perf", 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.588 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.589 INFO:teuthology.orchestra.run.smithi099.stdout: "contact": { 2024-03-30T00:37:37.589 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "contact", 2024-03-30T00:37:37.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.589 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.589 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.589 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.589 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.589 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.589 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.589 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: "description": { 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "description", 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.590 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "device_url": { 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "device_url", 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-30T00:37:37.591 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.592 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.592 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.592 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.592 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.592 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.592 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.592 INFO:teuthology.orchestra.run.smithi099.stdout: "enabled": { 2024-03-30T00:37:37.592 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "enabled", 2024-03-30T00:37:37.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.592 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:37.593 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "24", 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "8", 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "last_opt_revision": { 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "last_opt_revision", 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "1", 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.595 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard": { 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard", 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.596 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "leaderboard_description": { 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "leaderboard_description", 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.597 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.598 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.599 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.599 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.599 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.599 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.599 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.599 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.599 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.599 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.599 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.599 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.599 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.601 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.601 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.601 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.601 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.601 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.601 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.601 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.602 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.602 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.602 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.602 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.602 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.602 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.602 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.602 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.602 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.602 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.602 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.603 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: "organization": { 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "organization", 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.604 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: "proxy": { 2024-03-30T00:37:37.605 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "proxy", 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.606 INFO:teuthology.orchestra.run.smithi099.stdout: "url": { 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "url", 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.607 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "test_orchestrator", 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.608 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.609 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.610 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.611 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.612 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.613 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.614 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.615 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.615 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.615 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.615 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.615 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.615 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.615 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.615 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.615 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.617 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.618 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.618 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.618 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.618 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.618 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.618 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.618 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.618 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.618 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.618 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.618 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.619 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "max_concurrent_clones": { 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "max_concurrent_clones", 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.620 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "4", 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_delay": { 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_delay", 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.621 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "0", 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: "snapshot_clone_no_wait": { 2024-03-30T00:37:37.622 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "snapshot_clone_no_wait", 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "True", 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix", 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: "can_run": true, 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: "error_string": "", 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: "module_options": { 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: "discovery_interval": { 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "discovery_interval", 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "uint", 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.624 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "100", 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "identifier": { 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "identifier", 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.625 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: "interval": { 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "interval", 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "secs", 2024-03-30T00:37:37.626 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "60", 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "log_level": { 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_level", 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.627 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.628 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster": { 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster", 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.629 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_cluster_level": { 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_cluster_level", 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "info", 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.630 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [ 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: "", 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: "critical", 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: "debug", 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: "error", 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: "info", 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: "warning" 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.631 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.632 INFO:teuthology.orchestra.run.smithi099.stdout: "log_to_file": { 2024-03-30T00:37:37.632 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "log_to_file", 2024-03-30T00:37:37.632 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "bool", 2024-03-30T00:37:37.632 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.632 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-03-30T00:37:37.632 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "False", 2024-03-30T00:37:37.632 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.632 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.632 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.632 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_host": { 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_host", 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "", 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.633 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.634 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.634 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.634 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.634 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.634 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_port": { 2024-03-30T00:37:37.634 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_port", 2024-03-30T00:37:37.634 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "int", 2024-03-30T00:37:37.634 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.634 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.634 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "10051", 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "zabbix_sender": { 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "zabbix_sender", 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "str", 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "level": "advanced", 2024-03-30T00:37:37.635 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 0, 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: "min": "", 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: "max": "", 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: "enum_allowed": [], 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: "desc": "", 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: "long_desc": "", 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tags": [], 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: "see_also": [] 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.636 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "services": { 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "dashboard": "https://172.21.15.99:8443/" 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "always_on_modules": { 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "octopus": [ 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-03-30T00:37:37.637 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "pacific": [ 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-03-30T00:37:37.638 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: "quincy": [ 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-03-30T00:37:37.639 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "reef": [ 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "balancer", 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "crash", 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "devicehealth", 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "orchestrator", 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscaler", 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "progress", 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "rbd_support", 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "status", 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "telemetry", 2024-03-30T00:37:37.640 INFO:teuthology.orchestra.run.smithi099.stdout: "volumes" 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: "last_failure_osd_epoch": 3, 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: "active_clients": [ 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "devicehealth", 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1235964974 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.641 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "libcephsqlite", 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3421332352 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "rbd_support", 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 429965107 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "volumes", 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-03-30T00:37:37.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-30T00:37:37.644 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-03-30T00:37:37.644 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:0", 2024-03-30T00:37:37.644 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1630436412 2024-03-30T00:37:37.644 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.644 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-30T00:37:37.644 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:37.644 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-30T00:37:37.644 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:37:37.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:36 smithi161 ceph-mon[42221]: pgmap v390: 97 pgs: 97 active+clean; 41 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.3 MiB/s wr, 2.02k op/s 2024-03-30T00:37:37.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:36 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2047279986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:37:37.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:36 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3473375595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:37:37.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:36 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4127265247' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-30T00:37:37.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:36 smithi142 ceph-mon[41779]: pgmap v390: 97 pgs: 97 active+clean; 41 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.3 MiB/s wr, 2.02k op/s 2024-03-30T00:37:37.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:36 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2047279986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:37:37.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:36 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3473375595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:37:37.646 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:36 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4127265247' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-30T00:37:37.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 -- 172.21.15.99:0/4127265247 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74f40a1120 msgr2=0x7f74f40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:37.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74f40a1120 0x7f74f40a35e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f74f8005950 tx=0x7f74f800b410 comp rx=0 tx=0).stop 2024-03-30T00:37:37.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 -- 172.21.15.99:0/4127265247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750c100c80 msgr2=0x7f750c198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:37:37.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750c100c80 0x7f750c198f00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f750000d980 tx=0x7f750000dc90 comp rx=0 tx=0).stop 2024-03-30T00:37:37.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 -- 172.21.15.99:0/4127265247 shutdown_connections 2024-03-30T00:37:37.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f74f40a1120 0x7f74f40a35e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f750c102810 0x7f750c1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f750c100c80 0x7f750c198f00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 --2- 172.21.15.99:0/4127265247 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f750c101e70 0x7f750c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:37:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 -- 172.21.15.99:0/4127265247 >> 172.21.15.99:0/4127265247 conn(0x7f750c0fc1f0 msgr2=0x7f750c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:37:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 -- 172.21.15.99:0/4127265247 shutdown_connections 2024-03-30T00:37:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:37:36.187+0000 7f7514400700 1 -- 172.21.15.99:0/4127265247 wait complete. 2024-03-30T00:37:37.689 DEBUG:tasks.check_counter:Getting stats from a 2024-03-30T00:37:37.689 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.a perf dump 2024-03-30T00:37:37.690 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:36 smithi099 ceph-mon[38079]: pgmap v390: 97 pgs: 97 active+clean; 41 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.3 MiB/s wr, 2.02k op/s 2024-03-30T00:37:37.690 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:36 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2047279986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:37:37.690 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:36 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3473375595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:37:37.690 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:36 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4127265247' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-30T00:37:38.061 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:37:38.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:38 smithi099 ceph-mon[38079]: pgmap v391: 97 pgs: 97 active+clean; 987 KiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 MiB/s wr, 1.45k op/s 2024-03-30T00:37:38.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:38 smithi142 ceph-mon[41779]: pgmap v391: 97 pgs: 97 active+clean; 987 KiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 MiB/s wr, 1.45k op/s 2024-03-30T00:37:38.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:38 smithi161 ceph-mon[42221]: pgmap v391: 97 pgs: 97 active+clean; 987 KiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.5 MiB/s wr, 1.45k op/s 2024-03-30T00:37:40.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:40 smithi099 ceph-mon[38079]: pgmap v392: 97 pgs: 97 active+clean; 987 KiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 MiB/s wr, 1.12k op/s 2024-03-30T00:37:40.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:40 smithi142 ceph-mon[41779]: pgmap v392: 97 pgs: 97 active+clean; 987 KiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 MiB/s wr, 1.12k op/s 2024-03-30T00:37:40.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:40 smithi161 ceph-mon[42221]: pgmap v392: 97 pgs: 97 active+clean; 987 KiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 MiB/s wr, 1.12k op/s 2024-03-30T00:37:41.068 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:37:41.068 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:37:41.068 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 351, 2024-03-30T00:37:41.068 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 358, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 114955, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 312028, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 20, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 6, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.057040311, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.017317824, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.028936726, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.005678082, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 356, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.058014736, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000162962 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:41.069 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 486, 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000104851, 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000215 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 245, 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 247, 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 516856, 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 138291, 2024-03-30T00:37:41.070 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 12, 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.048104937, 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.021356158, 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.027628383, 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.003120803, 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 247, 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.040657684, 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000164606 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 153, 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000035545, 2024-03-30T00:37:41.071 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000232 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 481152, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 51856 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 59, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 116, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 30507, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 217068, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 8, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 6, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.015820946, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.006312931, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.007156133, 2024-03-30T00:37:41.072 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.001442220, 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 114, 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.023935855, 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000209963 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 34, 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000005250, 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000154 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 176704 2024-03-30T00:37:41.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 167, 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.030627776, 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000183399 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-03-30T00:37:41.074 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 73, 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.003914491, 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000053623 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 12, 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 11, 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 11, 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.008518212, 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000774382 2024-03-30T00:37:41.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 1, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 11, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 16, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 0, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 16, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-03-30T00:37:41.076 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 13, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 4, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 3, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 3, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 3, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 25, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 18, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 1, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 3, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 53, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-03-30T00:37:41.077 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 1, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 1, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 4, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 4, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 8, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 195298298, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 4, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-03-30T00:37:41.078 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 4, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 4, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 6, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 53, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 50, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 1, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 8 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.079 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 1, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 8, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 5, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 7, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 1, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 4, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 4, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-03-30T00:37:41.080 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 1, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 73, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-03-30T00:37:41.081 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 73, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 1, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 1, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4284849, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-03-30T00:37:41.082 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 58, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.266534454, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.004595421 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 20, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 26, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 6, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 19, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 22, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 3, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 16, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 28, 2024-03-30T00:37:41.083 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 12, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 3, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 8, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 5, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 49664, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 198964 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 17, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 1, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 12, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 4, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 5, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.084 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3, 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001953462, 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000651154 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.085 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3, 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000802090, 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000267363 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-03-30T00:37:41.086 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.087 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.088 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1, 2024-03-30T00:37:41.089 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000426730, 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000426730 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.090 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-03-30T00:37:41.091 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-03-30T00:37:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.093 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4, 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.005335930, 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001333982 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-03-30T00:37:41.094 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 1, 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 1, 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 0, 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 1, 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 8, 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 8, 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 180, 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-03-30T00:37:41.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:37:41.096 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-03-30T00:37:41.097 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 301271, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 113, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-03-30T00:37:41.098 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 8016, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 169, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 80853, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 341, 2024-03-30T00:37:41.099 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 182, 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 102855, 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 27, 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 182, 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 182, 2024-03-30T00:37:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.409965307, 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.007747062 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 182, 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 209 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 182, 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 5, 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 177, 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2024-03-30T00:37:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 1, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 59, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 12, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 79, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 4, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2024-03-30T00:37:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-03-30T00:37:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 87, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 4, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 12, 2024-03-30T00:37:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 12, 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 21, 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 0, 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 3 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 14 2024-03-30T00:37:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 67, 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 4, 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 4, 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 651, 2024-03-30T00:37:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 594667, 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 651, 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 651, 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 594667, 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-30T00:37:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 182, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 103259, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 182, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 103259, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 182, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 182, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 182, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 182, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 4, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 404, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 4, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 1, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 404, 2024-03-30T00:37:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5585291480a0": { 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 73, 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 90545, 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 73, 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-30T00:37:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-30T00:37:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 58, 2024-03-30T00:37:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 90545, 2024-03-30T00:37:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-30T00:37:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:41.112 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:37:41.454 INFO:tasks.check_counter:Daemon mds.a mds.exported=1 2024-03-30T00:37:41.454 INFO:tasks.check_counter:Daemon mds.a mds.imported=1 2024-03-30T00:37:41.454 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=1 2024-03-30T00:37:41.454 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=8 2024-03-30T00:37:41.454 DEBUG:tasks.check_counter:Getting stats from d 2024-03-30T00:37:41.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.d perf dump 2024-03-30T00:37:41.818 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:37:42.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:42 smithi099 ceph-mon[38079]: pgmap v393: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 MiB/s wr, 497 op/s 2024-03-30T00:37:42.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:42 smithi142 ceph-mon[41779]: pgmap v393: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 MiB/s wr, 497 op/s 2024-03-30T00:37:42.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:42 smithi161 ceph-mon[42221]: pgmap v393: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 MiB/s wr, 497 op/s 2024-03-30T00:37:42.966 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:37:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.003332329, 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000713600, 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1, 2024-03-30T00:37:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 62, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 160, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 86880, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 3, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 2, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.007063342, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.004874896, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.001341995, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 60, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.003540609, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000059010 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 86880 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 89, 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 84, 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 436576, 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 46352, 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-03-30T00:37:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.018782939, 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.005876214, 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.015022257, 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.001971797, 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 84, 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.007260456, 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000086434 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 436576, 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 46352 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:37:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:37:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:37:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 149894, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 29, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:37:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 86, 2024-03-30T00:37:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 421644, 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 86, 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 86, 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 421644, 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:42.977 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:37:43.355 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-30T00:37:43.356 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-30T00:37:43.356 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-30T00:37:43.356 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-30T00:37:43.356 DEBUG:tasks.check_counter:Getting stats from g 2024-03-30T00:37:43.356 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.g perf dump 2024-03-30T00:37:43.715 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:37:44.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:44 smithi099 ceph-mon[38079]: pgmap v394: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 723 KiB/s wr, 162 op/s 2024-03-30T00:37:44.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:44 smithi142 ceph-mon[41779]: pgmap v394: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 723 KiB/s wr, 162 op/s 2024-03-30T00:37:44.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:44 smithi161 ceph-mon[42221]: pgmap v394: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 723 KiB/s wr, 162 op/s 2024-03-30T00:37:44.882 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:37:44.882 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:37:44.882 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 86, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 81, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 435760, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 44576, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.020773117, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.006553631, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.013210924, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.001592586, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 81, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.006918584, 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000085414 2024-03-30T00:37:44.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 435760, 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 44576 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1, 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 60, 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 160, 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 86528, 2024-03-30T00:37:44.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 3, 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 2, 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.007081440, 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.004399075, 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.001926398, 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 58, 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.002962902, 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000051084 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:44.885 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 86528 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 0, 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2, 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 0, 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 0, 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.000862514, 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000000000, 2024-03-30T00:37:44.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.000707925, 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T00:37:44.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:37:44.888 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:37:44.889 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-03-30T00:37:44.890 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 148863, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 28, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-03-30T00:37:44.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-30T00:37:44.892 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 83, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 421254, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 83, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 83, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 421254, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:44.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:44.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:44.894 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:37:45.267 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-30T00:37:45.267 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-30T00:37:45.267 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-03-30T00:37:45.267 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-03-30T00:37:45.267 DEBUG:tasks.check_counter:Getting stats from j 2024-03-30T00:37:45.268 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.j perf dump 2024-03-30T00:37:45.627 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T00:37:46.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:46 smithi099 ceph-mon[38079]: pgmap v395: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 160 KiB/s wr, 28 op/s 2024-03-30T00:37:46.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:46 smithi142 ceph-mon[41779]: pgmap v395: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 160 KiB/s wr, 28 op/s 2024-03-30T00:37:46.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:46 smithi161 ceph-mon[42221]: pgmap v395: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 160 KiB/s wr, 28 op/s 2024-03-30T00:37:46.763 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 87, 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 84, 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 434160, 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 47088, 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.022287241, 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.006743986, 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.016089563, 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.001761099, 2024-03-30T00:37:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 83, 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.005492435, 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000066173 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 434160, 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 47088 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:37:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 63, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 160, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 87056, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 3, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 2, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.007209324, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.004601718, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.001783069, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 61, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.004546152, 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000074527 2024-03-30T00:37:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 87056 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 4, 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 4, 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 2960, 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 448, 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-03-30T00:37:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.001393244, 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000080995, 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.001283445, 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3, 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001363299, 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000454433 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 448 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:37:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:37:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:37:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 148880, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 29, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:37:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 88, 2024-03-30T00:37:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 421904, 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 88, 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 88, 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 421904, 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-30T00:37:46.774 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-30T00:37:47.139 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-30T00:37:47.140 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-30T00:37:47.140 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-30T00:37:47.140 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-30T00:37:47.140 DEBUG:tasks.check_counter:Getting stats from b 2024-03-30T00:37:47.140 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.b perf dump 2024-03-30T00:37:47.525 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:37:48.316 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:48 smithi142 ceph-mon[41779]: pgmap v396: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-03-30T00:37:48.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:48 smithi161 ceph-mon[42221]: pgmap v396: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-03-30T00:37:48.733 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-30T00:37:48.733 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:37:48.733 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 667, 2024-03-30T00:37:48.733 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 660, 2024-03-30T00:37:48.733 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 209521, 2024-03-30T00:37:48.733 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 586547, 2024-03-30T00:37:48.733 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 10, 2024-03-30T00:37:48.733 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.097023572, 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.026731078, 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.038139502, 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.017794470, 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 660, 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.386590940, 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000585743 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 471, 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000108180, 2024-03-30T00:37:48.734 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000229 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 616, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 585, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 643229, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 231086, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 7, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.084654954, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.025878885, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.044726367, 2024-03-30T00:37:48.735 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.015063923, 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 587, 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.552977949, 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000942040 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 397, 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000077872, 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000196 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 488928, 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 56528 2024-03-30T00:37:48.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 622, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 454, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 209871, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 922763, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 78, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 10, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.116071772, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.026615059, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.061839645, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.021853361, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 448, 2024-03-30T00:37:48.737 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.199321384, 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000444913 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 603, 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000120531, 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000199 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 54624, 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 184784 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-03-30T00:37:48.738 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 392, 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.109335446, 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000278916 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 23, 2024-03-30T00:37:48.739 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000763308, 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000033187 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 52, 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 50, 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 50, 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.092947424, 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001858948 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 2, 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-03-30T00:37:48.740 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 58, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 23, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 4, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 19, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 16, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 4, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 2, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 2, 2024-03-30T00:37:48.741 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 2, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 82, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 71, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 323, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 1, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 4, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 1, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-03-30T00:37:48.742 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 66, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 234, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 66, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 42, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 5, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 37, 2024-03-30T00:37:48.743 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 24, 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 21, 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 24, 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 51, 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 26, 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 2, 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 28 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-03-30T00:37:48.744 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 6, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 16, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 2, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 2, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-03-30T00:37:48.745 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-03-30T00:37:48.746 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-03-30T00:37:48.746 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-03-30T00:37:48.746 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-03-30T00:37:48.746 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-03-30T00:37:48.746 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 5, 2024-03-30T00:37:48.746 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 1, 2024-03-30T00:37:48.746 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-03-30T00:37:48.746 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-03-30T00:37:48.746 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-03-30T00:37:48.746 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-03-30T00:37:48.746 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 101, 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 90, 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 90, 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 11, 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 2, 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 13, 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 12, 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 12, 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 2, 2024-03-30T00:37:48.747 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-03-30T00:37:48.748 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 1, 2024-03-30T00:37:48.748 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4496101, 2024-03-30T00:37:48.748 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 4526043, 2024-03-30T00:37:48.748 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-03-30T00:37:48.748 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-03-30T00:37:48.748 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 82, 2024-03-30T00:37:48.748 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.216790596, 2024-03-30T00:37:48.748 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002643787 2024-03-30T00:37:48.748 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.748 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-03-30T00:37:48.748 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 23, 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 34, 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 11, 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 18, 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 20, 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 2, 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 23, 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 30, 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 7, 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 2, 2024-03-30T00:37:48.749 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 19, 2024-03-30T00:37:48.750 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 17, 2024-03-30T00:37:48.750 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 51852, 2024-03-30T00:37:48.750 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207156 2024-03-30T00:37:48.750 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.750 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-03-30T00:37:48.750 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 82, 2024-03-30T00:37:48.750 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 3, 2024-03-30T00:37:48.750 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 52, 2024-03-30T00:37:48.750 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 12, 2024-03-30T00:37:48.750 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 10, 2024-03-30T00:37:48.750 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 8, 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.008826862, 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001103357 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 20, 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.014720069, 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000736003 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.751 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.009738836, 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.009738836 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.752 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.753 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.754 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 8, 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.016016652, 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002002081 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.755 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.756 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-03-30T00:37:48.756 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.756 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.756 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.756 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.756 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-03-30T00:37:48.756 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.756 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.756 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.756 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.757 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-03-30T00:37:48.757 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-03-30T00:37:48.757 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.001442024, 2024-03-30T00:37:48.757 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001442024 2024-03-30T00:37:48.757 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.757 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-03-30T00:37:48.757 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2, 2024-03-30T00:37:48.757 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.003719226, 2024-03-30T00:37:48.757 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001859613 2024-03-30T00:37:48.757 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.758 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-03-30T00:37:48.758 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.758 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.758 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.758 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.758 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-03-30T00:37:48.758 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.758 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.758 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.758 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2, 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.013027757, 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006513878 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-03-30T00:37:48.759 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.001857203, 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001857203 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.760 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.761 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-03-30T00:37:48.762 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 7, 2024-03-30T00:37:48.762 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.023598795, 2024-03-30T00:37:48.762 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003371256 2024-03-30T00:37:48.762 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.762 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-03-30T00:37:48.762 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.762 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.762 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.762 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.762 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-03-30T00:37:48.762 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-03-30T00:37:48.763 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 15, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 15, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 192, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:37:48.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:37:48.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-03-30T00:37:48.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 291092, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 117, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 176, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 2, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:37:48.767 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 8016, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 169, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 93498, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 653, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-03-30T00:37:48.768 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 838, 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 400103, 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 31, 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 838, 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 838, 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 18.308646626, 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.021848027 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-03-30T00:37:48.769 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 838, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 998 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 838, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 78, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 760, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 38, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 28, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 7, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 84, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 47, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-03-30T00:37:48.770 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 514, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 71, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 56, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-03-30T00:37:48.771 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-03-30T00:37:48.772 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 87, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 5, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-03-30T00:37:48.773 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-03-30T00:37:48.774 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-03-30T00:37:48.774 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 143, 2024-03-30T00:37:48.774 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 0, 2024-03-30T00:37:48.774 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 10 2024-03-30T00:37:48.774 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.774 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-03-30T00:37:48.774 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-03-30T00:37:48.774 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-03-30T00:37:48.774 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-03-30T00:37:48.774 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-03-30T00:37:48.774 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 67 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 1, 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 1, 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 2, 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:37:48.775 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 1901, 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 875595, 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 1901, 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 1901, 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 875595, 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:48.776 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:48.777 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 838, 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 400305, 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 838, 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 400305, 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-03-30T00:37:48.778 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 838, 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 838, 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 838, 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 838, 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:48.779 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 2, 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 202, 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:48.780 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 2, 2024-03-30T00:37:48.781 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-03-30T00:37:48.781 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-03-30T00:37:48.781 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 2, 2024-03-30T00:37:48.781 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 202, 2024-03-30T00:37:48.781 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:48.781 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.781 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.781 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.781 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:48.781 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:48.781 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x559b37d140a0": { 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 101, 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 331739, 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 101, 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 82, 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 331739, 2024-03-30T00:37:48.782 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:48.783 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:48.783 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:48.783 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:48.783 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:48.783 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:48.783 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-30T00:37:48.784 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:48 smithi099 ceph-mon[38079]: pgmap v396: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-03-30T00:37:49.129 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-03-30T00:37:49.129 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-03-30T00:37:49.130 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=6 2024-03-30T00:37:49.130 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-03-30T00:37:49.130 DEBUG:tasks.check_counter:Getting stats from e 2024-03-30T00:37:49.130 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.e perf dump 2024-03-30T00:37:49.491 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:37:50.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:50 smithi142 ceph-mon[41779]: pgmap v397: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-30T00:37:50.674 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-30T00:37:50.675 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:37:50.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 373, 2024-03-30T00:37:50.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 363, 2024-03-30T00:37:50.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 564672, 2024-03-30T00:37:50.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 132313, 2024-03-30T00:37:50.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-03-30T00:37:50.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 1, 2024-03-30T00:37:50.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.061032697, 2024-03-30T00:37:50.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.023791831, 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.023477714, 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.010832430, 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 362, 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.027864457, 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000076973 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 473616, 2024-03-30T00:37:50.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 48032 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 7, 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 67, 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 3676, 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 151644, 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.006775985, 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.003210815, 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.002624225, 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000065454, 2024-03-30T00:37:50.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 66, 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.006932653, 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000105040 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 150960 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.678 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 523, 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 525, 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 224184, 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 162660, 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.046304509, 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.011053546, 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.014984279, 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.016777555, 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:50.679 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 523, 2024-03-30T00:37:50.680 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.021215005, 2024-03-30T00:37:50.680 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000040564 2024-03-30T00:37:50.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.680 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:50.680 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.680 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.680 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.680 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T00:37:50.681 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T00:37:50.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.681 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-03-30T00:37:50.681 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-03-30T00:37:50.681 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-03-30T00:37:50.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.682 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-03-30T00:37:50.682 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-03-30T00:37:50.682 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-03-30T00:37:50.682 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1370, 2024-03-30T00:37:50.682 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.037205352, 2024-03-30T00:37:50.682 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000027157 2024-03-30T00:37:50.682 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:50.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.682 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-03-30T00:37:50.682 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.683 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.684 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.684 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.684 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-03-30T00:37:50.684 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-03-30T00:37:50.684 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-03-30T00:37:50.684 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-03-30T00:37:50.684 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-03-30T00:37:50.684 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-03-30T00:37:50.684 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-03-30T00:37:50.684 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 5, 2024-03-30T00:37:50.684 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 0, 2024-03-30T00:37:50.685 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 5, 2024-03-30T00:37:50.685 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-03-30T00:37:50.685 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 5, 2024-03-30T00:37:50.685 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-03-30T00:37:50.685 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-03-30T00:37:50.685 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-03-30T00:37:50.685 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 1, 2024-03-30T00:37:50.685 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-03-30T00:37:50.685 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-03-30T00:37:50.686 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-03-30T00:37:50.686 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-03-30T00:37:50.686 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-03-30T00:37:50.686 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-03-30T00:37:50.686 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-03-30T00:37:50.686 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-03-30T00:37:50.686 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-03-30T00:37:50.686 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-03-30T00:37:50.686 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-03-30T00:37:50.686 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-03-30T00:37:50.687 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-03-30T00:37:50.688 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-03-30T00:37:50.689 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-03-30T00:37:50.690 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 73, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 1, 2024-03-30T00:37:50.691 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 4284849, 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 4283198, 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 73 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 4, 2024-03-30T00:37:50.692 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 16, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 12, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 2, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 11, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 9, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 5, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 17, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 12, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 47360, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207156 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.693 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-03-30T00:37:50.694 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-03-30T00:37:50.694 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-03-30T00:37:50.694 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-03-30T00:37:50.694 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-03-30T00:37:50.694 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-03-30T00:37:50.694 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-03-30T00:37:50.694 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.694 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.694 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.694 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.695 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.696 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.697 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-03-30T00:37:50.698 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.698 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.698 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.698 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.698 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-03-30T00:37:50.698 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.698 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.698 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.698 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.698 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-03-30T00:37:50.698 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.699 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.700 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.700 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-03-30T00:37:50.700 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.700 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.700 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.700 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.700 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-03-30T00:37:50.700 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.700 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.700 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.700 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.701 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-03-30T00:37:50.702 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.703 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.704 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.704 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-03-30T00:37:50.704 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.704 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.704 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.704 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.704 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-03-30T00:37:50.704 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.704 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.706 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-03-30T00:37:50.706 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.706 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.706 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.706 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-03-30T00:37:50.706 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-03-30T00:37:50.706 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-03-30T00:37:50.706 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.706 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.706 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.707 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:50.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.707 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-03-30T00:37:50.707 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-03-30T00:37:50.707 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 1, 2024-03-30T00:37:50.707 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2024-03-30T00:37:50.707 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-03-30T00:37:50.707 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-03-30T00:37:50.707 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-03-30T00:37:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-03-30T00:37:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 1630, 2024-03-30T00:37:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-03-30T00:37:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-03-30T00:37:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:37:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-03-30T00:37:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:37:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:37:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:37:50.708 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-03-30T00:37:50.709 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-03-30T00:37:50.710 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 182190, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 45, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 8016, 2024-03-30T00:37:50.711 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 169, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 15312, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 48, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-03-30T00:37:50.712 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 796, 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 796, 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 796, 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.512990118, 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000644459 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 796, 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 797 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 796, 2024-03-30T00:37:50.713 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 796, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 466, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 328, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-03-30T00:37:50.714 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-03-30T00:37:50.715 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-03-30T00:37:50.716 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-03-30T00:37:50.716 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-03-30T00:37:50.716 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-03-30T00:37:50.716 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-03-30T00:37:50.716 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-03-30T00:37:50.716 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-03-30T00:37:50.716 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-03-30T00:37:50.716 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-03-30T00:37:50.716 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-03-30T00:37:50.716 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-03-30T00:37:50.716 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 87, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 4, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 4, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 4, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-03-30T00:37:50.717 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-03-30T00:37:50.718 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 899, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 703530, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 899, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-03-30T00:37:50.719 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 899, 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 703530, 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:50.720 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 796, 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 90550, 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 796, 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 90550, 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-03-30T00:37:50.721 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 796, 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 796, 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 796, 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 796, 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:50.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:50.723 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x564e8262c5a0": { 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-03-30T00:37:50.724 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-03-30T00:37:50.725 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-03-30T00:37:50.725 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:50.725 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:50.725 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:50.725 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:50.725 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:50.725 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:50.725 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-30T00:37:50.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:50 smithi161 ceph-mon[42221]: pgmap v397: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-30T00:37:50.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:50 smithi099 ceph-mon[38079]: pgmap v397: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-30T00:37:51.066 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-30T00:37:51.066 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-30T00:37:51.066 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-30T00:37:51.066 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-03-30T00:37:51.066 DEBUG:tasks.check_counter:Getting stats from h 2024-03-30T00:37:51.067 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.h perf dump 2024-03-30T00:37:51.424 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:37:52.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:52 smithi142 ceph-mon[41779]: pgmap v398: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-30T00:37:52.673 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-30T00:37:52.673 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:37:52.673 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 85, 2024-03-30T00:37:52.673 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 82, 2024-03-30T00:37:52.673 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 433616, 2024-03-30T00:37:52.673 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 45904, 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 0, 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.021677168, 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.005364111, 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.013984859, 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.002281066, 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 81, 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.006839321, 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000084436 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:52.674 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 433616, 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 45904 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 5, 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 64, 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 3120, 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 57168, 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 3, 2024-03-30T00:37:52.675 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.006373454, 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.003364915, 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.001967495, 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 63, 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.005727717, 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000090916 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:52.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 57168 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 0, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 0, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 0, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 2, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 1, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.001126111, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.000000000, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.000982509, 2024-03-30T00:37:52.677 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:52.678 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 0, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:37:52.679 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:37:52.680 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:37:52.680 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:37:52.680 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:37:52.680 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:37:52.680 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-03-30T00:37:52.680 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:37:52.680 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-03-30T00:37:52.680 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:37:52.680 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:37:52.680 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:37:52.680 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:37:52.681 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 148823, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 27, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 0, 2024-03-30T00:37:52.682 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 0, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 928, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 2, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-03-30T00:37:52.683 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 86, 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 421644, 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 86, 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 86, 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 421644, 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:52.684 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-30T00:37:52.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:52 smithi161 ceph-mon[42221]: pgmap v398: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-30T00:37:52.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:52 smithi099 ceph-mon[38079]: pgmap v398: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-30T00:37:53.045 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.h 2024-03-30T00:37:53.045 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.h 2024-03-30T00:37:53.045 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.h 2024-03-30T00:37:53.045 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.h 2024-03-30T00:37:53.045 DEBUG:tasks.check_counter:Getting stats from k 2024-03-30T00:37:53.046 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.k perf dump 2024-03-30T00:37:53.402 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T00:37:54.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:54 smithi142 ceph-mon[41779]: pgmap v399: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-30T00:37:54.574 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-03-30T00:37:54.574 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:37:54.574 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 103, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 96, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 465730, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 50125, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 1, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.026242969, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.007797846, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.016143620, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.002197373, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 95, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.008292424, 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000087288 2024-03-30T00:37:54.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 464896, 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 49808 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 202, 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 267, 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 49628, 2024-03-30T00:37:54.576 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 215124, 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.027888876, 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.010000063, 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.007177210, 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.007237628, 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 265, 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.015481023, 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000058418 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.577 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 152608 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 556, 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 556, 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 168931, 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 172178, 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 3, 2024-03-30T00:37:54.578 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.067897751, 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.026220327, 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.017494507, 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.017559232, 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 555, 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.032535379, 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000058622 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.579 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1269, 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.038164330, 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000030074 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:54.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.581 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 0, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 0, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 0, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-03-30T00:37:54.582 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 1, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-03-30T00:37:54.583 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-03-30T00:37:54.584 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-03-30T00:37:54.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-03-30T00:37:54.586 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-03-30T00:37:54.587 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-03-30T00:37:54.587 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-03-30T00:37:54.587 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-03-30T00:37:54.587 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-03-30T00:37:54.587 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T00:37:54.587 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-03-30T00:37:54.587 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-03-30T00:37:54.587 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-03-30T00:37:54.587 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-03-30T00:37:54.587 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-03-30T00:37:54.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 2, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 1, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-03-30T00:37:54.588 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 4195340, 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 4194844, 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 2 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 3, 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 3, 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 0, 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 1, 2024-03-30T00:37:54.589 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 1, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 0, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 0, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 0, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 0, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 47232, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207156 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-03-30T00:37:54.590 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-03-30T00:37:54.591 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-03-30T00:37:54.591 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-03-30T00:37:54.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.591 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.591 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-03-30T00:37:54.591 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.591 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.592 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-03-30T00:37:54.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.592 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.592 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-03-30T00:37:54.592 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.592 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.593 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.594 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.595 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.596 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.597 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.599 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.600 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:54.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 1, 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 1, 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 1, 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 1638, 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:37:54.602 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:37:54.603 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:37:54.604 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 189239, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 48, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:37:54.605 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 8016, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 169, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 8320, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 13, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-03-30T00:37:54.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 754, 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 754, 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 754, 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.387992524, 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000514578 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-03-30T00:37:54.607 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 754, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 755 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 754, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 754, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 473, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 279, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-03-30T00:37:54.608 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-03-30T00:37:54.609 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-03-30T00:37:54.610 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 87, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 4, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 5, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 5, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.611 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-03-30T00:37:54.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 857, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 599026, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 857, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 857, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 599026, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.613 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 754, 2024-03-30T00:37:54.614 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 1041, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 754, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1041, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:54.615 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 754, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 754, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 754, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 754, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:54.616 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55ba643d8640": { 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-03-30T00:37:54.617 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-03-30T00:37:54.618 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-03-30T00:37:54.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:54 smithi161 ceph-mon[42221]: pgmap v399: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-30T00:37:54.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:54 smithi099 ceph-mon[38079]: pgmap v399: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-30T00:37:54.995 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-30T00:37:54.996 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-30T00:37:54.996 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-03-30T00:37:54.996 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-03-30T00:37:54.996 DEBUG:tasks.check_counter:Getting stats from c 2024-03-30T00:37:54.996 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.c perf dump 2024-03-30T00:37:55.387 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:37:56.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:56 smithi142 ceph-mon[41779]: pgmap v400: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:37:56.585 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-03-30T00:37:56.585 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:37:56.586 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 0, 2024-03-30T00:37:56.586 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 2, 2024-03-30T00:37:56.586 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 0, 2024-03-30T00:37:56.586 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 0, 2024-03-30T00:37:56.586 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-03-30T00:37:56.586 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2024-03-30T00:37:56.586 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.003461264, 2024-03-30T00:37:56.586 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000000000, 2024-03-30T00:37:56.586 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000736207, 2024-03-30T00:37:56.586 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T00:37:56.587 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 91, 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 86, 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 437120, 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 47536, 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.019149592, 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.005044624, 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.014536340, 2024-03-30T00:37:56.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.002436910, 2024-03-30T00:37:56.589 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:56.589 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 86, 2024-03-30T00:37:56.589 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.007968577, 2024-03-30T00:37:56.589 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000092657 2024-03-30T00:37:56.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:56.589 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:56.589 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:56.589 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:56.589 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:56.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:56.589 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 437120, 2024-03-30T00:37:56.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 47536 2024-03-30T00:37:56.590 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:56.590 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:37:56.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1, 2024-03-30T00:37:56.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 63, 2024-03-30T00:37:56.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 160, 2024-03-30T00:37:56.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 56896, 2024-03-30T00:37:56.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 3, 2024-03-30T00:37:56.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-03-30T00:37:56.591 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.005722790, 2024-03-30T00:37:56.591 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.003370756, 2024-03-30T00:37:56.591 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001262592, 2024-03-30T00:37:56.591 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:37:56.591 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:56.591 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 61, 2024-03-30T00:37:56.591 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.004775181, 2024-03-30T00:37:56.591 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000078281 2024-03-30T00:37:56.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:56.591 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:56.591 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:56.592 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:56.592 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:56.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:56.592 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-30T00:37:56.592 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 56896 2024-03-30T00:37:56.592 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:56.592 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-03-30T00:37:56.592 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2024-03-30T00:37:56.592 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-03-30T00:37:56.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:56.593 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-03-30T00:37:56.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:37:56.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-03-30T00:37:56.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:37:56.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:37:56.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:37:56.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:37:56.593 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:37:56.594 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:37:56.594 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:37:56.594 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:37:56.594 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:37:56.594 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:37:56.594 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:37:56.594 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:37:56.594 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:37:56.594 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-03-30T00:37:56.595 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:37:56.595 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-03-30T00:37:56.595 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:37:56.595 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:37:56.595 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:37:56.595 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:37:56.595 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:37:56.595 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:37:56.595 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:37:56.595 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-03-30T00:37:56.595 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:37:56.596 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:37:56.596 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:37:56.596 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-03-30T00:37:56.596 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-03-30T00:37:56.596 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-03-30T00:37:56.596 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:37:56.596 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:37:56.596 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:37:56.596 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:37:56.596 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 149894, 2024-03-30T00:37:56.597 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 29, 2024-03-30T00:37:56.597 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-03-30T00:37:56.597 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-03-30T00:37:56.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-03-30T00:37:56.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-03-30T00:37:56.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:37:56.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-03-30T00:37:56.597 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2024-03-30T00:37:56.598 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-03-30T00:37:56.599 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-03-30T00:37:56.599 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-03-30T00:37:56.599 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-03-30T00:37:56.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:56.599 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:37:56.599 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:37:56.599 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-30T00:37:56.599 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:37:56.599 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 88, 2024-03-30T00:37:56.600 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 421904, 2024-03-30T00:37:56.600 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:56.600 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 88, 2024-03-30T00:37:56.600 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-30T00:37:56.600 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-30T00:37:56.600 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 88, 2024-03-30T00:37:56.600 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 421904, 2024-03-30T00:37:56.600 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:37:56.600 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:56.601 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:56.601 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:56.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:37:56.601 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:37:56.601 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-03-30T00:37:56.604 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:56 smithi161 ceph-mon[42221]: pgmap v400: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:37:56.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:56 smithi099 ceph-mon[38079]: pgmap v400: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:37:56.981 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.c 2024-03-30T00:37:56.981 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.c 2024-03-30T00:37:56.982 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.c 2024-03-30T00:37:56.982 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.c 2024-03-30T00:37:56.982 DEBUG:tasks.check_counter:Getting stats from f 2024-03-30T00:37:56.982 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.f perf dump 2024-03-30T00:37:57.344 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:37:58.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:37:58 smithi142 ceph-mon[41779]: pgmap v401: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:37:58.587 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-03-30T00:37:58.587 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:37:58.587 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 28, 2024-03-30T00:37:58.587 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 30, 2024-03-30T00:37:58.587 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 6678, 2024-03-30T00:37:58.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 47337, 2024-03-30T00:37:58.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 13, 2024-03-30T00:37:58.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-03-30T00:37:58.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.013675434, 2024-03-30T00:37:58.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.001602649, 2024-03-30T00:37:58.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.006779244, 2024-03-30T00:37:58.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000399864, 2024-03-30T00:37:58.588 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:58.588 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 28, 2024-03-30T00:37:58.588 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.038255785, 2024-03-30T00:37:58.588 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001366278 2024-03-30T00:37:58.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.589 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:58.589 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 16, 2024-03-30T00:37:58.589 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000002233, 2024-03-30T00:37:58.589 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000139 2024-03-30T00:37:58.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.589 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T00:37:58.589 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T00:37:58.589 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.590 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:37:58.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 151, 2024-03-30T00:37:58.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 144, 2024-03-30T00:37:58.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 480419, 2024-03-30T00:37:58.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 157102, 2024-03-30T00:37:58.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 16, 2024-03-30T00:37:58.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 8, 2024-03-30T00:37:58.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.033129507, 2024-03-30T00:37:58.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.009771286, 2024-03-30T00:37:58.590 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.023803756, 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.002623626, 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 141, 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.031831088, 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000225752 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 10, 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000001037, 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000103 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 475184, 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 53056 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.591 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:37:58.592 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 235, 2024-03-30T00:37:58.592 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 310, 2024-03-30T00:37:58.592 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 98795, 2024-03-30T00:37:58.592 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 261558, 2024-03-30T00:37:58.592 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2024-03-30T00:37:58.592 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-03-30T00:37:58.592 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.037221390, 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.015348347, 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.021320959, 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.002070475, 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 311, 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.029540414, 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000094985 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 452, 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000118232, 2024-03-30T00:37:58.593 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000261 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 152704 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 46, 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.002060606, 2024-03-30T00:37:58.594 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000044795 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000027002, 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000013501 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 1, 2024-03-30T00:37:58.595 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 1, 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 11, 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-03-30T00:37:58.596 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-03-30T00:37:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 16, 2024-03-30T00:37:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 0, 2024-03-30T00:37:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 16, 2024-03-30T00:37:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-03-30T00:37:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 13, 2024-03-30T00:37:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 4, 2024-03-30T00:37:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-03-30T00:37:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-03-30T00:37:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 3, 2024-03-30T00:37:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 10, 2024-03-30T00:37:58.597 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 5, 2024-03-30T00:37:58.598 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 1, 2024-03-30T00:37:58.598 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 4, 2024-03-30T00:37:58.598 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-03-30T00:37:58.598 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-03-30T00:37:58.598 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-03-30T00:37:58.598 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-03-30T00:37:58.598 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-03-30T00:37:58.598 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-03-30T00:37:58.598 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-03-30T00:37:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-03-30T00:37:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-03-30T00:37:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-03-30T00:37:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-03-30T00:37:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-03-30T00:37:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 8, 2024-03-30T00:37:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 195298298, 2024-03-30T00:37:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-03-30T00:37:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-03-30T00:37:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-03-30T00:37:58.599 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T00:37:58.600 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-03-30T00:37:58.600 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-03-30T00:37:58.600 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T00:37:58.600 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-03-30T00:37:58.600 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-03-30T00:37:58.600 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-03-30T00:37:58.600 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-03-30T00:37:58.600 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-03-30T00:37:58.600 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-03-30T00:37:58.600 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T00:37:58.600 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T00:37:58.601 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-03-30T00:37:58.601 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-03-30T00:37:58.601 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-03-30T00:37:58.601 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-03-30T00:37:58.601 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.601 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-03-30T00:37:58.601 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-03-30T00:37:58.601 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 12, 2024-03-30T00:37:58.601 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 8, 2024-03-30T00:37:58.601 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 10, 2024-03-30T00:37:58.601 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-03-30T00:37:58.602 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-03-30T00:37:58.602 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-03-30T00:37:58.602 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-03-30T00:37:58.602 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-03-30T00:37:58.602 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-03-30T00:37:58.602 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-03-30T00:37:58.602 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-03-30T00:37:58.602 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-03-30T00:37:58.603 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-03-30T00:37:58.603 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-03-30T00:37:58.603 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-03-30T00:37:58.603 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-03-30T00:37:58.603 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T00:37:58.603 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-03-30T00:37:58.603 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-03-30T00:37:58.603 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-03-30T00:37:58.603 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-03-30T00:37:58.603 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 2, 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 2, 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 1, 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-03-30T00:37:58.604 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 1, 2024-03-30T00:37:58.605 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-03-30T00:37:58.605 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-03-30T00:37:58.605 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-03-30T00:37:58.605 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4195340, 2024-03-30T00:37:58.605 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-03-30T00:37:58.605 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-03-30T00:37:58.605 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-03-30T00:37:58.606 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.003543422, 2024-03-30T00:37:58.606 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001771711 2024-03-30T00:37:58.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.606 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-03-30T00:37:58.606 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.606 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-03-30T00:37:58.606 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 20, 2024-03-30T00:37:58.606 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 22, 2024-03-30T00:37:58.606 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 2, 2024-03-30T00:37:58.607 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 19, 2024-03-30T00:37:58.607 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 21, 2024-03-30T00:37:58.607 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 2, 2024-03-30T00:37:58.607 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 16, 2024-03-30T00:37:58.607 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 20, 2024-03-30T00:37:58.607 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 4, 2024-03-30T00:37:58.607 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-03-30T00:37:58.607 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-03-30T00:37:58.607 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-03-30T00:37:58.607 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 48512, 2024-03-30T00:37:58.607 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 198964 2024-03-30T00:37:58.608 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.608 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-03-30T00:37:58.608 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 1, 2024-03-30T00:37:58.608 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 1, 2024-03-30T00:37:58.608 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 1, 2024-03-30T00:37:58.608 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 5, 2024-03-30T00:37:58.608 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 2, 2024-03-30T00:37:58.608 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-03-30T00:37:58.608 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.609 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.610 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-03-30T00:37:58.611 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.611 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.611 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.611 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-03-30T00:37:58.611 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.611 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.611 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.611 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.612 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-03-30T00:37:58.612 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.612 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.612 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.612 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.612 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-03-30T00:37:58.612 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.612 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.612 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-03-30T00:37:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-03-30T00:37:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.613 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.614 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-03-30T00:37:58.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.614 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.614 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.614 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-03-30T00:37:58.614 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.614 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-03-30T00:37:58.615 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.616 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-03-30T00:37:58.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.620 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-03-30T00:37:58.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.620 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.620 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-03-30T00:37:58.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.620 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:37:58.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.622 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-03-30T00:37:58.622 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-03-30T00:37:58.622 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 1, 2024-03-30T00:37:58.622 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 0, 2024-03-30T00:37:58.622 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-03-30T00:37:58.622 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-03-30T00:37:58.622 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-03-30T00:37:58.622 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-03-30T00:37:58.622 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 197, 2024-03-30T00:37:58.622 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-03-30T00:37:58.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:37:58.623 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:37:58.624 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:37:58.625 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:37:58.626 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:37:58.626 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 234356, 2024-03-30T00:37:58.626 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 77, 2024-03-30T00:37:58.626 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-03-30T00:37:58.626 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-03-30T00:37:58.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-03-30T00:37:58.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-03-30T00:37:58.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:37:58.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-03-30T00:37:58.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:37:58.626 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8016, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 169, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 79629, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 320, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-03-30T00:37:58.627 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-03-30T00:37:58.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.628 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-03-30T00:37:58.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-03-30T00:37:58.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-03-30T00:37:58.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 37, 2024-03-30T00:37:58.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 11248, 2024-03-30T00:37:58.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 27, 2024-03-30T00:37:58.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 37, 2024-03-30T00:37:58.628 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-03-30T00:37:58.628 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 37, 2024-03-30T00:37:58.628 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.301993534, 2024-03-30T00:37:58.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.008161987 2024-03-30T00:37:58.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-03-30T00:37:58.629 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-03-30T00:37:58.629 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 37, 2024-03-30T00:37:58.629 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 59 2024-03-30T00:37:58.629 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 37, 2024-03-30T00:37:58.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 0, 2024-03-30T00:37:58.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 37, 2024-03-30T00:37:58.629 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-03-30T00:37:58.630 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-03-30T00:37:58.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-03-30T00:37:58.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-03-30T00:37:58.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 0, 2024-03-30T00:37:58.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 2, 2024-03-30T00:37:58.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 6, 2024-03-30T00:37:58.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-03-30T00:37:58.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-03-30T00:37:58.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-03-30T00:37:58.630 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-03-30T00:37:58.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 6, 2024-03-30T00:37:58.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-03-30T00:37:58.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-03-30T00:37:58.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-03-30T00:37:58.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 0, 2024-03-30T00:37:58.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-03-30T00:37:58.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-03-30T00:37:58.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-03-30T00:37:58.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-03-30T00:37:58.631 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-03-30T00:37:58.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-03-30T00:37:58.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-03-30T00:37:58.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T00:37:58.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-03-30T00:37:58.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-03-30T00:37:58.632 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-03-30T00:37:58.632 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-03-30T00:37:58.632 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-03-30T00:37:58.632 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-03-30T00:37:58.632 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-03-30T00:37:58.632 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-03-30T00:37:58.633 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-03-30T00:37:58.633 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-03-30T00:37:58.633 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-03-30T00:37:58.633 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-03-30T00:37:58.633 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-03-30T00:37:58.633 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-03-30T00:37:58.633 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-03-30T00:37:58.633 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-03-30T00:37:58.633 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-03-30T00:37:58.633 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-03-30T00:37:58.634 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-03-30T00:37:58.634 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 87, 2024-03-30T00:37:58.634 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-03-30T00:37:58.634 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 4, 2024-03-30T00:37:58.634 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 8, 2024-03-30T00:37:58.634 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 8, 2024-03-30T00:37:58.634 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-03-30T00:37:58.634 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-03-30T00:37:58.634 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 14, 2024-03-30T00:37:58.634 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 0, 2024-03-30T00:37:58.635 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 1 2024-03-30T00:37:58.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.635 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-03-30T00:37:58.635 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-03-30T00:37:58.635 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-03-30T00:37:58.635 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-03-30T00:37:58.635 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-03-30T00:37:58.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.635 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-03-30T00:37:58.635 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 2 2024-03-30T00:37:58.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.636 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-03-30T00:37:58.636 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-03-30T00:37:58.636 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-03-30T00:37:58.636 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-03-30T00:37:58.636 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-03-30T00:37:58.636 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-03-30T00:37:58.636 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-03-30T00:37:58.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.636 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:37:58.636 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 410, 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 539731, 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 410, 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 410, 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 539731, 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:37:58.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:58.638 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 37, 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 11248, 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 37, 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 11248, 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:37:58.639 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 37, 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 37, 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 37, 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 37, 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.640 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-30T00:37:58.641 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x5653e65e80a0": { 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:37:58.642 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 2, 2024-03-30T00:37:58.643 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1036, 2024-03-30T00:37:58.643 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:37:58.643 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 2, 2024-03-30T00:37:58.643 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-30T00:37:58.643 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-30T00:37:58.643 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2, 2024-03-30T00:37:58.643 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1036, 2024-03-30T00:37:58.643 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:37:58.643 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:37:58.643 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:37:58.643 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:37:58.644 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:37:58.644 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:37:58.644 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-03-30T00:37:58.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:37:58 smithi161 ceph-mon[42221]: pgmap v401: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:37:58.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:37:58 smithi099 ceph-mon[38079]: pgmap v401: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:37:59.008 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-30T00:37:59.008 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-30T00:37:59.008 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-30T00:37:59.008 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=12 2024-03-30T00:37:59.008 DEBUG:tasks.check_counter:Getting stats from i 2024-03-30T00:37:59.009 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.i perf dump 2024-03-30T00:37:59.372 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:38:00.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:00 smithi142 ceph-mon[41779]: pgmap v402: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:38:00.615 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 617, 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 619, 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 513928, 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 191897, 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.074181412, 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.024394263, 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.019654296, 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.020304805, 2024-03-30T00:38:00.616 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 617, 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.033099249, 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000053645 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:38:00.617 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 111, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 99, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 478392, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 49950, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.023613238, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.007676690, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.016900553, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.002187083, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 99, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.014110244, 2024-03-30T00:38:00.618 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000142527 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 476736, 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 48640 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 269, 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 331, 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 66159, 2024-03-30T00:38:00.619 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 236997, 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.034711124, 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.014299469, 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.007822530, 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.008943060, 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 329, 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.017207917, 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000052303 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.620 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 153648 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1460, 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.047844316, 2024-03-30T00:38:00.621 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000032770 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-03-30T00:38:00.622 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 16, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 4, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 12, 2024-03-30T00:38:00.623 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 9, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 2, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-03-30T00:38:00.624 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-03-30T00:38:00.625 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-03-30T00:38:00.626 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-03-30T00:38:00.627 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.628 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 101, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4496101, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 4526043, 2024-03-30T00:38:00.629 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 4524392, 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 101 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 14, 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 20, 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 6, 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 9, 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 11, 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 2, 2024-03-30T00:38:00.630 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 16, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 21, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 5, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 47488, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207156 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-03-30T00:38:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.632 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.633 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.634 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.635 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.636 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.637 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.638 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.639 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.640 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.641 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-03-30T00:38:00.642 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 3032, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:38:00.643 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:38:00.644 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:38:00.645 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 189239, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 48, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8016, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 169, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-03-30T00:38:00.646 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 53210, 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 252, 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 889, 2024-03-30T00:38:00.647 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 26, 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 4, 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 889, 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 889, 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.464990005, 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000523048 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 889, 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 890 2024-03-30T00:38:00.648 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 889, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 889, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 548, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 338, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-03-30T00:38:00.649 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 2, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T00:38:00.650 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-03-30T00:38:00.651 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 87, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 4, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 5, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 5, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-03-30T00:38:00.652 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-03-30T00:38:00.653 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 993, 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 960971, 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 993, 2024-03-30T00:38:00.654 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 993, 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 960971, 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-30T00:38:00.655 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 889, 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 331749, 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 889, 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 331749, 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:38:00.656 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 889, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 889, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 889, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 889, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.657 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-03-30T00:38:00.658 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55b7c7122640": { 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-30T00:38:00.659 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-03-30T00:38:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-03-30T00:38:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:38:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:38:00.660 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:38:00.660 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-03-30T00:38:00.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:00 smithi161 ceph-mon[42221]: pgmap v402: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:38:00.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:00 smithi099 ceph-mon[38079]: pgmap v402: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:38:00.998 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-30T00:38:00.998 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-30T00:38:00.998 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-03-30T00:38:00.998 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-03-30T00:38:00.998 DEBUG:tasks.check_counter:Getting stats from l 2024-03-30T00:38:00.998 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 -- ceph daemon mds.l perf dump 2024-03-30T00:38:01.359 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T00:38:02.534 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-03-30T00:38:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T00:38:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 92, 2024-03-30T00:38:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 89, 2024-03-30T00:38:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 435520, 2024-03-30T00:38:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 50048, 2024-03-30T00:38:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2024-03-30T00:38:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2024-03-30T00:38:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.023952083, 2024-03-30T00:38:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.007117901, 2024-03-30T00:38:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.015913897, 2024-03-30T00:38:02.535 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.002096431, 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 88, 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.007174072, 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000081523 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 435520, 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 50048 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:02.536 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 68, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 160, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 57776, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 3, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 2, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.006603910, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.003943571, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001420920, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 66, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.005414743, 2024-03-30T00:38:02.537 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000082041 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 57776 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4, 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 4, 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 2960, 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 448, 2024-03-30T00:38:02.538 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.001341235, 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000061331, 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001166037, 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001425368, 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000475122 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:02.539 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-03-30T00:38:02.540 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T00:38:02.541 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-03-30T00:38:02.542 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148823, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 27, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-03-30T00:38:02.543 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 93, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 422554, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 93, 2024-03-30T00:38:02.544 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-03-30T00:38:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-03-30T00:38:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 93, 2024-03-30T00:38:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 422554, 2024-03-30T00:38:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-03-30T00:38:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-03-30T00:38:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-03-30T00:38:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-03-30T00:38:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:38:02.545 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-03-30T00:38:02.545 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-03-30T00:38:02.554 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:02 smithi161 ceph-mon[42221]: pgmap v403: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:38:02.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:02 smithi142 ceph-mon[41779]: pgmap v403: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:38:02.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:02 smithi099 ceph-mon[38079]: pgmap v403: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:38:02.939 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-30T00:38:02.939 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-30T00:38:02.939 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-03-30T00:38:02.940 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-03-30T00:38:02.940 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-30T00:38:02.951 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-30T00:38:02.951 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:38:02.951 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:38:03.001 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:38:03.001 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:38:03.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:38:03.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.280+0000 7f34f1e00700 1 -- 172.21.15.99:0/978766048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec073810 msgr2=0x7f34ec073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.280+0000 7f34f1e00700 1 --2- 172.21.15.99:0/978766048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec073810 0x7f34ec073c90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009990 tx=0x7f34d8009ca0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.280+0000 7f34f1e00700 1 -- 172.21.15.99:0/978766048 shutdown_connections 2024-03-30T00:38:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.280+0000 7f34f1e00700 1 --2- 172.21.15.99:0/978766048 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34ec108b40 0x7f34ec10afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.280+0000 7f34f1e00700 1 --2- 172.21.15.99:0/978766048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ec074df0 0x7f34ec073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.280+0000 7f34f1e00700 1 --2- 172.21.15.99:0/978766048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec073810 0x7f34ec073c90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.280+0000 7f34f1e00700 1 -- 172.21.15.99:0/978766048 >> 172.21.15.99:0/978766048 conn(0x7f34ec0fc1f0 msgr2=0x7f34ec0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.280+0000 7f34f1e00700 1 -- 172.21.15.99:0/978766048 shutdown_connections 2024-03-30T00:38:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.280+0000 7f34f1e00700 1 -- 172.21.15.99:0/978766048 wait complete. 2024-03-30T00:38:03.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.281+0000 7f34f1e00700 1 Processor -- start 2024-03-30T00:38:03.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.281+0000 7f34f1e00700 1 -- start start 2024-03-30T00:38:03.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.281+0000 7f34f1e00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34ec073810 0x7f34ec198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:03.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.281+0000 7f34f1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec074df0 0x7f34ec199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.281+0000 7f34f1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ec108b40 0x7f34ec1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.281+0000 7f34f1e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec1a0870 con 0x7f34ec074df0 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.281+0000 7f34f1e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec1a09e0 con 0x7f34ec108b40 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.281+0000 7f34f1e00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec1a0ba0 con 0x7f34ec073810 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34ea000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec074df0 0x7f34ec199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34eaa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34ec073810 0x7f34ec198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34eaa00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34ec073810 0x7f34ec198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.99:48592/0 (socket says 172.21.15.99:48592) 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34eaa00700 1 -- 172.21.15.99:0/3165659011 learned_addr learned my addr 172.21.15.99:0/3165659011 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34eb400700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ec108b40 0x7f34ec1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34eaa00700 1 -- 172.21.15.99:0/3165659011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ec108b40 msgr2=0x7f34ec1a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34eaa00700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ec108b40 0x7f34ec1a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34eaa00700 1 -- 172.21.15.99:0/3165659011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec074df0 msgr2=0x7f34ec199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:03.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34eaa00700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec074df0 0x7f34ec199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34eaa00700 1 -- 172.21.15.99:0/3165659011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d80095f0 con 0x7f34ec073810 2024-03-30T00:38:03.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34ea000700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec074df0 0x7f34ec199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:38:03.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34eb400700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ec108b40 0x7f34ec1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:38:03.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.282+0000 7f34eaa00700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34ec073810 0x7f34ec198dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00d950 tx=0x7f34dc00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:03.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.283+0000 7f34f0a00700 1 -- 172.21.15.99:0/3165659011 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc009940 con 0x7f34ec073810 2024-03-30T00:38:03.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.283+0000 7f34f0a00700 1 -- 172.21.15.99:0/3165659011 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34dc010460 con 0x7f34ec073810 2024-03-30T00:38:03.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.283+0000 7f34f1e00700 1 -- 172.21.15.99:0/3165659011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec1a0e80 con 0x7f34ec073810 2024-03-30T00:38:03.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.283+0000 7f34f0a00700 1 -- 172.21.15.99:0/3165659011 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00f620 con 0x7f34ec073810 2024-03-30T00:38:03.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.283+0000 7f34f1e00700 1 -- 172.21.15.99:0/3165659011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec1a12a0 con 0x7f34ec073810 2024-03-30T00:38:03.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.284+0000 7f34f1e00700 1 -- 172.21.15.99:0/3165659011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec107900 con 0x7f34ec073810 2024-03-30T00:38:03.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.285+0000 7f34f0a00700 1 -- 172.21.15.99:0/3165659011 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f34dc0105d0 con 0x7f34ec073810 2024-03-30T00:38:03.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.287+0000 7f34f0a00700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f34d00a1120 0x7f34d00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:03.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.287+0000 7f34f0a00700 1 -- 172.21.15.99:0/3165659011 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f34dc0cc7a0 con 0x7f34ec073810 2024-03-30T00:38:03.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.287+0000 7f34ea000700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f34d00a1120 0x7f34d00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:03.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.287+0000 7f34ea000700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f34d00a1120 0x7f34d00a35e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f34d8005910 tx=0x7f34d800b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:03.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.291+0000 7f34f0a00700 1 -- 172.21.15.99:0/3165659011 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f34dc095eb0 con 0x7f34ec073810 2024-03-30T00:38:03.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.530+0000 7f34f1e00700 1 -- 172.21.15.99:0/3165659011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34ec067a10 con 0x7f34ec073810 2024-03-30T00:38:03.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.532+0000 7f34f0a00700 1 -- 172.21.15.99:0/3165659011 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22859 (secure 0 0 0) 0x7f34dc0994d0 con 0x7f34ec073810 2024-03-30T00:38:03.539 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:38:03.539 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:35:10.474499+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8},{"from":5,"to":7}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:38:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 -- 172.21.15.99:0/3165659011 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f34d00a1120 msgr2=0x7f34d00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f34d00a1120 0x7f34d00a35e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f34d8005910 tx=0x7f34d800b610 comp rx=0 tx=0).stop 2024-03-30T00:38:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 -- 172.21.15.99:0/3165659011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34ec073810 msgr2=0x7f34ec198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34ec073810 0x7f34ec198dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00d950 tx=0x7f34dc00dc60 comp rx=0 tx=0).stop 2024-03-30T00:38:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 -- 172.21.15.99:0/3165659011 shutdown_connections 2024-03-30T00:38:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f34d00a1120 0x7f34d00a35e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34ec108b40 0x7f34ec1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34ec073810 0x7f34ec198dd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 --2- 172.21.15.99:0/3165659011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ec074df0 0x7f34ec199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 -- 172.21.15.99:0/3165659011 >> 172.21.15.99:0/3165659011 conn(0x7f34ec0fc1f0 msgr2=0x7f34ec104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 -- 172.21.15.99:0/3165659011 shutdown_connections 2024-03-30T00:38:03.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.535+0000 7f34f1e00700 1 -- 172.21.15.99:0/3165659011 wait complete. 2024-03-30T00:38:03.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:33:14.085184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:33:15.090699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:38:03.561 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:38:03.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.753+0000 7effe7a00700 1 -- 172.21.15.99:0/525106607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0104180 msgr2=0x7effe01045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:03.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.753+0000 7effe7a00700 1 --2- 172.21.15.99:0/525106607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0104180 0x7effe01045e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7effcc0099c0 tx=0x7effcc009cd0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.753+0000 7effe7a00700 1 -- 172.21.15.99:0/525106607 shutdown_connections 2024-03-30T00:38:03.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.753+0000 7effe7a00700 1 --2- 172.21.15.99:0/525106607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0104b20 0x7effe010b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.753+0000 7effe7a00700 1 --2- 172.21.15.99:0/525106607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe0102f90 0x7effe01033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.753+0000 7effe7a00700 1 --2- 172.21.15.99:0/525106607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0104180 0x7effe01045e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.753+0000 7effe7a00700 1 -- 172.21.15.99:0/525106607 >> 172.21.15.99:0/525106607 conn(0x7effe00fe500 msgr2=0x7effe0100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:03.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.753+0000 7effe7a00700 1 -- 172.21.15.99:0/525106607 shutdown_connections 2024-03-30T00:38:03.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.753+0000 7effe7a00700 1 -- 172.21.15.99:0/525106607 wait complete. 2024-03-30T00:38:03.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.754+0000 7effe7a00700 1 Processor -- start 2024-03-30T00:38:03.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.754+0000 7effe7a00700 1 -- start start 2024-03-30T00:38:03.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.754+0000 7effe7a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0102f90 0x7effe0198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:03.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.754+0000 7effe7a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0104180 0x7effe0199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:03.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.754+0000 7effe7a00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe0104b20 0x7effe01a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:03.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.754+0000 7effe7a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe01a0970 con 0x7effe0102f90 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.754+0000 7effe7a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe01a0ab0 con 0x7effe0104180 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.754+0000 7effe7a00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe01a0c40 con 0x7effe0104b20 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effe5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0102f90 0x7effe0198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effe5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0102f90 0x7effe0198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37738/0 (socket says 172.21.15.99:37738) 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effe5000700 1 -- 172.21.15.99:0/3559061484 learned_addr learned my addr 172.21.15.99:0/3559061484 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effdfe00700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0104180 0x7effe0199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effe5a00700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe0104b20 0x7effe01a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effe5000700 1 -- 172.21.15.99:0/3559061484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe0104b20 msgr2=0x7effe01a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effe5000700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe0104b20 0x7effe01a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effe5000700 1 -- 172.21.15.99:0/3559061484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0104180 msgr2=0x7effe0199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effe5000700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0104180 0x7effe0199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effe5000700 1 -- 172.21.15.99:0/3559061484 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc0095f0 con 0x7effe0102f90 2024-03-30T00:38:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effe5000700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0102f90 0x7effe0198e40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7effd400d980 tx=0x7effd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effdd600700 1 -- 172.21.15.99:0/3559061484 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4004020 con 0x7effe0102f90 2024-03-30T00:38:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effdd600700 1 -- 172.21.15.99:0/3559061484 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7effd4009730 con 0x7effe0102f90 2024-03-30T00:38:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.755+0000 7effe7a00700 1 -- 172.21.15.99:0/3559061484 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe01a0f20 con 0x7effe0102f90 2024-03-30T00:38:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.756+0000 7effdd600700 1 -- 172.21.15.99:0/3559061484 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4003a00 con 0x7effe0102f90 2024-03-30T00:38:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.756+0000 7effe7a00700 1 -- 172.21.15.99:0/3559061484 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe01a1340 con 0x7effe0102f90 2024-03-30T00:38:03.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.757+0000 7effe7a00700 1 -- 172.21.15.99:0/3559061484 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe010a450 con 0x7effe0102f90 2024-03-30T00:38:03.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.759+0000 7effdd600700 1 -- 172.21.15.99:0/3559061484 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7effd40098a0 con 0x7effe0102f90 2024-03-30T00:38:03.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.760+0000 7effdd600700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1120 0x7effc80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:03.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.760+0000 7effdd600700 1 -- 172.21.15.99:0/3559061484 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7effd40cc660 con 0x7effe0102f90 2024-03-30T00:38:03.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.760+0000 7effdfe00700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1120 0x7effc80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:03.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.761+0000 7effdfe00700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1120 0x7effc80a35e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7effcc00b580 tx=0x7effcc005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:03.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.764+0000 7effdd600700 1 -- 172.21.15.99:0/3559061484 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7effd4095d10 con 0x7effe0102f90 2024-03-30T00:38:03.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.971+0000 7effe7a00700 1 -- 172.21.15.99:0/3559061484 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7effe010e950 con 0x7effe0102f90 2024-03-30T00:38:03.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.971+0000 7effdd600700 1 -- 172.21.15.99:0/3559061484 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7effd4099330 con 0x7effe0102f90 2024-03-30T00:38:03.978 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:38:03.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.973+0000 7effe7a00700 1 -- 172.21.15.99:0/3559061484 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1120 msgr2=0x7effc80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:03.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.973+0000 7effe7a00700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1120 0x7effc80a35e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7effcc00b580 tx=0x7effcc005e80 comp rx=0 tx=0).stop 2024-03-30T00:38:03.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.973+0000 7effe7a00700 1 -- 172.21.15.99:0/3559061484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0102f90 msgr2=0x7effe0198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:03.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.973+0000 7effe7a00700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0102f90 0x7effe0198e40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7effd400d980 tx=0x7effd400dc90 comp rx=0 tx=0).stop 2024-03-30T00:38:03.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.973+0000 7effe7a00700 1 -- 172.21.15.99:0/3559061484 shutdown_connections 2024-03-30T00:38:03.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.973+0000 7effe7a00700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effe0104b20 0x7effe01a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.973+0000 7effe7a00700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effe0102f90 0x7effe0198e40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.973+0000 7effe7a00700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7effc80a1120 0x7effc80a35e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.973+0000 7effe7a00700 1 --2- 172.21.15.99:0/3559061484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0104180 0x7effe0199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:03.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.973+0000 7effe7a00700 1 -- 172.21.15.99:0/3559061484 >> 172.21.15.99:0/3559061484 conn(0x7effe00fe500 msgr2=0x7effe0107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:03.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.974+0000 7effe7a00700 1 -- 172.21.15.99:0/3559061484 shutdown_connections 2024-03-30T00:38:03.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:03.974+0000 7effe7a00700 1 -- 172.21.15.99:0/3559061484 wait complete. 2024-03-30T00:38:03.998 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:38:04.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.188+0000 7fa74c600700 1 -- 172.21.15.99:0/2513057052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa744102910 msgr2=0x7fa7441091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:04.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.188+0000 7fa74c600700 1 --2- 172.21.15.99:0/2513057052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa744102910 0x7fa7441091c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00b0d0 tx=0x7fa73c00b3e0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.188+0000 7fa74c600700 1 -- 172.21.15.99:0/2513057052 shutdown_connections 2024-03-30T00:38:04.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.188+0000 7fa74c600700 1 --2- 172.21.15.99:0/2513057052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa744102910 0x7fa7441091c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.188+0000 7fa74c600700 1 --2- 172.21.15.99:0/2513057052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa744100d80 0x7fa7441011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.188+0000 7fa74c600700 1 --2- 172.21.15.99:0/2513057052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa744101f70 0x7fa7441023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.188+0000 7fa74c600700 1 -- 172.21.15.99:0/2513057052 >> 172.21.15.99:0/2513057052 conn(0x7fa7440fc2f0 msgr2=0x7fa7440fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:04.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.188+0000 7fa74c600700 1 -- 172.21.15.99:0/2513057052 shutdown_connections 2024-03-30T00:38:04.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.188+0000 7fa74c600700 1 -- 172.21.15.99:0/2513057052 wait complete. 2024-03-30T00:38:04.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.189+0000 7fa74c600700 1 Processor -- start 2024-03-30T00:38:04.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.189+0000 7fa74c600700 1 -- start start 2024-03-30T00:38:04.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.189+0000 7fa74c600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa744100d80 0x7fa744074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:04.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.189+0000 7fa74c600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa744101f70 0x7fa7440731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:04.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.189+0000 7fa74c600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa744102910 0x7fa7440736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:04.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.189+0000 7fa74c600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa744073e80 con 0x7fa744100d80 2024-03-30T00:38:04.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.189+0000 7fa74c600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa744073ff0 con 0x7fa744102910 2024-03-30T00:38:04.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.189+0000 7fa74c600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7441a0d90 con 0x7fa744101f70 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.189+0000 7fa749c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa744100d80 0x7fa744074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa749c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa744100d80 0x7fa744074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37754/0 (socket says 172.21.15.99:37754) 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa749c00700 1 -- 172.21.15.99:0/438396775 learned_addr learned my addr 172.21.15.99:0/438396775 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa74a600700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa744102910 0x7fa7440736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa749200700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa744101f70 0x7fa7440731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa749c00700 1 -- 172.21.15.99:0/438396775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa744101f70 msgr2=0x7fa7440731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa749c00700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa744101f70 0x7fa7440731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa749c00700 1 -- 172.21.15.99:0/438396775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa744102910 msgr2=0x7fa7440736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa749c00700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa744102910 0x7fa7440736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa749c00700 1 -- 172.21.15.99:0/438396775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c009d00 con 0x7fa744100d80 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa749c00700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa744100d80 0x7fa744074b10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa73800d980 tx=0x7fa73800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa742000700 1 -- 172.21.15.99:0/438396775 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa738004020 con 0x7fa744100d80 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa742000700 1 -- 172.21.15.99:0/438396775 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa73800f690 con 0x7fa744100d80 2024-03-30T00:38:04.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa74c600700 1 -- 172.21.15.99:0/438396775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7441a0fc0 con 0x7fa744100d80 2024-03-30T00:38:04.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.190+0000 7fa742000700 1 -- 172.21.15.99:0/438396775 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7380039e0 con 0x7fa744100d80 2024-03-30T00:38:04.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.191+0000 7fa74c600700 1 -- 172.21.15.99:0/438396775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7441a1490 con 0x7fa744100d80 2024-03-30T00:38:04.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.192+0000 7fa74c600700 1 -- 172.21.15.99:0/438396775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa744108180 con 0x7fa744100d80 2024-03-30T00:38:04.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.193+0000 7fa742000700 1 -- 172.21.15.99:0/438396775 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa73800f800 con 0x7fa744100d80 2024-03-30T00:38:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.195+0000 7fa742000700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa72c0a11a0 0x7fa72c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.195+0000 7fa742000700 1 -- 172.21.15.99:0/438396775 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fa7380cc4a0 con 0x7fa744100d80 2024-03-30T00:38:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.195+0000 7fa749200700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa72c0a11a0 0x7fa72c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:04.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.195+0000 7fa749200700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa72c0a11a0 0x7fa72c0a3660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa730005950 tx=0x7fa73000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:04.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.198+0000 7fa742000700 1 -- 172.21.15.99:0/438396775 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa738095b50 con 0x7fa744100d80 2024-03-30T00:38:04.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.408+0000 7fa74c600700 1 -- 172.21.15.99:0/438396775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa74402cda0 con 0x7fa744100d80 2024-03-30T00:38:04.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.408+0000 7fa742000700 1 -- 172.21.15.99:0/438396775 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa738099170 con 0x7fa744100d80 2024-03-30T00:38:04.415 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-30T00:38:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.410+0000 7fa74c600700 1 -- 172.21.15.99:0/438396775 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa72c0a11a0 msgr2=0x7fa72c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.410+0000 7fa74c600700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa72c0a11a0 0x7fa72c0a3660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa730005950 tx=0x7fa73000b410 comp rx=0 tx=0).stop 2024-03-30T00:38:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.410+0000 7fa74c600700 1 -- 172.21.15.99:0/438396775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa744100d80 msgr2=0x7fa744074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.410+0000 7fa74c600700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa744100d80 0x7fa744074b10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa73800d980 tx=0x7fa73800dc90 comp rx=0 tx=0).stop 2024-03-30T00:38:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.410+0000 7fa74c600700 1 -- 172.21.15.99:0/438396775 shutdown_connections 2024-03-30T00:38:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.410+0000 7fa74c600700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fa72c0a11a0 0x7fa72c0a3660 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.410+0000 7fa74c600700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa744102910 0x7fa7440736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.410+0000 7fa74c600700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa744100d80 0x7fa744074b10 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.410+0000 7fa74c600700 1 --2- 172.21.15.99:0/438396775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa744101f70 0x7fa7440731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.410+0000 7fa74c600700 1 -- 172.21.15.99:0/438396775 >> 172.21.15.99:0/438396775 conn(0x7fa7440fc2f0 msgr2=0x7fa7441052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.411+0000 7fa74c600700 1 -- 172.21.15.99:0/438396775 shutdown_connections 2024-03-30T00:38:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.411+0000 7fa74c600700 1 -- 172.21.15.99:0/438396775 wait complete. 2024-03-30T00:38:04.435 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:38:04.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:04 smithi142 ceph-mon[41779]: pgmap v404: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-30T00:38:04.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:04 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3165659011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:38:04.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:04 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3559061484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:38:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.623+0000 7f9024600700 1 -- 172.21.15.99:0/788639498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f901c100d80 msgr2=0x7f901c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.623+0000 7f9024600700 1 --2- 172.21.15.99:0/788639498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f901c100d80 0x7f901c1011a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f90100099c0 tx=0x7f9010009cd0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.623+0000 7f9024600700 1 -- 172.21.15.99:0/788639498 shutdown_connections 2024-03-30T00:38:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.623+0000 7f9024600700 1 --2- 172.21.15.99:0/788639498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c102910 0x7f901c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.623+0000 7f9024600700 1 --2- 172.21.15.99:0/788639498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f901c100d80 0x7f901c1011a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.623+0000 7f9024600700 1 --2- 172.21.15.99:0/788639498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f901c101f70 0x7f901c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.623+0000 7f9024600700 1 -- 172.21.15.99:0/788639498 >> 172.21.15.99:0/788639498 conn(0x7f901c0fc2f0 msgr2=0x7f901c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.623+0000 7f9024600700 1 -- 172.21.15.99:0/788639498 shutdown_connections 2024-03-30T00:38:04.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.623+0000 7f9024600700 1 -- 172.21.15.99:0/788639498 wait complete. 2024-03-30T00:38:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.624+0000 7f9024600700 1 Processor -- start 2024-03-30T00:38:04.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.624+0000 7f9024600700 1 -- start start 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9024600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f901c100d80 0x7f901c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9024600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f901c101f70 0x7f901c1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9024600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c102910 0x7f901c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9024600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1a09b0 con 0x7f901c100d80 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9024600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1a0af0 con 0x7f901c102910 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9024600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c1a0c80 con 0x7f901c101f70 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9021c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f901c100d80 0x7f901c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9021c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f901c100d80 0x7f901c198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37774/0 (socket says 172.21.15.99:37774) 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9021c00700 1 -- 172.21.15.99:0/3062238755 learned_addr learned my addr 172.21.15.99:0/3062238755 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9022600700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c102910 0x7f901c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9021200700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f901c101f70 0x7f901c1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9021c00700 1 -- 172.21.15.99:0/3062238755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f901c101f70 msgr2=0x7f901c1993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9021c00700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f901c101f70 0x7f901c1993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9021c00700 1 -- 172.21.15.99:0/3062238755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c102910 msgr2=0x7f901c1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9021c00700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c102910 0x7f901c1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9021c00700 1 -- 172.21.15.99:0/3062238755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90100095f0 con 0x7f901c100d80 2024-03-30T00:38:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.625+0000 7f9022600700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c102910 0x7f901c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:38:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.626+0000 7f9021c00700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f901c100d80 0x7f901c198e80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f901000bee0 tx=0x7f90100045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.626+0000 7f901a000700 1 -- 172.21.15.99:0/3062238755 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901001d070 con 0x7f901c100d80 2024-03-30T00:38:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.626+0000 7f901a000700 1 -- 172.21.15.99:0/3062238755 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9010003e20 con 0x7f901c100d80 2024-03-30T00:38:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.626+0000 7f9024600700 1 -- 172.21.15.99:0/3062238755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c1a0f00 con 0x7f901c100d80 2024-03-30T00:38:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.626+0000 7f901a000700 1 -- 172.21.15.99:0/3062238755 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901000fa30 con 0x7f901c100d80 2024-03-30T00:38:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.626+0000 7f9024600700 1 -- 172.21.15.99:0/3062238755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c1a12c0 con 0x7f901c100d80 2024-03-30T00:38:04.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.627+0000 7f9024600700 1 -- 172.21.15.99:0/3062238755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c067af0 con 0x7f901c100d80 2024-03-30T00:38:04.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.629+0000 7f901a000700 1 -- 172.21.15.99:0/3062238755 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f9010003990 con 0x7f901c100d80 2024-03-30T00:38:04.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.631+0000 7f901a000700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f90040a11a0 0x7f90040a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:04.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.631+0000 7f901a000700 1 -- 172.21.15.99:0/3062238755 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f90100cd650 con 0x7f901c100d80 2024-03-30T00:38:04.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.631+0000 7f9021200700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f90040a11a0 0x7f90040a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:04.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.632+0000 7f9021200700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f90040a11a0 0x7f90040a3660 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9008009c60 tx=0x7f9008009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:04.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.635+0000 7f901a000700 1 -- 172.21.15.99:0/3062238755 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9010096d90 con 0x7f901c100d80 2024-03-30T00:38:04.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:04 smithi161 ceph-mon[42221]: pgmap v404: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-30T00:38:04.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:04 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3165659011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:38:04.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:04 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3559061484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:38:04.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:04 smithi099 ceph-mon[38079]: pgmap v404: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-30T00:38:04.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:04 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3165659011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:38:04.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:04 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3559061484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:38:04.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.841+0000 7f9024600700 1 -- 172.21.15.99:0/3062238755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f901c1a1630 con 0x7f901c100d80 2024-03-30T00:38:04.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.842+0000 7f901a000700 1 -- 172.21.15.99:0/3062238755 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f901009a3b0 con 0x7f901c100d80 2024-03-30T00:38:04.848 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-30T00:38:04.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 -- 172.21.15.99:0/3062238755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f90040a11a0 msgr2=0x7f90040a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:04.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f90040a11a0 0x7f90040a3660 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9008009c60 tx=0x7f9008009260 comp rx=0 tx=0).stop 2024-03-30T00:38:04.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 -- 172.21.15.99:0/3062238755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f901c100d80 msgr2=0x7f901c198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:04.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f901c100d80 0x7f901c198e80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f901000bee0 tx=0x7f90100045e0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 -- 172.21.15.99:0/3062238755 shutdown_connections 2024-03-30T00:38:04.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f90040a11a0 0x7f90040a3660 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f901c102910 0x7f901c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f901c100d80 0x7f901c198e80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 --2- 172.21.15.99:0/3062238755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f901c101f70 0x7f901c1993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:04.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 -- 172.21.15.99:0/3062238755 >> 172.21.15.99:0/3062238755 conn(0x7f901c0fc2f0 msgr2=0x7f901c107760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:04.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 -- 172.21.15.99:0/3062238755 shutdown_connections 2024-03-30T00:38:04.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:04.844+0000 7f9024600700 1 -- 172.21.15.99:0/3062238755 wait complete. 2024-03-30T00:38:04.869 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:38:05.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.055+0000 7f8fe7200700 1 -- 172.21.15.99:0/3411235340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fe0102910 msgr2=0x7f8fe01091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:05.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.055+0000 7f8fe7200700 1 --2- 172.21.15.99:0/3411235340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fe0102910 0x7f8fe01091c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800b0d0 tx=0x7f8fd800b3e0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.055+0000 7f8fe7200700 1 -- 172.21.15.99:0/3411235340 shutdown_connections 2024-03-30T00:38:05.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.055+0000 7f8fe7200700 1 --2- 172.21.15.99:0/3411235340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fe0102910 0x7f8fe01091c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.055+0000 7f8fe7200700 1 --2- 172.21.15.99:0/3411235340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe0100d80 0x7f8fe01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.055+0000 7f8fe7200700 1 --2- 172.21.15.99:0/3411235340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe0101f70 0x7f8fe01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.055+0000 7f8fe7200700 1 -- 172.21.15.99:0/3411235340 >> 172.21.15.99:0/3411235340 conn(0x7f8fe00fc2f0 msgr2=0x7f8fe00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:05.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.055+0000 7f8fe7200700 1 -- 172.21.15.99:0/3411235340 shutdown_connections 2024-03-30T00:38:05.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.055+0000 7f8fe7200700 1 -- 172.21.15.99:0/3411235340 wait complete. 2024-03-30T00:38:05.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.056+0000 7f8fe7200700 1 Processor -- start 2024-03-30T00:38:05.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.056+0000 7f8fe7200700 1 -- start start 2024-03-30T00:38:05.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe7200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe0100d80 0x7f8fe0198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:05.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe7200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe0101f70 0x7f8fe0199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:05.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe7200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fe0102910 0x7f8fe01a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:05.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe7200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe01a0a70 con 0x7f8fe0102910 2024-03-30T00:38:05.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe7200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe01a0bb0 con 0x7f8fe0101f70 2024-03-30T00:38:05.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe7200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe01a0d40 con 0x7f8fe0100d80 2024-03-30T00:38:05.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe5200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fe0102910 0x7f8fe01a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:05.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe5200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fe0102910 0x7f8fe01a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58360/0 (socket says 172.21.15.99:58360) 2024-03-30T00:38:05.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe5200700 1 -- 172.21.15.99:0/2402907814 learned_addr learned my addr 172.21.15.99:0/2402907814 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fdfe00700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe0100d80 0x7f8fe0198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fdf400700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe0101f70 0x7f8fe0199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe5200700 1 -- 172.21.15.99:0/2402907814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe0100d80 msgr2=0x7f8fe0198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe5200700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe0100d80 0x7f8fe0198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe5200700 1 -- 172.21.15.99:0/2402907814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe0101f70 msgr2=0x7f8fe0199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe5200700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe0101f70 0x7f8fe0199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fe5200700 1 -- 172.21.15.99:0/2402907814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8009d00 con 0x7f8fe0102910 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.057+0000 7f8fdf400700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe0101f70 0x7f8fe0199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.058+0000 7f8fe5200700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fe0102910 0x7f8fe01a04a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8007af0 tx=0x7f8fd8007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.058+0000 7f8fdcc00700 1 -- 172.21.15.99:0/2402907814 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd800e040 con 0x7f8fe0102910 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.058+0000 7f8fdcc00700 1 -- 172.21.15.99:0/2402907814 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fd800be30 con 0x7f8fe0102910 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.058+0000 7f8fe7200700 1 -- 172.21.15.99:0/2402907814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe01a0fc0 con 0x7f8fe0102910 2024-03-30T00:38:05.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.058+0000 7f8fdcc00700 1 -- 172.21.15.99:0/2402907814 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd801b930 con 0x7f8fe0102910 2024-03-30T00:38:05.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.058+0000 7f8fe7200700 1 -- 172.21.15.99:0/2402907814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe01a1380 con 0x7f8fe0102910 2024-03-30T00:38:05.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.059+0000 7f8fe7200700 1 -- 172.21.15.99:0/2402907814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe0067af0 con 0x7f8fe0102910 2024-03-30T00:38:05.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.061+0000 7f8fdcc00700 1 -- 172.21.15.99:0/2402907814 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f8fd801d670 con 0x7f8fe0102910 2024-03-30T00:38:05.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.063+0000 7f8fdcc00700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fc40a1120 0x7f8fc40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:05.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.063+0000 7f8fdcc00700 1 -- 172.21.15.99:0/2402907814 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f8fd80ce070 con 0x7f8fe0102910 2024-03-30T00:38:05.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.063+0000 7f8fdfe00700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fc40a1120 0x7f8fc40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:05.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.064+0000 7f8fdfe00700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fc40a1120 0x7f8fc40a35e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0101dd0 tx=0x7f8fd0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:05.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.067+0000 7f8fdcc00700 1 -- 172.21.15.99:0/2402907814 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8fd80977b0 con 0x7f8fe0102910 2024-03-30T00:38:05.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.327+0000 7f8fe7200700 1 -- 172.21.15.99:0/2402907814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8fe01a1650 con 0x7f8fe0102910 2024-03-30T00:38:05.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.327+0000 7f8fdcc00700 1 -- 172.21.15.99:0/2402907814 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+591 (secure 0 0 0) 0x7f8fd809add0 con 0x7f8fe0102910 2024-03-30T00:38:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/1748928762 2024-03-31T00:27:08.932957+0000 2024-03-30T00:38:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2943078453 2024-03-31T00:27:08.932957+0000 2024-03-30T00:38:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/1133935143 2024-03-31T00:26:45.301121+0000 2024-03-30T00:38:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/1748928762 2024-03-31T00:27:08.932957+0000 2024-03-30T00:38:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/1133935143 2024-03-31T00:26:45.301121+0000 2024-03-30T00:38:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/479443591 2024-03-31T00:26:45.301121+0000 2024-03-30T00:38:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/4006319857 2024-03-31T00:26:45.301121+0000 2024-03-30T00:38:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2253836899 2024-03-31T00:27:08.932957+0000 2024-03-30T00:38:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2565551826 2024-03-31T00:26:45.301121+0000 2024-03-30T00:38:05.335 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/1147023397 2024-03-31T00:27:08.932957+0000 2024-03-30T00:38:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 -- 172.21.15.99:0/2402907814 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fc40a1120 msgr2=0x7f8fc40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fc40a1120 0x7f8fc40a35e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0101dd0 tx=0x7f8fd0009260 comp rx=0 tx=0).stop 2024-03-30T00:38:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 -- 172.21.15.99:0/2402907814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fe0102910 msgr2=0x7f8fe01a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:05.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fe0102910 0x7f8fe01a04a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8007af0 tx=0x7f8fd8007870 comp rx=0 tx=0).stop 2024-03-30T00:38:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 -- 172.21.15.99:0/2402907814 shutdown_connections 2024-03-30T00:38:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fe0102910 0x7f8fe01a04a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8fe0100d80 0x7f8fe0198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f8fc40a1120 0x7f8fc40a35e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 --2- 172.21.15.99:0/2402907814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe0101f70 0x7f8fe0199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 -- 172.21.15.99:0/2402907814 >> 172.21.15.99:0/2402907814 conn(0x7f8fe00fc2f0 msgr2=0x7f8fe0107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 -- 172.21.15.99:0/2402907814 shutdown_connections 2024-03-30T00:38:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.330+0000 7f8fe7200700 1 -- 172.21.15.99:0/2402907814 wait complete. 2024-03-30T00:38:05.337 INFO:teuthology.orchestra.run.smithi099.stderr:listed 10 entries 2024-03-30T00:38:05.354 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-30T00:38:05.354 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T00:38:05.397 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T00:38:05.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:05 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/438396775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:38:05.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:05 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3062238755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:38:05.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.622+0000 7ff02ba00700 1 -- 172.21.15.99:0/3587630714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff024100c80 msgr2=0x7ff0241010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:05.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.622+0000 7ff02ba00700 1 --2- 172.21.15.99:0/3587630714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff024100c80 0x7ff0241010a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff0180099c0 tx=0x7ff018009cd0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.622+0000 7ff02ba00700 1 -- 172.21.15.99:0/3587630714 shutdown_connections 2024-03-30T00:38:05.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.622+0000 7ff02ba00700 1 --2- 172.21.15.99:0/3587630714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff024102810 0x7ff0241090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.622+0000 7ff02ba00700 1 --2- 172.21.15.99:0/3587630714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff024100c80 0x7ff0241010a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.622+0000 7ff02ba00700 1 --2- 172.21.15.99:0/3587630714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff024101e70 0x7ff0241022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.622+0000 7ff02ba00700 1 -- 172.21.15.99:0/3587630714 >> 172.21.15.99:0/3587630714 conn(0x7ff0240fc1f0 msgr2=0x7ff0240fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:05.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.622+0000 7ff02ba00700 1 -- 172.21.15.99:0/3587630714 shutdown_connections 2024-03-30T00:38:05.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.622+0000 7ff02ba00700 1 -- 172.21.15.99:0/3587630714 wait complete. 2024-03-30T00:38:05.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.623+0000 7ff02ba00700 1 Processor -- start 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.623+0000 7ff02ba00700 1 -- start start 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.623+0000 7ff02ba00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff024100c80 0x7ff024198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff02ba00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff024101e70 0x7ff0241992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff02ba00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff024102810 0x7ff0241a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff02ba00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0241a0870 con 0x7ff024102810 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff02ba00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0241a09e0 con 0x7ff024100c80 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff02ba00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0241a0ba0 con 0x7ff024101e70 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff029a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff024102810 0x7ff0241a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff029a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff024102810 0x7ff0241a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58366/0 (socket says 172.21.15.99:58366) 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff029a00700 1 -- 172.21.15.99:0/4208254755 learned_addr learned my addr 172.21.15.99:0/4208254755 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff029000700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff024100c80 0x7ff024198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff023e00700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff024101e70 0x7ff0241992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:05.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff029a00700 1 -- 172.21.15.99:0/4208254755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff024101e70 msgr2=0x7ff0241992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff029a00700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff024101e70 0x7ff0241992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff029a00700 1 -- 172.21.15.99:0/4208254755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff024100c80 msgr2=0x7ff024198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff029a00700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff024100c80 0x7ff024198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff029a00700 1 -- 172.21.15.99:0/4208254755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0180095f0 con 0x7ff024102810 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.624+0000 7ff029000700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff024100c80 0x7ff024198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.625+0000 7ff029a00700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff024102810 0x7ff0241a02a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00ef30 tx=0x7ff01c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.625+0000 7ff021600700 1 -- 172.21.15.99:0/4208254755 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c018070 con 0x7ff024102810 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.625+0000 7ff021600700 1 -- 172.21.15.99:0/4208254755 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff01c00f040 con 0x7ff024102810 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.625+0000 7ff02ba00700 1 -- 172.21.15.99:0/4208254755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0241a0e20 con 0x7ff024102810 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.625+0000 7ff021600700 1 -- 172.21.15.99:0/4208254755 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c00cbc0 con 0x7ff024102810 2024-03-30T00:38:05.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.625+0000 7ff02ba00700 1 -- 172.21.15.99:0/4208254755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0241a1250 con 0x7ff024102810 2024-03-30T00:38:05.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.626+0000 7ff02ba00700 1 -- 172.21.15.99:0/4208254755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff024067a10 con 0x7ff024102810 2024-03-30T00:38:05.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.628+0000 7ff021600700 1 -- 172.21.15.99:0/4208254755 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ff01c00cd80 con 0x7ff024102810 2024-03-30T00:38:05.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.629+0000 7ff021600700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff00c0a11a0 0x7ff00c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:05.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.629+0000 7ff021600700 1 -- 172.21.15.99:0/4208254755 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7ff01c0cc410 con 0x7ff024102810 2024-03-30T00:38:05.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.630+0000 7ff029000700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff00c0a11a0 0x7ff00c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:05.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.630+0000 7ff029000700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff00c0a11a0 0x7ff00c0a3660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff018009990 tx=0x7ff01801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:05.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.634+0000 7ff021600700 1 -- 172.21.15.99:0/4208254755 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff01c095b50 con 0x7ff024102810 2024-03-30T00:38:05.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:05 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/438396775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:38:05.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:05 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3062238755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:38:05.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:05 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/438396775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:38:05.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:05 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3062238755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:38:05.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.860+0000 7ff02ba00700 1 -- 172.21.15.99:0/4208254755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff02410c390 con 0x7ff024102810 2024-03-30T00:38:05.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.862+0000 7ff021600700 1 -- 172.21.15.99:0/4208254755 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22859 (secure 0 0 0) 0x7ff01c099170 con 0x7ff024102810 2024-03-30T00:38:05.870 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-30T00:38:05.870 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":87,"fsid":"03801b1a-ee2c-11ee-b647-cb9ed24678a4","created":"2024-03-30T00:26:14.505850+0000","modified":"2024-03-30T00:35:10.474499+0000","last_up_change":"2024-03-30T00:32:32.037638+0000","last_in_change":"2024-03-30T00:32:12.669987+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T00:29:17.554395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T00:33:14.085184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T00:33:15.090699+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c244d549-0aff-48af-9523-f7096beda3e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6803","nonce":3911742857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6805","nonce":3911742857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6809","nonce":3911742857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":3911742857},{"type":"v1","addr":"172.21.15.99:6807","nonce":3911742857}]},"public_addr":"172.21.15.99:6803/3911742857","cluster_addr":"172.21.15.99:6805/3911742857","heartbeat_back_addr":"172.21.15.99:6809/3911742857","heartbeat_front_addr":"172.21.15.99:6807/3911742857","state":["exists","up"]},{"osd":1,"uuid":"0b541e0e-4b00-4af5-92c5-adb75a359bbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6801","nonce":912538361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6803","nonce":912538361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6807","nonce":912538361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":912538361},{"type":"v1","addr":"172.21.15.142:6805","nonce":912538361}]},"public_addr":"172.21.15.142:6801/912538361","cluster_addr":"172.21.15.142:6803/912538361","heartbeat_back_addr":"172.21.15.142:6807/912538361","heartbeat_front_addr":"172.21.15.142:6805/912538361","state":["exists","up"]},{"osd":2,"uuid":"e0faf760-1f43-4799-bc8d-bdc2f8ba5386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6801","nonce":1904456041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6803","nonce":1904456041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6807","nonce":1904456041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1904456041},{"type":"v1","addr":"172.21.15.161:6805","nonce":1904456041}]},"public_addr":"172.21.15.161:6801/1904456041","cluster_addr":"172.21.15.161:6803/1904456041","heartbeat_back_addr":"172.21.15.161:6807/1904456041","heartbeat_front_addr":"172.21.15.161:6805/1904456041","state":["exists","up"]},{"osd":3,"uuid":"5a75bf9a-8f16-4d28-9685-cdbc9a5a9325","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6811","nonce":1432967840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6813","nonce":1432967840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6817","nonce":1432967840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":1432967840},{"type":"v1","addr":"172.21.15.99:6815","nonce":1432967840}]},"public_addr":"172.21.15.99:6811/1432967840","cluster_addr":"172.21.15.99:6813/1432967840","heartbeat_back_addr":"172.21.15.99:6817/1432967840","heartbeat_front_addr":"172.21.15.99:6815/1432967840","state":["exists","up"]},{"osd":4,"uuid":"68d47704-9d21-440d-a85a-46cd077d50d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6809","nonce":1812504396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6811","nonce":1812504396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6815","nonce":1812504396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1812504396},{"type":"v1","addr":"172.21.15.142:6813","nonce":1812504396}]},"public_addr":"172.21.15.142:6809/1812504396","cluster_addr":"172.21.15.142:6811/1812504396","heartbeat_back_addr":"172.21.15.142:6815/1812504396","heartbeat_front_addr":"172.21.15.142:6813/1812504396","state":["exists","up"]},{"osd":5,"uuid":"64cdf569-3dfe-4792-bcb0-f0031e0dbf7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6809","nonce":235239367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6811","nonce":235239367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6815","nonce":235239367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":235239367},{"type":"v1","addr":"172.21.15.161:6813","nonce":235239367}]},"public_addr":"172.21.15.161:6809/235239367","cluster_addr":"172.21.15.161:6811/235239367","heartbeat_back_addr":"172.21.15.161:6815/235239367","heartbeat_front_addr":"172.21.15.161:6813/235239367","state":["exists","up"]},{"osd":6,"uuid":"ce11a006-0d8d-45d4-a9a8-e465307c6025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6819","nonce":2055935080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6821","nonce":2055935080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6825","nonce":2055935080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":2055935080},{"type":"v1","addr":"172.21.15.99:6823","nonce":2055935080}]},"public_addr":"172.21.15.99:6819/2055935080","cluster_addr":"172.21.15.99:6821/2055935080","heartbeat_back_addr":"172.21.15.99:6825/2055935080","heartbeat_front_addr":"172.21.15.99:6823/2055935080","state":["exists","up"]},{"osd":7,"uuid":"f762698a-4112-4f4e-9658-3f91393ef1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6817","nonce":445950815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6819","nonce":445950815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6823","nonce":445950815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":445950815},{"type":"v1","addr":"172.21.15.142:6821","nonce":445950815}]},"public_addr":"172.21.15.142:6817/445950815","cluster_addr":"172.21.15.142:6819/445950815","heartbeat_back_addr":"172.21.15.142:6823/445950815","heartbeat_front_addr":"172.21.15.142:6821/445950815","state":["exists","up"]},{"osd":8,"uuid":"9aeb4ef6-b4c1-4f0e-90d2-f5be79a8bcfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6817","nonce":2321115120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6819","nonce":2321115120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6823","nonce":2321115120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2321115120},{"type":"v1","addr":"172.21.15.161:6821","nonce":2321115120}]},"public_addr":"172.21.15.161:6817/2321115120","cluster_addr":"172.21.15.161:6819/2321115120","heartbeat_back_addr":"172.21.15.161:6823/2321115120","heartbeat_front_addr":"172.21.15.161:6821/2321115120","state":["exists","up"]},{"osd":9,"uuid":"3cb6f263-248a-417e-b8a4-37afed246071","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6827","nonce":3086664896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6829","nonce":3086664896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6833","nonce":3086664896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3086664896},{"type":"v1","addr":"172.21.15.99:6831","nonce":3086664896}]},"public_addr":"172.21.15.99:6827/3086664896","cluster_addr":"172.21.15.99:6829/3086664896","heartbeat_back_addr":"172.21.15.99:6833/3086664896","heartbeat_front_addr":"172.21.15.99:6831/3086664896","state":["exists","up"]},{"osd":10,"uuid":"c2d0d43f-b79c-4830-926f-8d51fb41eccd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6825","nonce":3224161760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6827","nonce":3224161760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6831","nonce":3224161760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":3224161760},{"type":"v1","addr":"172.21.15.142:6829","nonce":3224161760}]},"public_addr":"172.21.15.142:6825/3224161760","cluster_addr":"172.21.15.142:6827/3224161760","heartbeat_back_addr":"172.21.15.142:6831/3224161760","heartbeat_front_addr":"172.21.15.142:6829/3224161760","state":["exists","up"]},{"osd":11,"uuid":"2d7a3b48-85d3-41d2-8a24-288b8250e803","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6825","nonce":3737118097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6827","nonce":3737118097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6831","nonce":3737118097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3737118097},{"type":"v1","addr":"172.21.15.161:6829","nonce":3737118097}]},"public_addr":"172.21.15.161:6825/3737118097","cluster_addr":"172.21.15.161:6827/3737118097","heartbeat_back_addr":"172.21.15.161:6831/3737118097","heartbeat_front_addr":"172.21.15.161:6829/3737118097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:31.310229+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:28:52.420863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:13.974500+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:35.447171+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:29:56.809386+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:18.737684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:30:40.270746+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:02.064482+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:24.153130+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:31:45.885552+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:07.493601+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T00:32:29.631338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8},{"from":5,"to":7}]},{"pgid":"3.3d","mappings":[{"from":1,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6801/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2943078453":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6801/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:6800/1748928762":"2024-03-31T00:27:08.932957+0000","172.21.15.99:6800/1133935143":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/479443591":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/4006319857":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/2253836899":"2024-03-31T00:27:08.932957+0000","172.21.15.99:0/2565551826":"2024-03-31T00:26:45.301121+0000","172.21.15.99:0/1147023397":"2024-03-31T00:27:08.932957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T00:38:05.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.864+0000 7ff02ba00700 1 -- 172.21.15.99:0/4208254755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff00c0a11a0 msgr2=0x7ff00c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:05.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.864+0000 7ff02ba00700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff00c0a11a0 0x7ff00c0a3660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff018009990 tx=0x7ff01801a040 comp rx=0 tx=0).stop 2024-03-30T00:38:05.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.864+0000 7ff02ba00700 1 -- 172.21.15.99:0/4208254755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff024102810 msgr2=0x7ff0241a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:05.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.864+0000 7ff02ba00700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff024102810 0x7ff0241a02a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00ef30 tx=0x7ff01c00c3a0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.864+0000 7ff02ba00700 1 -- 172.21.15.99:0/4208254755 shutdown_connections 2024-03-30T00:38:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.864+0000 7ff02ba00700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff024102810 0x7ff0241a02a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.864+0000 7ff02ba00700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7ff00c0a11a0 0x7ff00c0a3660 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.864+0000 7ff02ba00700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff024100c80 0x7ff024198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.864+0000 7ff02ba00700 1 --2- 172.21.15.99:0/4208254755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff024101e70 0x7ff0241992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.864+0000 7ff02ba00700 1 -- 172.21.15.99:0/4208254755 >> 172.21.15.99:0/4208254755 conn(0x7ff0240fc1f0 msgr2=0x7ff024107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.865+0000 7ff02ba00700 1 -- 172.21.15.99:0/4208254755 shutdown_connections 2024-03-30T00:38:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:05.865+0000 7ff02ba00700 1 -- 172.21.15.99:0/4208254755 wait complete. 2024-03-30T00:38:05.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T00:29:17.554395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T00:33:14.085184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T00:33:15.090699+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T00:38:05.890 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T00:38:06.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.078+0000 7efd04200700 1 -- 172.21.15.99:0/3602840598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcfc102910 msgr2=0x7efcfc1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.078+0000 7efd04200700 1 --2- 172.21.15.99:0/3602840598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcfc102910 0x7efcfc1091c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efcf400b0d0 tx=0x7efcf400b3e0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.078+0000 7efd04200700 1 -- 172.21.15.99:0/3602840598 shutdown_connections 2024-03-30T00:38:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.078+0000 7efd04200700 1 --2- 172.21.15.99:0/3602840598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcfc102910 0x7efcfc1091c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.078+0000 7efd04200700 1 --2- 172.21.15.99:0/3602840598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcfc100d80 0x7efcfc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.078+0000 7efd04200700 1 --2- 172.21.15.99:0/3602840598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc101f70 0x7efcfc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.078+0000 7efd04200700 1 -- 172.21.15.99:0/3602840598 >> 172.21.15.99:0/3602840598 conn(0x7efcfc0fc2f0 msgr2=0x7efcfc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.078+0000 7efd04200700 1 -- 172.21.15.99:0/3602840598 shutdown_connections 2024-03-30T00:38:06.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.079+0000 7efd04200700 1 -- 172.21.15.99:0/3602840598 wait complete. 2024-03-30T00:38:06.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.079+0000 7efd04200700 1 Processor -- start 2024-03-30T00:38:06.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd04200700 1 -- start start 2024-03-30T00:38:06.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd04200700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcfc100d80 0x7efcfc074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd04200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcfc101f70 0x7efcfc0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd04200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc102910 0x7efcfc0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd04200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc073e80 con 0x7efcfc101f70 2024-03-30T00:38:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd04200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc073ff0 con 0x7efcfc102910 2024-03-30T00:38:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd04200700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1a0c80 con 0x7efcfc100d80 2024-03-30T00:38:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd00e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcfc101f70 0x7efcfc0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd00e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcfc101f70 0x7efcfc0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58390/0 (socket says 172.21.15.99:58390) 2024-03-30T00:38:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd00e00700 1 -- 172.21.15.99:0/3965790047 learned_addr learned my addr 172.21.15.99:0/3965790047 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:38:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd01800700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcfc100d80 0x7efcfc074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.080+0000 7efd02200700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc102910 0x7efcfc0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.081+0000 7efd00e00700 1 -- 172.21.15.99:0/3965790047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcfc100d80 msgr2=0x7efcfc074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.081+0000 7efd00e00700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcfc100d80 0x7efcfc074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.081+0000 7efd00e00700 1 -- 172.21.15.99:0/3965790047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc102910 msgr2=0x7efcfc0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.081+0000 7efd00e00700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc102910 0x7efcfc0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.081+0000 7efd00e00700 1 -- 172.21.15.99:0/3965790047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf4009d00 con 0x7efcfc101f70 2024-03-30T00:38:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.081+0000 7efd00e00700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcfc101f70 0x7efcfc0731a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efce800c9b0 tx=0x7efce800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.081+0000 7efcfa000700 1 -- 172.21.15.99:0/3965790047 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8007aa0 con 0x7efcfc101f70 2024-03-30T00:38:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.081+0000 7efcfa000700 1 -- 172.21.15.99:0/3965790047 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efce800ce50 con 0x7efcfc101f70 2024-03-30T00:38:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.081+0000 7efd04200700 1 -- 172.21.15.99:0/3965790047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc1a0f10 con 0x7efcfc101f70 2024-03-30T00:38:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.081+0000 7efcfa000700 1 -- 172.21.15.99:0/3965790047 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8018400 con 0x7efcfc101f70 2024-03-30T00:38:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.081+0000 7efd04200700 1 -- 172.21.15.99:0/3965790047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc1a13e0 con 0x7efcfc101f70 2024-03-30T00:38:06.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.082+0000 7efd04200700 1 -- 172.21.15.99:0/3965790047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc1080d0 con 0x7efcfc101f70 2024-03-30T00:38:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.084+0000 7efcfa000700 1 -- 172.21.15.99:0/3965790047 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7efce800f450 con 0x7efcfc101f70 2024-03-30T00:38:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.085+0000 7efcfa000700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7efce40a11a0 0x7efce40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.086+0000 7efcfa000700 1 -- 172.21.15.99:0/3965790047 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7efce80cc840 con 0x7efcfc101f70 2024-03-30T00:38:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.086+0000 7efd01800700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7efce40a11a0 0x7efce40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.086+0000 7efd01800700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7efce40a11a0 0x7efce40a3660 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efcfc101dd0 tx=0x7efcf000a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:06.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.089+0000 7efcfa000700 1 -- 172.21.15.99:0/3965790047 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efce8095ef0 con 0x7efcfc101f70 2024-03-30T00:38:06.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.298+0000 7efd04200700 1 -- 172.21.15.99:0/3965790047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcfc02ccb0 con 0x7efcfc101f70 2024-03-30T00:38:06.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.298+0000 7efcfa000700 1 -- 172.21.15.99:0/3965790047 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7efce8099510 con 0x7efcfc101f70 2024-03-30T00:38:06.305 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-30T00:38:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 -- 172.21.15.99:0/3965790047 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7efce40a11a0 msgr2=0x7efce40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7efce40a11a0 0x7efce40a3660 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efcfc101dd0 tx=0x7efcf000a3b0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 -- 172.21.15.99:0/3965790047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcfc101f70 msgr2=0x7efcfc0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcfc101f70 0x7efcfc0731a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efce800c9b0 tx=0x7efce800ccc0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 -- 172.21.15.99:0/3965790047 shutdown_connections 2024-03-30T00:38:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7efce40a11a0 0x7efce40a3660 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efcfc102910 0x7efcfc0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcfc100d80 0x7efcfc074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 --2- 172.21.15.99:0/3965790047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcfc101f70 0x7efcfc0731a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 -- 172.21.15.99:0/3965790047 >> 172.21.15.99:0/3965790047 conn(0x7efcfc0fc2f0 msgr2=0x7efcfc105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 -- 172.21.15.99:0/3965790047 shutdown_connections 2024-03-30T00:38:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.301+0000 7efd04200700 1 -- 172.21.15.99:0/3965790047 wait complete. 2024-03-30T00:38:06.325 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T00:38:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.513+0000 7f70f8600700 1 -- 172.21.15.99:0/562866104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f0101f70 msgr2=0x7f70f01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.513+0000 7f70f8600700 1 --2- 172.21.15.99:0/562866104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f0101f70 0x7f70f01023d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0099c0 tx=0x7f70dc009cd0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.513+0000 7f70f8600700 1 -- 172.21.15.99:0/562866104 shutdown_connections 2024-03-30T00:38:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.513+0000 7f70f8600700 1 --2- 172.21.15.99:0/562866104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70f0102910 0x7f70f01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.513+0000 7f70f8600700 1 --2- 172.21.15.99:0/562866104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f0100d80 0x7f70f01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.513+0000 7f70f8600700 1 --2- 172.21.15.99:0/562866104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f0101f70 0x7f70f01023d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.513+0000 7f70f8600700 1 -- 172.21.15.99:0/562866104 >> 172.21.15.99:0/562866104 conn(0x7f70f00fc2f0 msgr2=0x7f70f00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.513+0000 7f70f8600700 1 -- 172.21.15.99:0/562866104 shutdown_connections 2024-03-30T00:38:06.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.513+0000 7f70f8600700 1 -- 172.21.15.99:0/562866104 wait complete. 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.514+0000 7f70f8600700 1 Processor -- start 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.514+0000 7f70f8600700 1 -- start start 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.514+0000 7f70f8600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70f0100d80 0x7f70f0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f8600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f0101f70 0x7f70f00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f8600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f0102910 0x7f70f00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f8600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f0073e80 con 0x7f70f0101f70 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f8600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f0073ff0 con 0x7f70f0100d80 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f8600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f01a0ca0 con 0x7f70f0102910 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f5200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f0101f70 0x7f70f00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f5200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f0101f70 0x7f70f00731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58418/0 (socket says 172.21.15.99:58418) 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f5200700 1 -- 172.21.15.99:0/91727395 learned_addr learned my addr 172.21.15.99:0/91727395 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f6600700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f0102910 0x7f70f00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f5c00700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70f0100d80 0x7f70f0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f5200700 1 -- 172.21.15.99:0/91727395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f0102910 msgr2=0x7f70f00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f5200700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f0102910 0x7f70f00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f5200700 1 -- 172.21.15.99:0/91727395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70f0100d80 msgr2=0x7f70f0074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f5200700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70f0100d80 0x7f70f0074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.515+0000 7f70f5200700 1 -- 172.21.15.99:0/91727395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0095f0 con 0x7f70f0101f70 2024-03-30T00:38:06.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.516+0000 7f70f5200700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f0101f70 0x7f70f00731a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009990 tx=0x7f70dc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:06.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.516+0000 7f70ee000700 1 -- 172.21.15.99:0/91727395 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc01d070 con 0x7f70f0101f70 2024-03-30T00:38:06.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.516+0000 7f70ee000700 1 -- 172.21.15.99:0/91727395 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70dc00bd10 con 0x7f70f0101f70 2024-03-30T00:38:06.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.516+0000 7f70f8600700 1 -- 172.21.15.99:0/91727395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f01a0ed0 con 0x7f70f0101f70 2024-03-30T00:38:06.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.516+0000 7f70ee000700 1 -- 172.21.15.99:0/91727395 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc017650 con 0x7f70f0101f70 2024-03-30T00:38:06.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.516+0000 7f70f8600700 1 -- 172.21.15.99:0/91727395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f01a1240 con 0x7f70f0101f70 2024-03-30T00:38:06.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.517+0000 7f70f8600700 1 -- 172.21.15.99:0/91727395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f01082a0 con 0x7f70f0101f70 2024-03-30T00:38:06.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.519+0000 7f70ee000700 1 -- 172.21.15.99:0/91727395 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f70dc022470 con 0x7f70f0101f70 2024-03-30T00:38:06.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.520+0000 7f70ee000700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f70d80a1120 0x7f70d80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.520+0000 7f70ee000700 1 -- 172.21.15.99:0/91727395 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f70dc0ce0b0 con 0x7f70f0101f70 2024-03-30T00:38:06.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.520+0000 7f70f5c00700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f70d80a1120 0x7f70d80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.521+0000 7f70f5c00700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f70d80a1120 0x7f70d80a35e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f70f0101dd0 tx=0x7f70e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:06.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.522+0000 7f70ee000700 1 -- 172.21.15.99:0/91727395 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f70dc0977f0 con 0x7f70f0101f70 2024-03-30T00:38:06.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:06 smithi142 ceph-mon[41779]: pgmap v405: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:38:06.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:06 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/2402907814' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T00:38:06.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:06 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/4208254755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:38:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:06 smithi161 ceph-mon[42221]: pgmap v405: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:38:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:06 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/2402907814' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T00:38:06.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:06 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/4208254755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:38:06.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.728+0000 7f70f8600700 1 -- 172.21.15.99:0/91727395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70f010c490 con 0x7f70f0101f70 2024-03-30T00:38:06.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.729+0000 7f70ee000700 1 -- 172.21.15.99:0/91727395 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f70dc09ae10 con 0x7f70f0101f70 2024-03-30T00:38:06.735 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-30T00:38:06.736 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:06 smithi099 ceph-mon[38079]: pgmap v405: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-30T00:38:06.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:06 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/2402907814' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T00:38:06.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:06 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/4208254755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T00:38:06.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.731+0000 7f70f8600700 1 -- 172.21.15.99:0/91727395 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f70d80a1120 msgr2=0x7f70d80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.731+0000 7f70f8600700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f70d80a1120 0x7f70d80a35e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f70f0101dd0 tx=0x7f70e4009260 comp rx=0 tx=0).stop 2024-03-30T00:38:06.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.731+0000 7f70f8600700 1 -- 172.21.15.99:0/91727395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f0101f70 msgr2=0x7f70f00731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.731+0000 7f70f8600700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f0101f70 0x7f70f00731a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009990 tx=0x7f70dc00fa00 comp rx=0 tx=0).stop 2024-03-30T00:38:06.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.732+0000 7f70f8600700 1 -- 172.21.15.99:0/91727395 shutdown_connections 2024-03-30T00:38:06.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.732+0000 7f70f8600700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f0102910 0x7f70f00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.732+0000 7f70f8600700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f70d80a1120 0x7f70d80a35e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.732+0000 7f70f8600700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70f0100d80 0x7f70f0074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.732+0000 7f70f8600700 1 --2- 172.21.15.99:0/91727395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70f0101f70 0x7f70f00731a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.732+0000 7f70f8600700 1 -- 172.21.15.99:0/91727395 >> 172.21.15.99:0/91727395 conn(0x7f70f00fc2f0 msgr2=0x7f70f01053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:06.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.732+0000 7f70f8600700 1 -- 172.21.15.99:0/91727395 shutdown_connections 2024-03-30T00:38:06.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.732+0000 7f70f8600700 1 -- 172.21.15.99:0/91727395 wait complete. 2024-03-30T00:38:06.757 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T00:38:06.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.946+0000 7f7e4f600700 1 -- 172.21.15.99:0/720541360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48100d80 msgr2=0x7f7e481011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.946+0000 7f7e4f600700 1 --2- 172.21.15.99:0/720541360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48100d80 0x7f7e481011a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0099c0 tx=0x7f7e3c009cd0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.946+0000 7f7e4f600700 1 -- 172.21.15.99:0/720541360 shutdown_connections 2024-03-30T00:38:06.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.946+0000 7f7e4f600700 1 --2- 172.21.15.99:0/720541360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e48102910 0x7f7e481091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.946+0000 7f7e4f600700 1 --2- 172.21.15.99:0/720541360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48100d80 0x7f7e481011a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.946+0000 7f7e4f600700 1 --2- 172.21.15.99:0/720541360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e48101f70 0x7f7e481023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.946+0000 7f7e4f600700 1 -- 172.21.15.99:0/720541360 >> 172.21.15.99:0/720541360 conn(0x7f7e480fc2f0 msgr2=0x7f7e480fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:06.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.946+0000 7f7e4f600700 1 -- 172.21.15.99:0/720541360 shutdown_connections 2024-03-30T00:38:06.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.946+0000 7f7e4f600700 1 -- 172.21.15.99:0/720541360 wait complete. 2024-03-30T00:38:06.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.947+0000 7f7e4f600700 1 Processor -- start 2024-03-30T00:38:06.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.947+0000 7f7e4f600700 1 -- start start 2024-03-30T00:38:06.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4f600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48100d80 0x7f7e48196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4f600700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e48101f70 0x7f7e48197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e48102910 0x7f7e4819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4f600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4819e740 con 0x7f7e48100d80 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4819e880 con 0x7f7e48102910 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4f600700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4819ea10 con 0x7f7e48101f70 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48100d80 0x7f7e48196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48100d80 0x7f7e48196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58444/0 (socket says 172.21.15.99:58444) 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4cc00700 1 -- 172.21.15.99:0/1406125561 learned_addr learned my addr 172.21.15.99:0/1406125561 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e47e00700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e48101f70 0x7f7e48197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4d600700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e48102910 0x7f7e4819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4cc00700 1 -- 172.21.15.99:0/1406125561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e48101f70 msgr2=0x7f7e48197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4cc00700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e48101f70 0x7f7e48197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4cc00700 1 -- 172.21.15.99:0/1406125561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e48102910 msgr2=0x7f7e4819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4cc00700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e48102910 0x7f7e4819e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:06.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.948+0000 7f7e4cc00700 1 -- 172.21.15.99:0/1406125561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c0095f0 con 0x7f7e48100d80 2024-03-30T00:38:06.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.949+0000 7f7e4cc00700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48100d80 0x7f7e48196c10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009f50 tx=0x7f7e3c003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:06.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.949+0000 7f7e45600700 1 -- 172.21.15.99:0/1406125561 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c01d070 con 0x7f7e48100d80 2024-03-30T00:38:06.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.949+0000 7f7e45600700 1 -- 172.21.15.99:0/1406125561 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e3c00bda0 con 0x7f7e48100d80 2024-03-30T00:38:06.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.949+0000 7f7e4f600700 1 -- 172.21.15.99:0/1406125561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4819ec90 con 0x7f7e48100d80 2024-03-30T00:38:06.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.949+0000 7f7e45600700 1 -- 172.21.15.99:0/1406125561 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c022d60 con 0x7f7e48100d80 2024-03-30T00:38:06.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.949+0000 7f7e4f600700 1 -- 172.21.15.99:0/1406125561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4819f110 con 0x7f7e48100d80 2024-03-30T00:38:06.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.950+0000 7f7e4f600700 1 -- 172.21.15.99:0/1406125561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e48108250 con 0x7f7e48100d80 2024-03-30T00:38:06.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.952+0000 7f7e45600700 1 -- 172.21.15.99:0/1406125561 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7e3c022500 con 0x7f7e48100d80 2024-03-30T00:38:06.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.953+0000 7f7e45600700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e300a11a0 0x7f7e300a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:06.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.953+0000 7f7e45600700 1 -- 172.21.15.99:0/1406125561 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7f7e3c0cdff0 con 0x7f7e48100d80 2024-03-30T00:38:06.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.953+0000 7f7e47e00700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e300a11a0 0x7f7e300a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:06.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.954+0000 7f7e47e00700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e300a11a0 0x7f7e300a3660 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400ac50 tx=0x7f7e3400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:06.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:06.955+0000 7f7e45600700 1 -- 172.21.15.99:0/1406125561 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7e3c097730 con 0x7f7e48100d80 2024-03-30T00:38:07.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.163+0000 7f7e4f600700 1 -- 172.21.15.99:0/1406125561 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e4810c490 con 0x7f7e48100d80 2024-03-30T00:38:07.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.163+0000 7f7e45600700 1 -- 172.21.15.99:0/1406125561 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e3c09ad50 con 0x7f7e48100d80 2024-03-30T00:38:07.170 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-30T00:38:07.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.165+0000 7f7e4f600700 1 -- 172.21.15.99:0/1406125561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e300a11a0 msgr2=0x7f7e300a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:07.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.165+0000 7f7e4f600700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e300a11a0 0x7f7e300a3660 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400ac50 tx=0x7f7e3400a380 comp rx=0 tx=0).stop 2024-03-30T00:38:07.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.166+0000 7f7e4f600700 1 -- 172.21.15.99:0/1406125561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48100d80 msgr2=0x7f7e48196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:07.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.166+0000 7f7e4f600700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48100d80 0x7f7e48196c10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009f50 tx=0x7f7e3c003a50 comp rx=0 tx=0).stop 2024-03-30T00:38:07.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.166+0000 7f7e4f600700 1 -- 172.21.15.99:0/1406125561 shutdown_connections 2024-03-30T00:38:07.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.166+0000 7f7e4f600700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7f7e300a11a0 0x7f7e300a3660 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.166+0000 7f7e4f600700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7e48102910 0x7f7e4819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.166+0000 7f7e4f600700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e48100d80 0x7f7e48196c10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.166+0000 7f7e4f600700 1 --2- 172.21.15.99:0/1406125561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e48101f70 0x7f7e48197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.166+0000 7f7e4f600700 1 -- 172.21.15.99:0/1406125561 >> 172.21.15.99:0/1406125561 conn(0x7f7e480fc2f0 msgr2=0x7f7e48105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.166+0000 7f7e4f600700 1 -- 172.21.15.99:0/1406125561 shutdown_connections 2024-03-30T00:38:07.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.166+0000 7f7e4f600700 1 -- 172.21.15.99:0/1406125561 wait complete. 2024-03-30T00:38:07.190 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T00:38:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.378+0000 7fbbeec00700 1 -- 172.21.15.99:0/4117810331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbe8100d80 msgr2=0x7fbbe81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.378+0000 7fbbeec00700 1 --2- 172.21.15.99:0/4117810331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbe8100d80 0x7fbbe81011a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80099c0 tx=0x7fbbd8009cd0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.378+0000 7fbbeec00700 1 -- 172.21.15.99:0/4117810331 shutdown_connections 2024-03-30T00:38:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.378+0000 7fbbeec00700 1 --2- 172.21.15.99:0/4117810331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe8102910 0x7fbbe81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.378+0000 7fbbeec00700 1 --2- 172.21.15.99:0/4117810331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbe8100d80 0x7fbbe81011a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.378+0000 7fbbeec00700 1 --2- 172.21.15.99:0/4117810331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8101f70 0x7fbbe81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.378+0000 7fbbeec00700 1 -- 172.21.15.99:0/4117810331 >> 172.21.15.99:0/4117810331 conn(0x7fbbe80fc2f0 msgr2=0x7fbbe80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.379+0000 7fbbeec00700 1 -- 172.21.15.99:0/4117810331 shutdown_connections 2024-03-30T00:38:07.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.379+0000 7fbbeec00700 1 -- 172.21.15.99:0/4117810331 wait complete. 2024-03-30T00:38:07.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.379+0000 7fbbeec00700 1 Processor -- start 2024-03-30T00:38:07.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.380+0000 7fbbeec00700 1 -- start start 2024-03-30T00:38:07.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.380+0000 7fbbeec00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8100d80 0x7fbbe8074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:07.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.380+0000 7fbbeec00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbe8101f70 0x7fbbe80731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:07.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.380+0000 7fbbeec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe8102910 0x7fbbe80736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:07.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.380+0000 7fbbeec00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8073e80 con 0x7fbbe8101f70 2024-03-30T00:38:07.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.380+0000 7fbbeec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8073ff0 con 0x7fbbe8102910 2024-03-30T00:38:07.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.380+0000 7fbbeec00700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe81a0c70 con 0x7fbbe8100d80 2024-03-30T00:38:07.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.380+0000 7fbbe7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbe8101f70 0x7fbbe80731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:07.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.380+0000 7fbbe7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbe8101f70 0x7fbbe80731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58468/0 (socket says 172.21.15.99:58468) 2024-03-30T00:38:07.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.380+0000 7fbbe7400700 1 -- 172.21.15.99:0/3348381388 learned_addr learned my addr 172.21.15.99:0/3348381388 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.380+0000 7fbbe7e00700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8100d80 0x7fbbe8074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbecc00700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe8102910 0x7fbbe80736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbe7400700 1 -- 172.21.15.99:0/3348381388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8100d80 msgr2=0x7fbbe8074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbe7400700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8100d80 0x7fbbe8074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbe7400700 1 -- 172.21.15.99:0/3348381388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe8102910 msgr2=0x7fbbe80736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbe7400700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe8102910 0x7fbbe80736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbe7400700 1 -- 172.21.15.99:0/3348381388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd80095f0 con 0x7fbbe8101f70 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbe7400700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbe8101f70 0x7fbbe80731a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e9a0 tx=0x7fbbd400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbe4c00700 1 -- 172.21.15.99:0/3348381388 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4009790 con 0x7fbbe8101f70 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbe4c00700 1 -- 172.21.15.99:0/3348381388 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbd4010430 con 0x7fbbe8101f70 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbeec00700 1 -- 172.21.15.99:0/3348381388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe81a0f00 con 0x7fbbe8101f70 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbe4c00700 1 -- 172.21.15.99:0/3348381388 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4018680 con 0x7fbbe8101f70 2024-03-30T00:38:07.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.381+0000 7fbbeec00700 1 -- 172.21.15.99:0/3348381388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe81a13d0 con 0x7fbbe8101f70 2024-03-30T00:38:07.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.382+0000 7fbbeec00700 1 -- 172.21.15.99:0/3348381388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe8108250 con 0x7fbbe8101f70 2024-03-30T00:38:07.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.384+0000 7fbbe4c00700 1 -- 172.21.15.99:0/3348381388 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fbbd40105a0 con 0x7fbbe8101f70 2024-03-30T00:38:07.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.385+0000 7fbbe4c00700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbcc0a11a0 0x7fbbcc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T00:38:07.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.385+0000 7fbbe4c00700 1 -- 172.21.15.99:0/3348381388 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8364+0+0 (secure 0 0 0) 0x7fbbd4014070 con 0x7fbbe8101f70 2024-03-30T00:38:07.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.386+0000 7fbbe7e00700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbcc0a11a0 0x7fbbcc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T00:38:07.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.386+0000 7fbbe7e00700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbcc0a11a0 0x7fbbcc0a3660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009990 tx=0x7fbbd8003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T00:38:07.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.389+0000 7fbbe4c00700 1 -- 172.21.15.99:0/3348381388 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbbd40529f0 con 0x7fbbe8101f70 2024-03-30T00:38:07.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:07 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3965790047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:38:07.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:07 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/91727395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:38:07.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:07 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/1406125561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:38:07.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:07 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3965790047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:38:07.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:07 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/91727395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:38:07.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:07 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/1406125561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:38:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.646+0000 7fbbeec00700 1 -- 172.21.15.99:0/3348381388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbbe8067af0 con 0x7fbbe8101f70 2024-03-30T00:38:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.647+0000 7fbbe4c00700 1 -- 172.21.15.99:0/3348381388 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+591 (secure 0 0 0) 0x7fbbd40961a0 con 0x7fbbe8101f70 2024-03-30T00:38:07.653 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/1748928762 2024-03-31T00:27:08.932957+0000 2024-03-30T00:38:07.653 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2943078453 2024-03-31T00:27:08.932957+0000 2024-03-30T00:38:07.653 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6801/1133935143 2024-03-31T00:26:45.301121+0000 2024-03-30T00:38:07.653 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/1748928762 2024-03-31T00:27:08.932957+0000 2024-03-30T00:38:07.653 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:6800/1133935143 2024-03-31T00:26:45.301121+0000 2024-03-30T00:38:07.654 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/479443591 2024-03-31T00:26:45.301121+0000 2024-03-30T00:38:07.654 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/4006319857 2024-03-31T00:26:45.301121+0000 2024-03-30T00:38:07.654 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2253836899 2024-03-31T00:27:08.932957+0000 2024-03-30T00:38:07.654 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/2565551826 2024-03-31T00:26:45.301121+0000 2024-03-30T00:38:07.654 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99:0/1147023397 2024-03-31T00:27:08.932957+0000 2024-03-30T00:38:07.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 -- 172.21.15.99:0/3348381388 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbcc0a11a0 msgr2=0x7fbbcc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbcc0a11a0 0x7fbbcc0a3660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009990 tx=0x7fbbd8003680 comp rx=0 tx=0).stop 2024-03-30T00:38:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 -- 172.21.15.99:0/3348381388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbe8101f70 msgr2=0x7fbbe80731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T00:38:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbe8101f70 0x7fbbe80731a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e9a0 tx=0x7fbbd400ecb0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 -- 172.21.15.99:0/3348381388 shutdown_connections 2024-03-30T00:38:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.99:6800/1501731896,v1:172.21.15.99:6801/1501731896] conn(0x7fbbcc0a11a0 0x7fbbcc0a3660 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe8102910 0x7fbbe80736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbe8100d80 0x7fbbe8074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 --2- 172.21.15.99:0/3348381388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbe8101f70 0x7fbbe80731a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T00:38:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 -- 172.21.15.99:0/3348381388 >> 172.21.15.99:0/3348381388 conn(0x7fbbe80fc2f0 msgr2=0x7fbbe8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T00:38:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 -- 172.21.15.99:0/3348381388 shutdown_connections 2024-03-30T00:38:07.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-30T00:38:07.649+0000 7fbbeec00700 1 -- 172.21.15.99:0/3348381388 wait complete. 2024-03-30T00:38:07.657 INFO:teuthology.orchestra.run.smithi099.stderr:listed 10 entries 2024-03-30T00:38:07.674 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-03-30T00:38:07.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:38:07.675 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T00:38:07.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:07 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3965790047' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T00:38:07.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:07 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/91727395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T00:38:07.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:07 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/1406125561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T00:38:07.742 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T00:38:07.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[100990]: fuse finished with error 0 and tester_r 0 2024-03-30T00:38:08.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:08 smithi099 ceph-mon[38079]: pgmap v406: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:38:08.487 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:08 smithi099 ceph-mon[38079]: from='client.? 172.21.15.99:0/3348381388' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T00:38:08.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:08 smithi142 ceph-mon[41779]: pgmap v406: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:38:08.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:08 smithi142 ceph-mon[41779]: from='client.? 172.21.15.99:0/3348381388' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T00:38:08.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:08 smithi161 ceph-mon[42221]: pgmap v406: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-30T00:38:08.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:08 smithi161 ceph-mon[42221]: from='client.? 172.21.15.99:0/3348381388' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T00:38:10.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:10 smithi142 ceph-mon[41779]: pgmap v407: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-30T00:38:10.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:10 smithi161 ceph-mon[42221]: pgmap v407: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-30T00:38:10.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:10 smithi099 ceph-mon[38079]: pgmap v407: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-30T00:38:12.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:12 smithi142 ceph-mon[41779]: pgmap v408: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-30T00:38:12.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:12 smithi161 ceph-mon[42221]: pgmap v408: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-30T00:38:12.737 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:12 smithi099 ceph-mon[38079]: pgmap v408: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-30T00:38:13.844 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-30T00:38:13.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:38:13.844 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T00:38:13.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:38:13.909 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-30T00:38:13.953 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T00:38:13.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:38:13.954 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T00:38:14.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T00:38:14.062 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-03-30T00:38:14.163 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-30T00:38:14.174 INFO:tasks.cephadm:Teardown begin 2024-03-30T00:38:14.174 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/cephadm.py", line 1750, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-30T00:38:14.175 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T00:38:14.244 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T00:38:14.321 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T00:38:14.399 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-30T00:38:14.400 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-30T00:38:14.442 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-30T00:38:14.487 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-30T00:38:14.531 INFO:tasks.cephadm:Stopping all daemons... 2024-03-30T00:38:14.531 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-30T00:38:14.531 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.a 2024-03-30T00:38:14.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:14 smithi142 ceph-mon[41779]: pgmap v409: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-30T00:38:14.588 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:14 smithi099 ceph-mon[38079]: pgmap v409: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-30T00:38:14.719 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:14 smithi161 ceph-mon[42221]: pgmap v409: 97 pgs: 97 active+clean; 994 KiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-30T00:38:14.888 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:14 smithi099 systemd[1]: Stopping Ceph mon.a for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:38:15.237 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:14 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-a[38056]: 2024-03-30T00:38:14.881+0000 7f14ff200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:38:15.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:14 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-a[38056]: 2024-03-30T00:38:14.881+0000 7f14ff200700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-30T00:38:15.241 INFO:journalctl@ceph.mon.a.smithi099.stdout:Mar 30 00:38:15 smithi099 bash[107757]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-a 2024-03-30T00:38:15.475 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.a.service' 2024-03-30T00:38:15.575 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:38:15.576 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-30T00:38:15.576 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-30T00:38:15.576 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.b 2024-03-30T00:38:15.932 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:15 smithi142 systemd[1]: Stopping Ceph mon.b for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:38:16.196 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-b[41755]: 2024-03-30T00:38:15.923+0000 7fb09b000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:38:16.197 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-b[41755]: 2024-03-30T00:38:15.923+0000 7fb09b000700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-30T00:38:16.566 INFO:journalctl@ceph.mon.b.smithi142.stdout:Mar 30 00:38:16 smithi142 bash[81103]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-b 2024-03-30T00:38:16.624 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.b.service' 2024-03-30T00:38:16.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:38:16.722 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-30T00:38:16.722 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-30T00:38:16.723 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.c 2024-03-30T00:38:17.105 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:16 smithi161 systemd[1]: Stopping Ceph mon.c for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:38:17.355 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:17 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-c[42186]: 2024-03-30T00:38:17.096+0000 7f5b8fa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:38:17.355 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:17 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-c[42186]: 2024-03-30T00:38:17.096+0000 7f5b8fa00700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-30T00:38:17.718 INFO:journalctl@ceph.mon.c.smithi161.stdout:Mar 30 00:38:17 smithi161 bash[83115]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mon-c 2024-03-30T00:38:17.844 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mon.c.service' 2024-03-30T00:38:17.949 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:38:17.950 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-30T00:38:17.950 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-30T00:38:17.950 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.x 2024-03-30T00:38:18.331 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:38:18 smithi099 systemd[1]: Stopping Ceph mgr.x for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:38:18.331 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:38:18 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x[38332]: 2024-03-30T00:38:18.321+0000 7f2ea6a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:38:18.580 INFO:journalctl@ceph.mgr.x.smithi099.stdout:Mar 30 00:38:18 smithi099 bash[107904]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-x 2024-03-30T00:38:18.980 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.x.service' 2024-03-30T00:38:19.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:38:19.065 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-30T00:38:19.065 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-30T00:38:19.066 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.y 2024-03-30T00:38:19.566 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:38:19 smithi142 systemd[1]: Stopping Ceph mgr.y for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:38:20.066 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:38:19 smithi142 bash[81262]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-y 2024-03-30T00:38:20.066 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Mar 30 00:38:19 smithi142 systemd[1]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-30T00:38:20.107 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.y.service' 2024-03-30T00:38:20.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:38:20.192 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-30T00:38:20.192 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-30T00:38:20.192 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.z 2024-03-30T00:38:20.719 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:38:20 smithi161 systemd[1]: Stopping Ceph mgr.z for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:38:21.039 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:38:20 smithi161 bash[83266]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mgr-z 2024-03-30T00:38:21.308 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Mar 30 00:38:21 smithi161 systemd[1]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-30T00:38:21.330 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mgr.z.service' 2024-03-30T00:38:21.415 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:38:21.415 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-30T00:38:21.415 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-30T00:38:21.415 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.0 2024-03-30T00:38:21.737 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:38:21 smithi099 systemd[1]: Stopping Ceph osd.0 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:38:22.237 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:38:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-0[55748]: 2024-03-30T00:38:21.795+0000 7fee5da00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:38:22.237 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:38:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-0[55748]: 2024-03-30T00:38:21.795+0000 7fee5da00700 -1 osd.0 87 *** Got signal Terminated *** 2024-03-30T00:38:22.237 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:38:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-0[55748]: 2024-03-30T00:38:21.795+0000 7fee5da00700 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:38:27.737 INFO:journalctl@ceph.osd.0.smithi099.stdout:Mar 30 00:38:27 smithi099 bash[108043]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-0 2024-03-30T00:38:28.912 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.0.service' 2024-03-30T00:38:28.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:38:28.999 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-30T00:38:28.999 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-30T00:38:29.000 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.1 2024-03-30T00:38:29.316 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:38:29 smithi142 systemd[1]: Stopping Ceph osd.1 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:38:29.816 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:38:29 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-1[46863]: 2024-03-30T00:38:29.372+0000 7f3e73000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:38:29.816 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:38:29 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-1[46863]: 2024-03-30T00:38:29.372+0000 7f3e73000700 -1 osd.1 87 *** Got signal Terminated *** 2024-03-30T00:38:29.816 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:38:29 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-1[46863]: 2024-03-30T00:38:29.372+0000 7f3e73000700 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:38:35.316 INFO:journalctl@ceph.osd.1.smithi142.stdout:Mar 30 00:38:35 smithi142 bash[81423]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-1 2024-03-30T00:38:36.550 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.1.service' 2024-03-30T00:38:36.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:38:36.639 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-30T00:38:36.639 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-30T00:38:36.639 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.2 2024-03-30T00:38:36.969 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:38:36 smithi161 systemd[1]: Stopping Ceph osd.2 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:38:37.469 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:38:37 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-2[47835]: 2024-03-30T00:38:37.002+0000 7fd1ccc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:38:37.469 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:38:37 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-2[47835]: 2024-03-30T00:38:37.002+0000 7fd1ccc00700 -1 osd.2 87 *** Got signal Terminated *** 2024-03-30T00:38:37.469 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:38:37 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-2[47835]: 2024-03-30T00:38:37.002+0000 7fd1ccc00700 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:38:42.719 INFO:journalctl@ceph.osd.2.smithi161.stdout:Mar 30 00:38:42 smithi161 bash[83422]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-2 2024-03-30T00:38:44.075 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.2.service' 2024-03-30T00:38:44.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:38:44.163 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-30T00:38:44.163 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-30T00:38:44.163 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.3 2024-03-30T00:38:44.487 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 30 00:38:44 smithi099 systemd[1]: Stopping Ceph osd.3 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:38:44.987 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 30 00:38:44 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-3[64120]: 2024-03-30T00:38:44.532+0000 7f7ea5200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:38:44.987 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 30 00:38:44 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-3[64120]: 2024-03-30T00:38:44.532+0000 7f7ea5200700 -1 osd.3 87 *** Got signal Terminated *** 2024-03-30T00:38:44.987 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 30 00:38:44 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-3[64120]: 2024-03-30T00:38:44.532+0000 7f7ea5200700 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:38:48.916 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:48 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:48.479+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:49.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:48 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:48.910+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:38:49.237 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 30 00:38:48 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-3[64120]: 2024-03-30T00:38:48.736+0000 7f7ea1400700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.186470+0000 front 2024-03-30T00:38:25.186363+0000 (oldest deadline 2024-03-30T00:38:48.086487+0000) 2024-03-30T00:38:49.737 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:49 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:49.294+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:38:49.884 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:49 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:49.474+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:50.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:49 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:49.878+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:38:50.487 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 30 00:38:50 smithi099 bash[108319]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-3 2024-03-30T00:38:50.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:50 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:50.284+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:38:50.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:50 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:50.426+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:51.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:50 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:50.841+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:38:51.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:50 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:50.974+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:38:51.316 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:38:51 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:38:51.145+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:38:51.646 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:51 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:51.308+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:38:51.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:51 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:51.388+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:51.816 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:51 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:51.507+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:38:51.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.3.service' 2024-03-30T00:38:51.896 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 30 00:38:51 smithi099 systemd[1]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.3.service: Succeeded. 2024-03-30T00:38:51.896 INFO:journalctl@ceph.osd.3.smithi099.stdout:Mar 30 00:38:51 smithi099 systemd[1]: Stopped Ceph osd.3 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:38:51.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:38:51.921 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-30T00:38:51.921 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-30T00:38:51.921 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.4 2024-03-30T00:38:52.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:51 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:51.885+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:38:52.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:52 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:52.004+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:38:52.299 INFO:journalctl@ceph.osd.4.smithi142.stdout:Mar 30 00:38:52 smithi142 systemd[1]: Stopping Ceph osd.4 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:38:52.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:38:52 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:38:52.179+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:38:52.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:52 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:52.277+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:38:52.566 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:52 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:52.535+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:38:52.566 INFO:journalctl@ceph.osd.4.smithi142.stdout:Mar 30 00:38:52 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-4[54766]: 2024-03-30T00:38:52.291+0000 7fc7c7400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:38:52.567 INFO:journalctl@ceph.osd.4.smithi142.stdout:Mar 30 00:38:52 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-4[54766]: 2024-03-30T00:38:52.291+0000 7fc7c7400700 -1 osd.4 87 *** Got signal Terminated *** 2024-03-30T00:38:52.567 INFO:journalctl@ceph.osd.4.smithi142.stdout:Mar 30 00:38:52 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-4[54766]: 2024-03-30T00:38:52.291+0000 7fc7c7400700 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:38:52.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:52 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:52.382+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:53.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:52 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:52.845+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:38:53.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:52 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:52.995+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:38:53.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:38:53 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:38:53.161+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:38:53.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:53 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:53.233+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:38:53.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:53 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:53.407+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:53.816 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:53 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:53.519+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:38:54.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:53 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:53.879+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:38:54.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:54 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:54.020+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:38:54.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:38:54 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:38:54.175+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:38:54.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:54 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:54.254+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:38:54.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:54 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:54.370+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:54.816 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:54 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:54.519+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:38:55.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:54 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:54.843+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:38:55.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:54 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:54.991+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:38:55.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:38:55 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:38:55.213+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:38:55.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:55 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:55.263+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:38:55.563 INFO:journalctl@ceph.osd.4.smithi142.stdout:Mar 30 00:38:55 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-4[54766]: 2024-03-30T00:38:55.160+0000 7fc7be600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.196227+0000 front 2024-03-30T00:38:33.196269+0000 (oldest deadline 2024-03-30T00:38:54.896417+0000) 2024-03-30T00:38:55.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:55 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:55.364+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:55.816 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:55 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:55.556+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:38:56.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:55 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:55.822+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:38:56.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:55 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:55.950+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:38:56.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:38:56 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:38:56.209+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:38:56.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:56 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:56.271+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:38:56.566 INFO:journalctl@ceph.osd.4.smithi142.stdout:Mar 30 00:38:56 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-4[54766]: 2024-03-30T00:38:56.158+0000 7fc7be600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.196227+0000 front 2024-03-30T00:38:33.196269+0000 (oldest deadline 2024-03-30T00:38:54.896417+0000) 2024-03-30T00:38:56.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:56 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:56.410+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:56.719 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:56 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:56.410+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.504028+0000 front 2024-03-30T00:38:33.504206+0000 (oldest deadline 2024-03-30T00:38:56.305052+0000) 2024-03-30T00:38:57.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:56 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:56.591+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:38:57.069 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:38:56 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:38:56.665+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:38:57.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:56 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:56.818+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:38:57.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:56 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:56.917+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:38:57.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:38:57 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:38:57.182+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:38:57.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:57 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:57.225+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:38:57.566 INFO:journalctl@ceph.osd.4.smithi142.stdout:Mar 30 00:38:57 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-4[54766]: 2024-03-30T00:38:57.157+0000 7fc7be600700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.196227+0000 front 2024-03-30T00:38:33.196269+0000 (oldest deadline 2024-03-30T00:38:54.896417+0000) 2024-03-30T00:38:57.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:57 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:57.450+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:57.719 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:57 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:57.450+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.504028+0000 front 2024-03-30T00:38:33.504206+0000 (oldest deadline 2024-03-30T00:38:56.305052+0000) 2024-03-30T00:38:57.907 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:38:57 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:38:57.666+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:38:57.908 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:57 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:57.622+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:38:57.908 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:57 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:57.622+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:38:58.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:57 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:57.854+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:38:58.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:57 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:57.956+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:38:58.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:57 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:57.956+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:38:58.316 INFO:journalctl@ceph.osd.4.smithi142.stdout:Mar 30 00:38:57 smithi142 bash[81682]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-4 2024-03-30T00:38:58.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:38:58 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:38:58.192+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:38:58.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:58 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:58.233+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:38:58.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:58 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:58.233+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:38:58.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:58 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:58.446+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:58.719 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:58 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:58.446+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.504028+0000 front 2024-03-30T00:38:33.504206+0000 (oldest deadline 2024-03-30T00:38:56.305052+0000) 2024-03-30T00:38:59.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:38:58 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:38:58.651+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:38:59.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:58 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:58.585+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:38:59.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:58 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:58.585+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:38:59.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:58 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:58.805+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:38:59.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:58 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:58.986+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:38:59.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:58 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:58.986+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:38:59.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:59 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:59.194+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:38:59.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:38:59 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:38:59.194+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:38:59.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:38:59 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:38:59.200+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:38:59.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:38:59 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:38:59.200+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:38:59.574 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.4.service' 2024-03-30T00:38:59.660 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:38:59.661 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-30T00:38:59.661 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-30T00:38:59.661 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.5 2024-03-30T00:38:59.726 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:59 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:59.465+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:38:59.726 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:59 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:38:59.465+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.504028+0000 front 2024-03-30T00:38:33.504206+0000 (oldest deadline 2024-03-30T00:38:56.305052+0000) 2024-03-30T00:39:00.046 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:59 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:59.829+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:00.047 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:38:59 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:38:59.829+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:00.047 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:59 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:59.962+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:00.047 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:38:59 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:38:59.962+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:00.047 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:38:59 smithi161 systemd[1]: Stopping Ceph osd.5 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:39:00.047 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:00 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:00.040+0000 7f8c4ea00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:39:00.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:38:59 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:38:59.692+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:00.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:59 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:59.620+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:00.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:38:59 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:38:59.620+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:00.425 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:00 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:00.040+0000 7f8c4ea00700 -1 osd.5 87 *** Got signal Terminated *** 2024-03-30T00:39:00.426 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:00 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:00.040+0000 7f8c4ea00700 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:39:00.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:00 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:00.242+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:00.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:00 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:00.242+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:00.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:00 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:00.146+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:00.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:00 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:00.146+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:00.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:00 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:00.419+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:39:00.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:00 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:00.419+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.504028+0000 front 2024-03-30T00:38:33.504206+0000 (oldest deadline 2024-03-30T00:38:56.305052+0000) 2024-03-30T00:39:01.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:00 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:00.690+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:01.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:00 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:00.602+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:01.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:00 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:00.602+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:01.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:00 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:00.850+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:01.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:00 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:00.850+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:01.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:00 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:00.965+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:01.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:00 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:00.965+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:01.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:01 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:01.223+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:01.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:01 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:01.224+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:01.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:01 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:01.113+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:01.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:01 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:01.113+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:01.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:01 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:01.375+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:39:01.719 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:01 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:01.375+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.504028+0000 front 2024-03-30T00:38:33.504206+0000 (oldest deadline 2024-03-30T00:38:56.305052+0000) 2024-03-30T00:39:02.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:01 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:01.708+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:02.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:01 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:01.647+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:02.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:01 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:01.647+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:02.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:01 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:01.845+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:02.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:01 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:01.845+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:02.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:01 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:01.952+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:02.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:01 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:01.952+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:02.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:02.206+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:02.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:02.206+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:02.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:02.147+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:02.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:02 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:02.147+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:02.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:02 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:02.337+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:39:02.719 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:02 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:02.337+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.504028+0000 front 2024-03-30T00:38:33.504206+0000 (oldest deadline 2024-03-30T00:38:56.305052+0000) 2024-03-30T00:39:03.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:02 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:02.693+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:03.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:02 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:02.670+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:03.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:02 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:02.670+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:03.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:02 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:02.670+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:03.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:02 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:02.885+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:03.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:02 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:02.885+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:03.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:02 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:02.942+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:03.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:02 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:02.942+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:03.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:03 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:03.183+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:03.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:03 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:03.183+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:03.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:03 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:03.153+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:03.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:03 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:03.153+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:03.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:03 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:03.153+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:03.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:03 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:03.350+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:39:03.719 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:03 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:03.350+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.504028+0000 front 2024-03-30T00:38:33.504206+0000 (oldest deadline 2024-03-30T00:38:56.305052+0000) 2024-03-30T00:39:04.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:03 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:03.700+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:04.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:03 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:03.690+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:04.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:03 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:03.690+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:04.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:03 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:03.690+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:04.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:03 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:03.849+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:04.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:03 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:03.849+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:04.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:03 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:03.969+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:04.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:03 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:03.969+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:04.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:04 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:04.166+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:04.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:04 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:04.166+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:04.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:04 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:04.166+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:04.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:04 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:04.211+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:04.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:04 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:04.211+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:04.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:04 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:04.211+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.096717+0000 front 2024-03-30T00:38:39.096780+0000 (oldest deadline 2024-03-30T00:39:03.196628+0000) 2024-03-30T00:39:04.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:04 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:04.302+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.701181+0000 front 2024-03-30T00:38:26.701214+0000 (oldest deadline 2024-03-30T00:38:47.801308+0000) 2024-03-30T00:39:04.718 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:04 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5[56485]: 2024-03-30T00:39:04.302+0000 7f8c45c00700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.504028+0000 front 2024-03-30T00:38:33.504206+0000 (oldest deadline 2024-03-30T00:38:56.305052+0000) 2024-03-30T00:39:05.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:04 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:04.672+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:05.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:04 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:04.701+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:05.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:04 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:04.701+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:05.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:04 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:04.701+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:05.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:04 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:04.856+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:05.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:04 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:04.856+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:05.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:04 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:04.856+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:05.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:04 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:04.926+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:05.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:04 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:04.926+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:05.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:04 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:04.926+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:05.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:05 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:05.209+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:05.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:05 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:05.209+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:05.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:05 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:05.209+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:05.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:05 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:05.198+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:05.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:05 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:05.198+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:05.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:05 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:05.198+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.096717+0000 front 2024-03-30T00:38:39.096780+0000 (oldest deadline 2024-03-30T00:39:03.196628+0000) 2024-03-30T00:39:06.067 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:05 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:05.623+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:06.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:05 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:05.710+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:06.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:05 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:05.710+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:06.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:05 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:05.710+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:06.219 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:05 smithi161 bash[83676]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-5 2024-03-30T00:39:06.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:05 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:05.847+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:06.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:05 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:05.847+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:06.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:05 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:05.847+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:06.220 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:05 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:05.941+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:06.220 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:05 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:05.941+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:06.220 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:05 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:05.941+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:06.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:06 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:06.211+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:06.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:06 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:06.211+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:06.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:06 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:06.211+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.096717+0000 front 2024-03-30T00:38:39.096780+0000 (oldest deadline 2024-03-30T00:39:03.196628+0000) 2024-03-30T00:39:06.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:06 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:06.227+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:06.488 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:06 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:06.227+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:06.488 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:06 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:06.227+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:07.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:06 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:06.596+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:07.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:06 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:06.754+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:07.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:06 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:06.754+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:07.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:06 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:06.754+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:07.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:06 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:06.833+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:07.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:06 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:06.833+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:07.138 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:06 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:06.833+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:07.138 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:06 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:06.921+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:07.138 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:06 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:06.921+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:07.139 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:06 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:06.921+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:07.368 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.5.service' 2024-03-30T00:39:07.428 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:07 smithi161 systemd[1]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.5.service: Succeeded. 2024-03-30T00:39:07.428 INFO:journalctl@ceph.osd.5.smithi161.stdout:Mar 30 00:39:07 smithi161 systemd[1]: Stopped Ceph osd.5 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4. 2024-03-30T00:39:07.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:39:07.455 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-30T00:39:07.455 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-30T00:39:07.456 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.6 2024-03-30T00:39:07.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:07 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:07.181+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:07.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:07 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:07.181+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:07.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:07 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:07.181+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.096717+0000 front 2024-03-30T00:38:39.096780+0000 (oldest deadline 2024-03-30T00:39:03.196628+0000) 2024-03-30T00:39:07.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:07 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:07.261+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:07.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:07 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:07.261+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:07.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:07 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:07.261+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:07.854 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:07 smithi099 systemd[1]: Stopping Ceph osd.6 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:39:08.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:07 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:07.638+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:08.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:07 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:07.638+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:08.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:07 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:07.779+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:08.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:07 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:07.779+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:08.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:07 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:07.779+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:08.190 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:07 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:07.847+0000 7f9ba7600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:39:08.190 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:07 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:07.847+0000 7f9ba7600700 -1 osd.6 87 *** Got signal Terminated *** 2024-03-30T00:39:08.191 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:07 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:07.847+0000 7f9ba7600700 -1 osd.6 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:39:08.191 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:08 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:08.184+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:08.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:07 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:07.797+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:08.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:07 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:07.797+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:08.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:07 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:07.797+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:08.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:07 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:07.877+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:08.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:07 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:07.877+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:08.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:07 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:07.877+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:08.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:08 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:08.184+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:08.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:08 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:08.184+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.096717+0000 front 2024-03-30T00:38:39.096780+0000 (oldest deadline 2024-03-30T00:39:03.196628+0000) 2024-03-30T00:39:08.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:08 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:08.214+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:08.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:08 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:08.214+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:08.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:08 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:08.214+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:09.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:08 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:08.607+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:09.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:08 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:08.607+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:09.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:08 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:08.825+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:09.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:08 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:08.825+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:09.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:08 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:08.825+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:09.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:08 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:08.765+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:09.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:08 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:08.765+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:09.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:08 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:08.765+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:09.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:08 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:08.851+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:09.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:08 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:08.851+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:09.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:08 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:08.851+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:09.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:09 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:09.187+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:09.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:09 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:09.187+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:09.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:09 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:09.187+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.096717+0000 front 2024-03-30T00:38:39.096780+0000 (oldest deadline 2024-03-30T00:39:03.196628+0000) 2024-03-30T00:39:09.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:09 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:09.174+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:09.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:09 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:09.174+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:09.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:09 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:09.174+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:10.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:09 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:09.633+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:10.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:09 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:09.633+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:10.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:09 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:09.867+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:10.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:09 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:09.867+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:10.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:09 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:09.867+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:10.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:09 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:09.760+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:10.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:09 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:09.760+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:10.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:09 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:09.760+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:10.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:09 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:09.868+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:10.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:09 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:09.868+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:10.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:09 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:09.868+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:10.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:10 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:10.225+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:10.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:10 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:10.225+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:10.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:10 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:10.225+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.096717+0000 front 2024-03-30T00:38:39.096780+0000 (oldest deadline 2024-03-30T00:39:03.196628+0000) 2024-03-30T00:39:10.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:10 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:10.134+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:10.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:10 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:10.134+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:10.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:10 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:10.134+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:11.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:10 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:10.666+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:11.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:10 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:10.666+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:11.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:10 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:10.839+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:11.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:10 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:10.839+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:11.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:10 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:10.839+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:11.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:10 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:10.877+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:11.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:10 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:10.877+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:11.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:10 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:10.877+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:11.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:10 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:10.772+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:11.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:10 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:10.772+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:11.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:10 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:10.772+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:11.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:11 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:11.258+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:11.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:11 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:11.258+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:11.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:11 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:11.258+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.096717+0000 front 2024-03-30T00:38:39.096780+0000 (oldest deadline 2024-03-30T00:39:03.196628+0000) 2024-03-30T00:39:11.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:11 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:11.159+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:11.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:11 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:11.159+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:11.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:11 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:11.159+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:12.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:11 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:11.636+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:12.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:11 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:11.636+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:12.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:11 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:11.857+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:12.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:11 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:11.857+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:12.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:11 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:11.857+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:12.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:11 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:11.783+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:12.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:11 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:11.783+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:12.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:11 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:11.783+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:12.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:11 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:11.873+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:12.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:11 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:11.873+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:12.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:11 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:11.873+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:12.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:12 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:12.274+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:25.694019+0000 front 2024-03-30T00:38:25.693997+0000 (oldest deadline 2024-03-30T00:38:50.994286+0000) 2024-03-30T00:39:12.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:12 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:12.274+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:32.695749+0000 front 2024-03-30T00:38:32.695806+0000 (oldest deadline 2024-03-30T00:38:59.096211+0000) 2024-03-30T00:39:12.487 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:12 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6[72166]: 2024-03-30T00:39:12.274+0000 7f9b9e800700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.096717+0000 front 2024-03-30T00:38:39.096780+0000 (oldest deadline 2024-03-30T00:39:03.196628+0000) 2024-03-30T00:39:12.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:12 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:12.120+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:12.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:12 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:12.120+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:12.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:12 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:12.120+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:12.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:12 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:12.120+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:12.902 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:12 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:12.590+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:12.902 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:12 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:12.590+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:12.902 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:12 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:12.590+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:47.213549+0000 front 2024-03-30T00:38:47.213482+0000 (oldest deadline 2024-03-30T00:39:12.513493+0000) 2024-03-30T00:39:13.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:12 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:12.806+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:13.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:12 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:12.806+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:13.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:12 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:12.806+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:13.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:12 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:12.897+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:13.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:12 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:12.897+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:13.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:12 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:12.897+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:13.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:12 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:12.895+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:13.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:12 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:12.895+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:13.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:12 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:12.895+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:13.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:13 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:13.163+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:13.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:13 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:13.163+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:13.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:13 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:13.163+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:13.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:13 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:13.163+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:13.909 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:13 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:13.586+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:13.909 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:13 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:13.586+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:13.910 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:13 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:13.586+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:47.213549+0000 front 2024-03-30T00:38:47.213482+0000 (oldest deadline 2024-03-30T00:39:12.513493+0000) 2024-03-30T00:39:13.987 INFO:journalctl@ceph.osd.6.smithi099.stdout:Mar 30 00:39:13 smithi099 bash[108579]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-6 2024-03-30T00:39:14.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:13 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:13.784+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:14.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:13 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:13.784+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:14.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:13 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:13.784+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:14.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:13 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:13.929+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:14.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:13 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:13.929+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:14.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:13 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:13.929+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:14.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:13 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:13.903+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:14.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:13 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:13.903+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:14.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:13 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:13.903+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:14.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:14 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:14.188+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:14.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:14 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:14.188+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:14.488 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:14 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:14.188+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:14.488 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:14 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:14.188+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:14.925 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:14 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:14.614+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:14.925 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:14 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:14.614+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:14.925 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:14 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:14.614+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:47.213549+0000 front 2024-03-30T00:38:47.213482+0000 (oldest deadline 2024-03-30T00:39:12.513493+0000) 2024-03-30T00:39:15.022 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.6.service' 2024-03-30T00:39:15.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:39:15.151 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-30T00:39:15.152 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-30T00:39:15.152 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.7 2024-03-30T00:39:15.216 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:14 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:14.918+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:15.216 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:14 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:14.918+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:15.217 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:14 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:14.918+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:15.217 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:14 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:14.918+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:15.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:14 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:14.820+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:15.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:14 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:14.820+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:15.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:14 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:14.820+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:15.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:14 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:14.926+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:15.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:14 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:14.926+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:15.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:14 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:14.926+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:15.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:15 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:15.143+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:15.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:15 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:15.143+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:15.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:15 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:15.143+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:15.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:15 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:15.143+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:15.520 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:15 smithi142 systemd[1]: Stopping Ceph osd.7 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:39:15.816 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:15.513+0000 7f08c0800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:39:15.816 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:15.513+0000 7f08c0800700 -1 osd.7 87 *** Got signal Terminated *** 2024-03-30T00:39:15.816 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:15.513+0000 7f08c0800700 -1 osd.7 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:39:15.816 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:15.631+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:15.816 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:15.631+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:15.817 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:15.631+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:47.213549+0000 front 2024-03-30T00:38:47.213482+0000 (oldest deadline 2024-03-30T00:39:12.513493+0000) 2024-03-30T00:39:16.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:15 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:15.795+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:16.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:15 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:15.795+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:16.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:15 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:15.795+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:16.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:15 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:15.795+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:16.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:15 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:15.893+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:16.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:15 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:15.893+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:16.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:15 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:15.893+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:16.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:15.869+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:16.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:15.869+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:16.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:15.869+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:16.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:15 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:15.869+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:16.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:16 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:16.156+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:16.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:16 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:16.156+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:16.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:16 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:16.156+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:16.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:16 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:16.156+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:17.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:16 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:16.669+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:17.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:16 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:16.669+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:17.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:16 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:16.669+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:47.213549+0000 front 2024-03-30T00:38:47.213482+0000 (oldest deadline 2024-03-30T00:39:12.513493+0000) 2024-03-30T00:39:17.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:16 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:16.902+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:17.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:16 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:16.902+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:17.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:16 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:16.902+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:17.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:16 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:16.902+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:17.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:16 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:16.927+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:17.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:16 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:16.927+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:17.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:16 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:16.927+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:17.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:16 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:16.824+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:17.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:16 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:16.824+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:17.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:16 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:16.824+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:17.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:16 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:16.824+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:17.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:17 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:17.170+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:17.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:17 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:17.170+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:17.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:17 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:17.170+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:17.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:17 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:17.170+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:18.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:17 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:17.696+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:18.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:17 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:17.696+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:18.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:17 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:17.696+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:47.213549+0000 front 2024-03-30T00:38:47.213482+0000 (oldest deadline 2024-03-30T00:39:12.513493+0000) 2024-03-30T00:39:18.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:17 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:17.856+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:18.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:17 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:17.856+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:18.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:17 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:17.856+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:18.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:17 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:17.856+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:18.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:17 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:17.972+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:18.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:17 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:17.972+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:18.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:17 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:17.972+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:18.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:17 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:17.799+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:18.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:17 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:17.799+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:18.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:17 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:17.799+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:18.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:17 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:17.799+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:18.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:18 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:18.193+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:18.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:18 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:18.193+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:18.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:18 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:18.193+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:18.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:18 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:18.193+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:19.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:18.702+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:19.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:18.702+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:19.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:18.702+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:47.213549+0000 front 2024-03-30T00:38:47.213482+0000 (oldest deadline 2024-03-30T00:39:12.513493+0000) 2024-03-30T00:39:19.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:18.702+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.414482+0000 front 2024-03-30T00:38:55.414438+0000 (oldest deadline 2024-03-30T00:39:18.314568+0000) 2024-03-30T00:39:19.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:18.810+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:19.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:18.810+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:19.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:18.810+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:19.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:18 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:18.810+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:19.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:18 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:18.934+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:19.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:18 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:18.934+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:19.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:18 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:18.934+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:19.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:18 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:18.750+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:19.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:18 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:18.750+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:19.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:18 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:18.750+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:19.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:18 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:18.750+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:19.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:18 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:18.750+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.179670+0000 front 2024-03-30T00:38:55.179527+0000 (oldest deadline 2024-03-30T00:39:18.079618+0000) 2024-03-30T00:39:19.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:19 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:19.176+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:19.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:19 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:19.176+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:19.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:19 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:19.176+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:19.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:19 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:19.176+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:20.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:19 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:19.692+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.909603+0000 front 2024-03-30T00:38:33.909628+0000 (oldest deadline 2024-03-30T00:38:56.110626+0000) 2024-03-30T00:39:20.066 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:19 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:19.692+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:41.912804+0000 front 2024-03-30T00:38:41.912712+0000 (oldest deadline 2024-03-30T00:39:07.212782+0000) 2024-03-30T00:39:20.067 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:19 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:19.692+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:47.213549+0000 front 2024-03-30T00:38:47.213482+0000 (oldest deadline 2024-03-30T00:39:12.513493+0000) 2024-03-30T00:39:20.067 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:19 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7[62827]: 2024-03-30T00:39:19.692+0000 7f08bca00700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.414482+0000 front 2024-03-30T00:38:55.414438+0000 (oldest deadline 2024-03-30T00:39:18.314568+0000) 2024-03-30T00:39:20.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:19 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:19.812+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:20.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:19 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:19.812+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:20.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:19 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:19.812+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:20.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:19 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:19.812+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:20.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:19 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:19.776+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:20.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:19 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:19.776+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:20.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:19 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:19.776+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:20.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:19 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:19.776+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:20.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:19 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:19.776+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.179670+0000 front 2024-03-30T00:38:55.179527+0000 (oldest deadline 2024-03-30T00:39:18.079618+0000) 2024-03-30T00:39:20.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:19 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:19.968+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:20.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:19 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:19.968+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:20.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:19 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:19.968+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:20.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:19 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:19.968+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:20.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:20 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:20.197+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:20.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:20 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:20.197+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:20.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:20 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:20.197+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:20.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:20 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:20.197+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:21.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:20 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:20.770+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:21.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:20 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:20.770+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:21.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:20 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:20.770+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:21.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:20 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:20.770+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:21.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:20 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:20.770+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:21.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:20 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:20.825+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:21.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:20 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:20.825+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:21.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:20 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:20.825+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:21.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:20 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:20.825+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:21.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:20 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:20.825+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.179670+0000 front 2024-03-30T00:38:55.179527+0000 (oldest deadline 2024-03-30T00:39:18.079618+0000) 2024-03-30T00:39:21.220 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:20 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:20.950+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:21.220 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:20 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:20.950+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:21.220 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:20 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:20.950+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:21.220 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:20 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:20.950+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:21.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:21.187+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:21.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:21.187+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:21.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:21.187+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:21.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:21 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:21.187+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:21.566 INFO:journalctl@ceph.osd.7.smithi142.stdout:Mar 30 00:39:21 smithi142 bash[81939]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-7 2024-03-30T00:39:22.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:21 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:21.804+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:22.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:21 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:21.804+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:22.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:21 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:21.804+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:22.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:21 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:21.804+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:22.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:21 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:21.804+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:22.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:21 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:21.943+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:22.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:21 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:21.943+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:22.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:21 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:21.943+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:22.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:21 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:21.943+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:22.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:21 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:21.867+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:22.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:21 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:21.867+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:22.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:21 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:21.867+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:22.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:21 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:21.867+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:22.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:21 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:21.867+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.179670+0000 front 2024-03-30T00:38:55.179527+0000 (oldest deadline 2024-03-30T00:39:18.079618+0000) 2024-03-30T00:39:22.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:22 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:22.140+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:22.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:22 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:22.140+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:22.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:22 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:22.140+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:22.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:22 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:22.140+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:22.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:22 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:22.140+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:22.747 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.7.service' 2024-03-30T00:39:22.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:39:22.833 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-30T00:39:22.833 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-30T00:39:22.833 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.8 2024-03-30T00:39:23.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:22 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:22.836+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:23.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:22 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:22.836+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:23.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:22 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:22.836+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:23.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:22 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:22.836+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:23.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:22 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:22.836+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:23.202 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:22 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:22.905+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:23.202 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:22 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:22.905+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:23.202 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:22 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:22.905+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:23.202 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:22 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:22.905+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:23.202 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:22 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:22.905+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.179670+0000 front 2024-03-30T00:38:55.179527+0000 (oldest deadline 2024-03-30T00:39:18.079618+0000) 2024-03-30T00:39:23.202 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:22 smithi161 systemd[1]: Stopping Ceph osd.8 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:39:23.202 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:22 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:22.960+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:23.202 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:22 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:22.960+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:23.202 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:22 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:22.960+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:23.202 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:22 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:22.960+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:23.468 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:23.195+0000 7ff836a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:39:23.469 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:23.195+0000 7ff836a00700 -1 osd.8 87 *** Got signal Terminated *** 2024-03-30T00:39:23.469 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:23.195+0000 7ff836a00700 -1 osd.8 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:39:23.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:23 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:23.182+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:23.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:23 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:23.182+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:23.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:23 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:23.182+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:23.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:23 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:23.182+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:23.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:23 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:23.182+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:24.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:23.932+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:24.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:23.932+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:24.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:23.932+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:24.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:23.932+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:24.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:23.902+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:24.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:23.902+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:24.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:23.902+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:24.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:23.902+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:24.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:23 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:23.902+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.179670+0000 front 2024-03-30T00:38:55.179527+0000 (oldest deadline 2024-03-30T00:39:18.079618+0000) 2024-03-30T00:39:24.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:23 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:23.813+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:24.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:23 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:23.813+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:24.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:23 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:23.813+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:24.317 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:23 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:23.813+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:24.317 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:23 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:23.813+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:24.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:24 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:24.140+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:24.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:24 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:24.140+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:24.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:24 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:24.140+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:24.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:24 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:24.140+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:24.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:24 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:24.140+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:25.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:24 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:24.903+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:25.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:24 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:24.903+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:25.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:24 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:24.903+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:25.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:24 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:24.903+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:25.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:24 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:24.871+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:25.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:24 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:24.871+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:25.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:24 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:24.871+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:25.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:24 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:24.871+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:25.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:24 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:24.871+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.179670+0000 front 2024-03-30T00:38:55.179527+0000 (oldest deadline 2024-03-30T00:39:18.079618+0000) 2024-03-30T00:39:25.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:24 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:24.855+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:25.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:24 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:24.855+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:25.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:24 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:24.855+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:25.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:24 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:24.855+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:25.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:24 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:24.855+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:25.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:25 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:25.157+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:25.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:25 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:25.157+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:25.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:25 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:25.157+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:25.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:25 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:25.157+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:25.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:25 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:25.157+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:26.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:25 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:25.910+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:26.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:25 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:25.910+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:26.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:25 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:25.910+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:26.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:25 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:25.910+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:26.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:25 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:25.864+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:26.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:25 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:25.864+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:26.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:25 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:25.864+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:26.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:25 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:25.864+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:26.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:25 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:25.864+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.179670+0000 front 2024-03-30T00:38:55.179527+0000 (oldest deadline 2024-03-30T00:39:18.079618+0000) 2024-03-30T00:39:26.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:25 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:25.869+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:26.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:25 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:25.869+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:26.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:25 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:25.869+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:26.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:25 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:25.869+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:26.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:25 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:25.869+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:26.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:26 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:26.114+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:26.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:26 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:26.114+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:26.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:26 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:26.114+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:26.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:26 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:26.114+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:26.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:26 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:26.114+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:27.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:26 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:26.818+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:27.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:26 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:26.818+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:27.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:26 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:26.818+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:27.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:26 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:26.818+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:27.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:26 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:26.818+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.179670+0000 front 2024-03-30T00:38:55.179527+0000 (oldest deadline 2024-03-30T00:39:18.079618+0000) 2024-03-30T00:39:27.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:26 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:26.884+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:27.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:26 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:26.884+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:27.220 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:26 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:26.884+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:27.220 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:26 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:26.884+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:27.220 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:26 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:26.884+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:27.317 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:26 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:26.857+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:27.317 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:26 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:26.857+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:27.317 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:26 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:26.857+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:27.317 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:26 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:26.857+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:27.317 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:26 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:26.857+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:27.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:27 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:27.155+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:27.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:27 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:27.155+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:27.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:27 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:27.155+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:27.488 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:27 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:27.155+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:27.488 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:27 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:27.155+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:28.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:27 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:27.789+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:22.974665+0000 front 2024-03-30T00:38:22.974513+0000 (oldest deadline 2024-03-30T00:38:48.274517+0000) 2024-03-30T00:39:28.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:27 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:27.789+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:34.076680+0000 front 2024-03-30T00:38:34.076659+0000 (oldest deadline 2024-03-30T00:38:59.277215+0000) 2024-03-30T00:39:28.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:27 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:27.789+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.277675+0000 front 2024-03-30T00:38:39.277704+0000 (oldest deadline 2024-03-30T00:39:04.577943+0000) 2024-03-30T00:39:28.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:27 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:27.789+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.279254+0000 front 2024-03-30T00:38:49.279488+0000 (oldest deadline 2024-03-30T00:39:15.179086+0000) 2024-03-30T00:39:28.219 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:27 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8[64388]: 2024-03-30T00:39:27.789+0000 7ff832c00700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:55.179670+0000 front 2024-03-30T00:38:55.179527+0000 (oldest deadline 2024-03-30T00:39:18.079618+0000) 2024-03-30T00:39:28.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:27 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:27.932+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:28.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:27 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:27.932+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:28.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:27 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:27.932+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:28.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:27 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:27.932+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:28.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:27 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:27.932+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:28.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:27 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:27.856+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:28.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:27 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:27.856+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:28.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:27 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:27.856+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:28.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:27 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:27.856+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:28.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:27 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:27.856+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:28.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:27 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:27.856+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:28.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:28 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:28.148+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:28.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:28 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:28.148+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:28.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:28 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:28.148+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:28.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:28 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:28.148+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:28.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:28 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:28.148+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:29.218 INFO:journalctl@ceph.osd.8.smithi161.stdout:Mar 30 00:39:28 smithi161 bash[83935]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-8 2024-03-30T00:39:29.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:28 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:28.886+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:29.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:28 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:28.886+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:29.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:28 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:28.886+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:29.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:28 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:28.886+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:29.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:28 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:28.886+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:29.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:28 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:28.836+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:29.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:28 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:28.836+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:29.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:28 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:28.836+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:29.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:28 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:28.836+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:29.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:28 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:28.836+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:29.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:28 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:28.836+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:29.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:29 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:29.176+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:29.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:29 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:29.176+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:29.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:29 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:29.176+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:29.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:29 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:29.176+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:29.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:29 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:29.176+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:30.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:29 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:29.933+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:30.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:29 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:29.933+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:30.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:29 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:29.933+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:30.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:29 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:29.933+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:30.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:29 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:29.933+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:30.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:29 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:29.843+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:30.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:29 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:29.843+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:30.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:29 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:29.843+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:30.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:29 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:29.843+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:30.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:29 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:29.843+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:30.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:29 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:29.843+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:30.367 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.8.service' 2024-03-30T00:39:30.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:39:30.452 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-30T00:39:30.452 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-30T00:39:30.452 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.9 2024-03-30T00:39:30.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:30 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:30.144+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:30.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:30 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:30.144+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:30.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:30 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:30.144+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:30.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:30 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:30.144+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:30.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:30 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:30.144+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:30.820 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:30 smithi099 systemd[1]: Stopping Ceph osd.9 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:39:31.130 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:30 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:30.813+0000 7f3f0a000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:39:31.130 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:30 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:30.813+0000 7f3f0a000700 -1 osd.9 87 *** Got signal Terminated *** 2024-03-30T00:39:31.130 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:30 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:30.813+0000 7f3f0a000700 -1 osd.9 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:39:31.130 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:31 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:31.124+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:31.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:30 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:30.954+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:31.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:30 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:30.954+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:31.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:30 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:30.954+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:31.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:30 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:30.954+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:31.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:30 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:30.954+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:31.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:30 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:30.878+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:31.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:30 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:30.878+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:31.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:30 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:30.878+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:31.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:30 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:30.878+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:31.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:30 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:30.878+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:31.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:30 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:30.878+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:31.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:31 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:31.124+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:31.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:31 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:31.124+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:31.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:31 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:31.124+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:31.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:31 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:31.124+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:32.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:31 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:31.881+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:32.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:31 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:31.881+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:32.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:31 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:31.881+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:32.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:31 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:31.881+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:32.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:31 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:31.881+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:32.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:31 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:31.881+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:32.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:31 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:31.971+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:32.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:31 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:31.971+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:32.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:31 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:31.971+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:32.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:31 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:31.971+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:32.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:31 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:31.971+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:32.486 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:32 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:32.109+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:32.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:32 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:32.109+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:32.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:32 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:32.109+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:32.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:32 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:32.109+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:32.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:32 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:32.109+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:33.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:32 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:32.934+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:33.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:32 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:32.934+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:33.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:32 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:32.934+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:33.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:32 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:32.934+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:33.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:32 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:32.934+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:33.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:32 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:32.849+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:33.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:32 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:32.849+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:33.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:32 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:32.849+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:33.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:32 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:32.849+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:33.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:32 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:32.849+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:33.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:32 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:32.849+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:33.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:33 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:33.109+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:33.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:33 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:33.109+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:33.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:33 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:33.109+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:33.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:33 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:33.109+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:33.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:33 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:33.109+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:34.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:33 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:33.935+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:34.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:33 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:33.935+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:34.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:33 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:33.935+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:34.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:33 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:33.935+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:34.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:33 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:33.935+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:34.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:33 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:33.841+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:34.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:33 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:33.841+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:34.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:33 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:33.841+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:34.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:33 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:33.841+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:34.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:33 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:33.841+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:34.317 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:33 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:33.841+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:34.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:34 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:34.112+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:34.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:34 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:34.112+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:34.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:34 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:34.112+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:34.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:34 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:34.112+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:34.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:34 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:34.112+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:35.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:34 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:34.941+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:35.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:34 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:34.941+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:35.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:34 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:34.941+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:35.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:34 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:34.941+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:35.219 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:34 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:34.941+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:35.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:34 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:34.839+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:35.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:34 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:34.839+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:35.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:34 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:34.839+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:35.317 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:34 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:34.839+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:35.317 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:34 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:34.839+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:35.317 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:34 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:34.839+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:35.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:35 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:35.154+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:23.248278+0000 front 2024-03-30T00:38:23.248098+0000 (oldest deadline 2024-03-30T00:38:48.548345+0000) 2024-03-30T00:39:35.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:35 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:35.154+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.849828+0000 front 2024-03-30T00:38:33.849580+0000 (oldest deadline 2024-03-30T00:38:57.349573+0000) 2024-03-30T00:39:35.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:35 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:35.154+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:37.350149+0000 front 2024-03-30T00:38:37.350165+0000 (oldest deadline 2024-03-30T00:39:02.650323+0000) 2024-03-30T00:39:35.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:35 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:35.154+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:46.751314+0000 front 2024-03-30T00:38:46.751366+0000 (oldest deadline 2024-03-30T00:39:11.451515+0000) 2024-03-30T00:39:35.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:35 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:35.154+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.152389+0000 front 2024-03-30T00:38:56.152499+0000 (oldest deadline 2024-03-30T00:39:21.452439+0000) 2024-03-30T00:39:35.487 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:35 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9[80301]: 2024-03-30T00:39:35.154+0000 7f3f06200700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:08.354238+0000 front 2024-03-30T00:39:08.354264+0000 (oldest deadline 2024-03-30T00:39:34.254439+0000) 2024-03-30T00:39:36.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:35 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:35.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:36.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:35 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:35.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:36.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:35 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:35.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:36.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:35 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:35.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:36.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:35 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:35.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:36.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:35 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:35.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:36.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:35 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:35.972+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:36.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:35 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:35.972+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:36.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:35 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:35.972+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:36.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:35 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:35.972+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:36.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:35 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:35.972+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:36.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:35 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:35.972+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:36.737 INFO:journalctl@ceph.osd.9.smithi099.stdout:Mar 30 00:39:36 smithi099 bash[108842]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-9 2024-03-30T00:39:37.320 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:36 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:36.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:37.475 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:36 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:36.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:37.476 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:36 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:36.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:37.476 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:36 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:36.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:37.476 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:36 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:36.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:37.476 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:36 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:36.837+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:37.476 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:36 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:36.963+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:37.477 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:36 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:36.963+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:37.477 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:36 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:36.963+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:37.477 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:36 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:36.963+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:37.477 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:36 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:36.963+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:37.477 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:36 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:36.963+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:37.894 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.9.service' 2024-03-30T00:39:38.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:39:38.020 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-30T00:39:38.020 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-30T00:39:38.020 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.10 2024-03-30T00:39:38.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:37 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:37.790+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:38.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:37 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:37.790+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:38.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:37 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:37.790+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:38.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:37 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:37.790+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:38.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:37 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:37.790+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:38.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:37 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:37.790+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:38.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:37 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:37.790+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:12.798396+0000 front 2024-03-30T00:39:12.798355+0000 (oldest deadline 2024-03-30T00:39:36.898253+0000) 2024-03-30T00:39:38.391 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:38 smithi142 systemd[1]: Stopping Ceph osd.10 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:39:38.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:37 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:37.997+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:38.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:37 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:37.997+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:38.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:37 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:37.997+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:38.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:37 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:37.997+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:38.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:37 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:37.997+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:38.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:37 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:37.997+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:38.761 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:38 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:38.384+0000 7f5919600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:39:38.761 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:38 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:38.384+0000 7f5919600700 -1 osd.10 87 *** Got signal Terminated *** 2024-03-30T00:39:38.761 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:38 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:38.384+0000 7f5919600700 -1 osd.10 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:39:38.762 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:38 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:38.754+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:39.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:38 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:38.754+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:39.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:38 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:38.754+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:39.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:38 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:38.754+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:39.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:38 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:38.754+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:39.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:38 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:38.754+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:39.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:38 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:38.754+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:12.798396+0000 front 2024-03-30T00:39:12.798355+0000 (oldest deadline 2024-03-30T00:39:36.898253+0000) 2024-03-30T00:39:39.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:38 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:38.988+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:39.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:38 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:38.988+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:39.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:38 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:38.988+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:39.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:38 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:38.988+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:39.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:38 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:38.988+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:39.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:38 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:38.988+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:40.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:39 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:39.739+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:40.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:39 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:39.739+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:40.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:39 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:39.739+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:40.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:39 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:39.739+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:40.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:39 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:39.739+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:40.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:39 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:39.739+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:40.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:39 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:39.739+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:12.798396+0000 front 2024-03-30T00:39:12.798355+0000 (oldest deadline 2024-03-30T00:39:36.898253+0000) 2024-03-30T00:39:40.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:40 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:40.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:40.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:40 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:40.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:40.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:40 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:40.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:40.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:40 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:40.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:40.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:40 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:40.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:40.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:40 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:40.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:41.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:40 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:40.699+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:41.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:40 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:40.699+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:41.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:40 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:40.699+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:41.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:40 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:40.699+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:41.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:40 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:40.699+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:41.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:40 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:40.699+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:41.067 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:40 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:40.699+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:12.798396+0000 front 2024-03-30T00:39:12.798355+0000 (oldest deadline 2024-03-30T00:39:36.898253+0000) 2024-03-30T00:39:41.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:41 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:41.065+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:41.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:41 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:41.065+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:41.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:41 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:41.065+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:41.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:41 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:41.065+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:41.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:41 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:41.065+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:41.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:41 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:41.065+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:42.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:41 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:41.672+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:42.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:41 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:41.672+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:42.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:41 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:41.672+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:42.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:41 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:41.672+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:42.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:41 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:41.672+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:42.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:41 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:41.672+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:42.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:41 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:41.672+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:12.798396+0000 front 2024-03-30T00:39:12.798355+0000 (oldest deadline 2024-03-30T00:39:36.898253+0000) 2024-03-30T00:39:42.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:42 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:42.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:42.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:42 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:42.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:42.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:42 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:42.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:42.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:42 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:42.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:42.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:42 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:42.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:42.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:42 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:42.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:42.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:42 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:42.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-03-30T00:39:18.872229+0000 front 2024-03-30T00:39:18.872310+0000 (oldest deadline 2024-03-30T00:39:41.172353+0000) 2024-03-30T00:39:43.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:42 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:42.678+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:24.989946+0000 front 2024-03-30T00:38:24.990100+0000 (oldest deadline 2024-03-30T00:38:50.889811+0000) 2024-03-30T00:39:43.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:42 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:42.678+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:31.390879+0000 front 2024-03-30T00:38:31.390865+0000 (oldest deadline 2024-03-30T00:38:57.291155+0000) 2024-03-30T00:39:43.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:42 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:42.678+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:39.592493+0000 front 2024-03-30T00:38:39.592609+0000 (oldest deadline 2024-03-30T00:39:02.492412+0000) 2024-03-30T00:39:43.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:42 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:42.678+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6814 osd.3 since back 2024-03-30T00:38:49.295412+0000 front 2024-03-30T00:38:49.295392+0000 (oldest deadline 2024-03-30T00:39:14.595504+0000) 2024-03-30T00:39:43.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:42 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:42.678+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:54.595951+0000 front 2024-03-30T00:38:54.596019+0000 (oldest deadline 2024-03-30T00:39:19.895988+0000) 2024-03-30T00:39:43.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:42 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:42.678+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:38:59.896393+0000 front 2024-03-30T00:38:59.896407+0000 (oldest deadline 2024-03-30T00:39:26.897036+0000) 2024-03-30T00:39:43.066 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:42 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10[70956]: 2024-03-30T00:39:42.678+0000 7f5915800700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:12.798396+0000 front 2024-03-30T00:39:12.798355+0000 (oldest deadline 2024-03-30T00:39:36.898253+0000) 2024-03-30T00:39:43.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:43 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:43.025+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:43.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:43 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:43.025+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:43.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:43 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:43.025+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:43.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:43 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:43.025+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:43.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:43 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:43.025+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:43.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:43 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:43.025+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:43.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:43 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:43.025+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-03-30T00:39:18.872229+0000 front 2024-03-30T00:39:18.872310+0000 (oldest deadline 2024-03-30T00:39:41.172353+0000) 2024-03-30T00:39:44.316 INFO:journalctl@ceph.osd.10.smithi142.stdout:Mar 30 00:39:44 smithi142 bash[82205]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-10 2024-03-30T00:39:44.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:44 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:44.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:44.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:44 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:44.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:44.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:44 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:44.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:44.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:44 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:44.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:44.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:44 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:44.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:44.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:44 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:44.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:44.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:44 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:44.035+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-03-30T00:39:18.872229+0000 front 2024-03-30T00:39:18.872310+0000 (oldest deadline 2024-03-30T00:39:41.172353+0000) 2024-03-30T00:39:45.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:45 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:45.039+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:45.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:45 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:45.039+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:45.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:45 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:45.039+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:45.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:45 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:45.039+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:45.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:45 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:45.039+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:45.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:45 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:45.039+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:45.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:45 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:45.039+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-03-30T00:39:18.872229+0000 front 2024-03-30T00:39:18.872310+0000 (oldest deadline 2024-03-30T00:39:41.172353+0000) 2024-03-30T00:39:45.597 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.10.service' 2024-03-30T00:39:45.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:39:45.722 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-30T00:39:45.722 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-30T00:39:45.723 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.11 2024-03-30T00:39:46.075 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:45 smithi161 systemd[1]: Stopping Ceph osd.11 for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:39:46.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:46 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:46.068+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:46.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:46 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:46.068+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:46.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:46 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:46.068+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:46.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:46 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:46.068+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:46.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:46 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:46.068+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:46.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:46 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:46.068+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:46.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:46 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:46.068+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-03-30T00:39:18.872229+0000 front 2024-03-30T00:39:18.872310+0000 (oldest deadline 2024-03-30T00:39:41.172353+0000) 2024-03-30T00:39:46.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:46 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:46.091+0000 7fb353e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:39:46.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:46 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:46.091+0000 7fb353e00700 -1 osd.11 87 *** Got signal Terminated *** 2024-03-30T00:39:46.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:46 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:46.091+0000 7fb353e00700 -1 osd.11 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T00:39:47.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:47 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:47.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:47.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:47 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:47.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:47.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:47 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:47.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:47.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:47 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:47.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:47.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:47 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:47.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:47.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:47 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:47.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:47.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:47 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:47.019+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-03-30T00:39:18.872229+0000 front 2024-03-30T00:39:18.872310+0000 (oldest deadline 2024-03-30T00:39:41.172353+0000) 2024-03-30T00:39:48.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:48 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:48.034+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:48.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:48 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:48.034+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:48.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:48 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:48.034+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:48.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:48 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:48.034+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:48.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:48 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:48.034+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:48.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:48 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:48.034+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:48.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:48 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:48.034+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-03-30T00:39:18.872229+0000 front 2024-03-30T00:39:18.872310+0000 (oldest deadline 2024-03-30T00:39:41.172353+0000) 2024-03-30T00:39:49.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:49 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:49.082+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:49.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:49 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:49.082+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:49.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:49 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:49.082+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:49.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:49 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:49.082+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:49.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:49 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:49.082+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:49.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:49 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:49.082+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:49.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:49 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:49.082+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-03-30T00:39:18.872229+0000 front 2024-03-30T00:39:18.872310+0000 (oldest deadline 2024-03-30T00:39:41.172353+0000) 2024-03-30T00:39:50.468 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:50 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:50.048+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:50.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:50 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:50.048+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:50.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:50 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:50.048+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:50.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:50 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:50.048+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:50.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:50 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:50.048+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:50.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:50 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:50.048+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:50.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:50 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:50.048+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-03-30T00:39:18.872229+0000 front 2024-03-30T00:39:18.872310+0000 (oldest deadline 2024-03-30T00:39:41.172353+0000) 2024-03-30T00:39:51.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:51 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:51.037+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6806 osd.0 since back 2024-03-30T00:38:26.661576+0000 front 2024-03-30T00:38:26.661588+0000 (oldest deadline 2024-03-30T00:38:50.161798+0000) 2024-03-30T00:39:51.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:51 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:51.037+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-03-30T00:38:33.663309+0000 front 2024-03-30T00:38:33.663198+0000 (oldest deadline 2024-03-30T00:38:57.163200+0000) 2024-03-30T00:39:51.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:51 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:51.037+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-03-30T00:38:40.564768+0000 front 2024-03-30T00:38:40.564796+0000 (oldest deadline 2024-03-30T00:39:04.064964+0000) 2024-03-30T00:39:51.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:51 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:51.037+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-03-30T00:38:56.168704+0000 front 2024-03-30T00:38:56.168627+0000 (oldest deadline 2024-03-30T00:39:19.668448+0000) 2024-03-30T00:39:51.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:51 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:51.037+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-03-30T00:39:03.769570+0000 front 2024-03-30T00:39:03.769614+0000 (oldest deadline 2024-03-30T00:39:26.669671+0000) 2024-03-30T00:39:51.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:51 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:51.037+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.99:6822 osd.6 since back 2024-03-30T00:39:11.370851+0000 front 2024-03-30T00:39:11.370840+0000 (oldest deadline 2024-03-30T00:39:35.470637+0000) 2024-03-30T00:39:51.469 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:51 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11[72962]: 2024-03-30T00:39:51.037+0000 7fb34b000700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-03-30T00:39:18.872229+0000 front 2024-03-30T00:39:18.872310+0000 (oldest deadline 2024-03-30T00:39:41.172353+0000) 2024-03-30T00:39:52.218 INFO:journalctl@ceph.osd.11.smithi161.stdout:Mar 30 00:39:51 smithi161 bash[84202]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-osd-11 2024-03-30T00:39:53.523 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@osd.11.service' 2024-03-30T00:39:53.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:39:53.608 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-30T00:39:53.608 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-30T00:39:53.608 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.a 2024-03-30T00:39:53.987 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:39:53 smithi099 systemd[1]: Stopping Ceph mds.a for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:39:54.487 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:39:53 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-a[88085]: 2024-03-30T00:39:53.986+0000 7f325fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:39:54.487 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:39:53 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-a[88085]: 2024-03-30T00:39:53.986+0000 7f325fe00700 -1 mds.a *** got signal Terminated *** 2024-03-30T00:39:56.737 INFO:journalctl@ceph.mds.a.smithi099.stdout:Mar 30 00:39:56 smithi099 bash[109103]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-a 2024-03-30T00:39:56.762 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.a.service' 2024-03-30T00:39:56.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:39:56.842 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-30T00:39:56.842 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-30T00:39:56.842 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.d 2024-03-30T00:39:57.189 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 30 00:39:56 smithi099 systemd[1]: Stopping Ceph mds.d for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:39:57.486 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 30 00:39:57 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-d[87108]: 2024-03-30T00:39:57.182+0000 7ff7f6200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:39:57.487 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 30 00:39:57 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-d[87108]: 2024-03-30T00:39:57.182+0000 7ff7f6200700 -1 mds.d *** got signal Terminated *** 2024-03-30T00:40:01.237 INFO:journalctl@ceph.mds.d.smithi099.stdout:Mar 30 00:40:01 smithi099 bash[109242]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-d 2024-03-30T00:40:01.524 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.d.service' 2024-03-30T00:40:01.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:40:01.605 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-30T00:40:01.605 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-30T00:40:01.605 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.g 2024-03-30T00:40:01.947 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 30 00:40:01 smithi099 systemd[1]: Stopping Ceph mds.g for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:40:02.237 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 30 00:40:01 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-g[87715]: 2024-03-30T00:40:01.939+0000 7f6b73c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:40:02.237 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 30 00:40:01 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-g[87715]: 2024-03-30T00:40:01.939+0000 7f6b73c00700 -1 mds.g *** got signal Terminated *** 2024-03-30T00:40:04.237 INFO:journalctl@ceph.mds.g.smithi099.stdout:Mar 30 00:40:03 smithi099 bash[109384]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-g 2024-03-30T00:40:04.405 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.g.service' 2024-03-30T00:40:04.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:40:04.484 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-30T00:40:04.484 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-30T00:40:04.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.j 2024-03-30T00:40:04.737 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 30 00:40:04 smithi099 systemd[1]: Stopping Ceph mds.j for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:40:05.237 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 30 00:40:04 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-j[86741]: 2024-03-30T00:40:04.814+0000 7f948ce00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:40:05.237 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 30 00:40:04 smithi099 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-j[86741]: 2024-03-30T00:40:04.814+0000 7f948ce00700 -1 mds.j *** got signal Terminated *** 2024-03-30T00:40:06.987 INFO:journalctl@ceph.mds.j.smithi099.stdout:Mar 30 00:40:06 smithi099 bash[109524]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-j 2024-03-30T00:40:07.159 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.j.service' 2024-03-30T00:40:07.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:40:07.237 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-30T00:40:07.237 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-30T00:40:07.238 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.b 2024-03-30T00:40:07.567 INFO:journalctl@ceph.mds.b.smithi142.stdout:Mar 30 00:40:07 smithi142 systemd[1]: Stopping Ceph mds.b for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:40:08.066 INFO:journalctl@ceph.mds.b.smithi142.stdout:Mar 30 00:40:07 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-b[77897]: 2024-03-30T00:40:07.588+0000 7f4a60600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:40:08.218 INFO:journalctl@ceph.mds.b.smithi142.stdout:Mar 30 00:40:07 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-b[77897]: 2024-03-30T00:40:07.588+0000 7f4a60600700 -1 mds.b *** got signal Terminated *** 2024-03-30T00:40:17.316 INFO:journalctl@ceph.mds.b.smithi142.stdout:Mar 30 00:40:16 smithi142 bash[82462]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-b 2024-03-30T00:40:17.360 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.b.service' 2024-03-30T00:40:17.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:40:17.440 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-30T00:40:17.440 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-30T00:40:17.441 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.e 2024-03-30T00:40:17.793 INFO:journalctl@ceph.mds.e.smithi142.stdout:Mar 30 00:40:17 smithi142 systemd[1]: Stopping Ceph mds.e for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:40:18.066 INFO:journalctl@ceph.mds.e.smithi142.stdout:Mar 30 00:40:17 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-e[77210]: 2024-03-30T00:40:17.786+0000 7f922b200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:40:18.066 INFO:journalctl@ceph.mds.e.smithi142.stdout:Mar 30 00:40:17 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-e[77210]: 2024-03-30T00:40:17.786+0000 7f922b200700 -1 mds.e *** got signal Terminated *** 2024-03-30T00:40:20.030 INFO:journalctl@ceph.mds.e.smithi142.stdout:Mar 30 00:40:19 smithi142 bash[82600]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-e 2024-03-30T00:40:20.057 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.e.service' 2024-03-30T00:40:20.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:40:20.138 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-30T00:40:20.138 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-30T00:40:20.139 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.h 2024-03-30T00:40:20.316 INFO:journalctl@ceph.mds.h.smithi142.stdout:Mar 30 00:40:20 smithi142 systemd[1]: Stopping Ceph mds.h for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:40:20.816 INFO:journalctl@ceph.mds.h.smithi142.stdout:Mar 30 00:40:20 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-h[77554]: 2024-03-30T00:40:20.487+0000 7fd6c1000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:40:20.816 INFO:journalctl@ceph.mds.h.smithi142.stdout:Mar 30 00:40:20 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-h[77554]: 2024-03-30T00:40:20.487+0000 7fd6c1000700 -1 mds.h *** got signal Terminated *** 2024-03-30T00:40:23.316 INFO:journalctl@ceph.mds.h.smithi142.stdout:Mar 30 00:40:23 smithi142 bash[82746]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-h 2024-03-30T00:40:23.468 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.h.service' 2024-03-30T00:40:23.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:40:23.547 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-30T00:40:23.547 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-30T00:40:23.547 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.k 2024-03-30T00:40:23.816 INFO:journalctl@ceph.mds.k.smithi142.stdout:Mar 30 00:40:23 smithi142 systemd[1]: Stopping Ceph mds.k for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:40:24.316 INFO:journalctl@ceph.mds.k.smithi142.stdout:Mar 30 00:40:23 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-k[76866]: 2024-03-30T00:40:23.894+0000 7f9765400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:40:24.316 INFO:journalctl@ceph.mds.k.smithi142.stdout:Mar 30 00:40:23 smithi142 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-k[76866]: 2024-03-30T00:40:23.894+0000 7f9765400700 -1 mds.k *** got signal Terminated *** 2024-03-30T00:40:28.737 INFO:journalctl@ceph.mds.k.smithi142.stdout:Mar 30 00:40:28 smithi142 bash[82885]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-k 2024-03-30T00:40:28.764 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.k.service' 2024-03-30T00:40:28.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:40:28.842 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-30T00:40:28.842 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-30T00:40:28.842 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.c 2024-03-30T00:40:29.217 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 30 00:40:28 smithi161 systemd[1]: Stopping Ceph mds.c for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:40:29.468 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 30 00:40:29 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-c[79580]: 2024-03-30T00:40:29.209+0000 7fd78cc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:40:29.468 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 30 00:40:29 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-c[79580]: 2024-03-30T00:40:29.209+0000 7fd78cc00700 -1 mds.c *** got signal Terminated *** 2024-03-30T00:40:32.956 INFO:journalctl@ceph.mds.c.smithi161.stdout:Mar 30 00:40:32 smithi161 bash[84464]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-c 2024-03-30T00:40:32.991 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.c.service' 2024-03-30T00:40:33.076 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:40:33.076 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-30T00:40:33.076 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-30T00:40:33.076 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.f 2024-03-30T00:40:33.440 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 30 00:40:33 smithi161 systemd[1]: Stopping Ceph mds.f for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:40:33.718 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 30 00:40:33 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-f[79238]: 2024-03-30T00:40:33.433+0000 7fea94000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:40:33.718 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 30 00:40:33 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-f[79238]: 2024-03-30T00:40:33.433+0000 7fea94000700 -1 mds.f *** got signal Terminated *** 2024-03-30T00:40:37.718 INFO:journalctl@ceph.mds.f.smithi161.stdout:Mar 30 00:40:37 smithi161 bash[84611]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-f 2024-03-30T00:40:37.788 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.f.service' 2024-03-30T00:40:37.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:40:37.867 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-30T00:40:37.867 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-30T00:40:37.868 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.i 2024-03-30T00:40:38.218 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 30 00:40:37 smithi161 systemd[1]: Stopping Ceph mds.i for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:40:38.718 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 30 00:40:38 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-i[79923]: 2024-03-30T00:40:38.215+0000 7f6463000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:40:38.718 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 30 00:40:38 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-i[79923]: 2024-03-30T00:40:38.215+0000 7f6463000700 -1 mds.i *** got signal Terminated *** 2024-03-30T00:40:41.968 INFO:journalctl@ceph.mds.i.smithi161.stdout:Mar 30 00:40:41 smithi161 bash[84748]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-i 2024-03-30T00:40:42.092 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.i.service' 2024-03-30T00:40:42.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:40:42.169 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-30T00:40:42.169 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-30T00:40:42.169 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.l 2024-03-30T00:40:42.468 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 30 00:40:42 smithi161 systemd[1]: Stopping Ceph mds.l for 03801b1a-ee2c-11ee-b647-cb9ed24678a4... 2024-03-30T00:40:42.968 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 30 00:40:42 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-l[78898]: 2024-03-30T00:40:42.548+0000 7f33ed000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T00:40:42.968 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 30 00:40:42 smithi161 ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-l[78898]: 2024-03-30T00:40:42.548+0000 7f33ed000700 -1 mds.l *** got signal Terminated *** 2024-03-30T00:40:46.419 INFO:journalctl@ceph.mds.l.smithi161.stdout:Mar 30 00:40:46 smithi161 bash[84886]: ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4-mds-l 2024-03-30T00:40:46.442 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-03801b1a-ee2c-11ee-b647-cb9ed24678a4@mds.l.service' 2024-03-30T00:40:46.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T00:40:46.520 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-30T00:40:46.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-03-30T00:40:46.781 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:40:49.652 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-03-30T00:40:49.909 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:40:52.711 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-03-30T00:40:52.969 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:40:55.815 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T00:40:55.910 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T00:40:55.986 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T00:40:56.055 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-30T00:40:56.057 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224/remote/smithi099/crash 2024-03-30T00:40:56.058 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/crash -- . 2024-03-30T00:40:56.136 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-03-30T00:40:56.136 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2024-03-30T00:40:56.142 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224/remote/smithi142/crash 2024-03-30T00:40:56.142 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/crash -- . 2024-03-30T00:40:56.234 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-03-30T00:40:56.235 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-03-30T00:40:56.238 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224/remote/smithi161/crash 2024-03-30T00:40:56.239 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/crash -- . 2024-03-30T00:40:56.349 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-03-30T00:40:56.349 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-03-30T00:40:56.352 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-30T00:40:56.352 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-30T00:40:56.427 INFO:teuthology.orchestra.run.smithi099.stderr:grep: /var/log/ceph/03801b1a-ee2c-11ee-b647-cb9ed24678a4/ceph.log: No such file or directory 2024-03-30T00:40:56.433 INFO:tasks.cephadm:Compressing logs... 2024-03-30T00:40:56.433 DEBUG:teuthology.orchestra.run.smithi099:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T00:40:56.437 DEBUG:teuthology.orchestra.run.smithi142:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T00:40:56.441 DEBUG:teuthology.orchestra.run.smithi161:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T00:40:56.497 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-30T00:40:56.503 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-30T00:40:56.505 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-30T00:41:15.189 INFO:tasks.cephadm:Archiving logs... 2024-03-30T00:41:15.190 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224/remote/smithi099/log 2024-03-30T00:41:15.191 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T00:41:17.193 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224/remote/smithi142/log 2024-03-30T00:41:17.194 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T00:41:18.349 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224/remote/smithi161/log 2024-03-30T00:41:18.350 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T00:41:19.510 INFO:tasks.cephadm:Removing cluster... 2024-03-30T00:41:19.511 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 --force 2024-03-30T00:41:19.770 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:41:20.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 --force 2024-03-30T00:41:20.491 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:41:20.955 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 03801b1a-ee2c-11ee-b647-cb9ed24678a4 --force 2024-03-30T00:41:21.211 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 03801b1a-ee2c-11ee-b647-cb9ed24678a4 2024-03-30T00:41:21.678 INFO:tasks.cephadm:Removing cephadm ... 2024-03-30T00:41:21.679 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-30T00:41:21.726 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-30T00:41:21.771 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-30T00:41:21.812 INFO:tasks.cephadm:Teardown complete 2024-03-30T00:41:21.813 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-30T00:41:21.824 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-30T00:41:21.824 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-30T00:41:21.825 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T00:41:21.829 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T00:41:21.834 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T00:41:21.982 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-30T00:41:21.992 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-30T00:41:21.993 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T00:41:21.997 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T00:41:22.001 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T00:41:22.033 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2024-03-30T00:41:22.037 INFO:teuthology.orchestra.run.smithi142.stderr:bash: ntpq: command not found 2024-03-30T00:41:22.041 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2024-03-30T00:41:22.041 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T00:41:22.041 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-03-30T00:41:22.041 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 97 +136us[ +123us] +/- 75ms 2024-03-30T00:41:22.041 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv02.front.sepia.ceph.com 3 7 377 95 +846us[ +846us] +/- 53ms 2024-03-30T00:41:22.041 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -1232us[-1232us] +/- 66ms 2024-03-30T00:41:22.042 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:41:22.044 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T00:41:22.044 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-03-30T00:41:22.044 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 +165us[ +165us] +/- 76ms 2024-03-30T00:41:22.044 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv02.front.sepia.ceph.com 3 6 377 31 +831us[ +831us] +/- 54ms 2024-03-30T00:41:22.044 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -1193us[-1212us] +/- 66ms 2024-03-30T00:41:22.044 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:41:22.048 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T00:41:22.049 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-03-30T00:41:22.049 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +164us[ +164us] +/- 76ms 2024-03-30T00:41:22.049 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 +822us[ +822us] +/- 54ms 2024-03-30T00:41:22.049 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -1135us[-1142us] +/- 66ms 2024-03-30T00:41:22.049 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T00:41:22.050 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-30T00:41:22.060 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-30T00:41:22.060 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-30T00:41:22.071 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-30T00:41:22.217 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-30T00:41:22.343 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-30T00:41:22.472 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T00:41:22.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:41:22.541 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T00:41:22.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:41:22.610 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T00:41:22.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:41:22.679 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-30T00:41:22.693 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-30T00:41:22.703 INFO:teuthology.task.internal:Duration was 1817.123745 seconds 2024-03-30T00:41:22.703 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-30T00:41:22.714 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-30T00:41:22.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T00:41:22.718 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T00:41:22.722 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T00:41:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T00:41:22.830 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T00:41:22.836 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T00:41:23.270 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-30T00:41:23.270 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-03-30T00:41:23.270 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T00:41:23.327 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-03-30T00:41:23.327 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T00:41:23.384 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-03-30T00:41:23.384 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T00:41:23.441 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-30T00:41:23.442 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T00:41:23.446 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T00:41:23.450 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T00:41:23.542 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-30T00:41:23.543 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T00:41:23.547 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T00:41:23.552 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T00:41:23.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-30T00:41:23.708 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-30T00:41:23.709 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T00:41:23.774 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T00:41:23.839 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T00:41:23.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-30T00:41:23.918 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T00:41:23.922 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T00:41:23.926 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T00:41:23.982 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-03-30T00:41:23.986 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-03-30T00:41:23.990 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-03-30T00:41:24.047 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T00:41:24.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:41:24.087 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T00:41:24.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:41:24.128 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T00:41:24.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T00:41:24.167 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-30T00:41:24.179 INFO:teuthology.task.internal:Transferring archived files... 2024-03-30T00:41:24.180 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224/remote/smithi099 2024-03-30T00:41:24.180 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T00:41:24.262 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224/remote/smithi142 2024-03-30T00:41:24.263 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T00:41:24.349 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224/remote/smithi161 2024-03-30T00:41:24.349 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T00:41:24.434 INFO:teuthology.task.internal:Removing archive directory... 2024-03-30T00:41:24.434 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T00:41:24.439 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T00:41:24.444 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T00:41:24.486 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-30T00:41:24.507 INFO:teuthology.task.internal:Not uploading archives. 2024-03-30T00:41:24.507 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-30T00:41:24.527 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-30T00:41:24.527 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T00:41:24.532 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T00:41:24.536 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T00:41:24.569 INFO:teuthology.orchestra.run.smithi099.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 00:41 /home/ubuntu/cephtest 2024-03-30T00:41:24.574 INFO:teuthology.orchestra.run.smithi142.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 00:41 /home/ubuntu/cephtest 2024-03-30T00:41:24.578 INFO:teuthology.orchestra.run.smithi161.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 00:41 /home/ubuntu/cephtest 2024-03-30T00:41:24.582 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-30T00:41:24.593 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-30T00:41:24.606 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-30T00:41:24.606 INFO:teuthology.nuke:Checking targets against current locks 2024-03-30T00:41:24.626 DEBUG:teuthology.nuke:shortname: smithi099 2024-03-30T00:41:24.630 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T00:41:24.653 DEBUG:teuthology.nuke:shortname: smithi142 2024-03-30T00:41:24.653 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T00:41:24.658 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 23:59:04.700389', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIoSjW3CRxe29HNsWJoUM5ocmpfMDfRmVCCK08neLVO2axhrGNYo0sBlmCg3K89B+9r0wgIXYaUBhxXcjs2K9ao='} 2024-03-30T00:41:24.674 DEBUG:teuthology.nuke:shortname: smithi161 2024-03-30T00:41:24.674 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T00:41:24.677 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 23:59:04.698954', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDjqJ1yuF+P8cmOP1ktVk8gvV+4ub9nveBx1TeiCA1+pjAR2qF9IPIo7qExVfYfgjINT5fjScaW1x0hAX6I6KHM='} 2024-03-30T00:41:24.696 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629224', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 23:59:04.701442', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBObwWGzVz+IdEJn2UBpojh+5QUNYR2pvwgk7/PzEjR44QcnwvA7Vm2rn3m0tcg3TJF587MZ6TCyINXYgp6ySLfE='} 2024-03-30T00:41:24.709 INFO:teuthology.orchestra.console.smithi099:Power off 2024-03-30T00:41:24.709 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T00:41:24.731 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2024-03-30T00:41:24.732 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:41:24.746 INFO:teuthology.orchestra.console.smithi142:Power off 2024-03-30T00:41:24.746 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T00:41:24.762 INFO:teuthology.orchestra.console.smithi161:Power off 2024-03-30T00:41:24.762 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T00:41:24.771 DEBUG:teuthology.orchestra.console.smithi142:power off output: Chassis Power Control: Down/Off 2024-03-30T00:41:24.771 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:41:24.781 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2024-03-30T00:41:24.784 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off 2024-03-30T00:41:24.784 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:41:24.796 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-03-30T00:41:24.806 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-03-30T00:41:28.782 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:41:28.797 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:41:28.807 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:41:28.892 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2024-03-30T00:41:28.907 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-03-30T00:41:28.916 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-03-30T00:41:32.898 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:41:32.908 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:41:32.917 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:41:33.009 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2024-03-30T00:41:33.019 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-03-30T00:41:33.028 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-03-30T00:41:33.110 INFO:teuthology.orchestra.console.smithi099:Power off completed 2024-03-30T00:41:37.024 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:41:37.032 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T00:41:37.135 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2024-03-30T00:41:37.142 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2024-03-30T00:41:37.237 INFO:teuthology.orchestra.console.smithi142:Power off completed 2024-03-30T00:41:37.243 INFO:teuthology.orchestra.console.smithi161:Power off completed 2024-03-30T00:41:37.349 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} duration: 1817.1237449645996 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default mon.a-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.b-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.c-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 owner: scheduled_rishabh@teuthology status: fail success: false 2024-03-30T00:41:37.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-30T00:41:37.427 INFO:teuthology.run:FAIL